A set of visualizations for UX and Product to evaluate how well we're doing with improving user's experience of BrowserID.
There are two main components, KPI Backend and KPI Frontend.
During the users interaction with the dialog we capture various information:
- timestamp: the time that the interaction started, rounded off to 10 minute intervals
- email_type: In the event that the interaction results in the user selecting an email address to use to sign in, we include the type of email used: "primary" is an email address from a domain that has browserid support, "secondary" is an email address from a domain that does not directly support browserid
- number_emails: When an interaction proceeds to the point where the user authenticates to the personaid service we include the number of emails that this user has verified with BrowserID
- new_account: If during an interaction a new browserid account is created, this property is true (as opposed to an interaction which represents sign in using an existing account)
- lang: The language that was displayed to this user during the interaction
- number_sites_remembered: If the user is authenticated to the Persona servers at any point during the interaction, we include the number of distinct sites that the user has logged into recently using browserid.
- number_sites_signed_in: A subset of "number_sites_remembered" is "number_sites_signed_in". If a site is remembered and the user is signed in at that site, when the user visits the site an assertion will be generated automatically. If a site is remembered but the user is not signed in at that site, an assertion will not be generated.
- orphaned: If the dialog is opened but the user never logs into a site, the dialog is considered orphaned.
- sample_rate: Rate at which the server is sampling clients data for KPI messages. 0.1 would be a 10% sample rate. We plan on shipping with 100% traffic, so 1.0
- user_agent A generalized version of user agent which includes coarse grained details for Operating System, Browser, and Browser version. Does not contain original user agent string. Implementation will prefer non-fingerprintable values over technical accuracy (okay to encode 'unknown' for Operating System or 'Windows' if reg doesn't match a given version of Windows NT). No Operating system version is collected. Browser version will be as course grained as possible, while remaining useful. Examples: Firefox 13, Chrome 13, Safari 5.1 - no build number or other fine grained value is recorded.
- BrowserID collects the data
- KPIggyBank is the data store
- KPI Dashboard is the frontend which will be behind VPN
|Environment||Test Link||Persona Login||Backend||Dashboard|
This lives on the BrowserID server, so no new public domain names.
New internal sub-domains for the backend and Dashboard will be created.
- Crystal Beasley - UX
- Dan Mills - Product
- Annie Elliot - Metrics
- Raymond Forbes - Security
- Alina Hua - Privacy
- Katie Parlante, Shane Tomlinson - Development
- Gene Wood - Services
- John Morrison - QA
Phase I Backend
- 4/23 - Security Review KPI Backend
4/25TBD - Data Safety Meeting
- 4/25 - 5/9 Document one KPI
- 4/25 - 5/9 Dev sprint for KPI Backend
- 5/9 - 5/23 Document all KPIs
Phase II Dashboard
- 5/9 - 6/30 Dev sprint for "implement first cut visualization of one KPI"
- 5/23 - 6/5 - Stage Env for Backend
- 5/23 - 7/6 Dev sprint for "implement visualization of all KPIs"
- 7/7 - 7/20 Dev sprint for Dashboard polish
- ? - Collect test data
- ? - Prod Env for Backend
- ? - Security Review KPI Dashboard
- ? - Legal & Privacy Review KPI Dashboard
- ? - Stage Env for Frontend
- ? - Prod Env for Frontend
- ? - Deploy Backend and Collect data
- ? - Deploy Dashboard to display data
- 8/24 - nmalkin's Internship ends
Bugs or Forms
- bug 742796 - Security Review for Backend
- bug 746245 - Privacy/Legal Review for Backend
- KPI Data Saftey Review Etherpad
- Privacy Review Questions Etherpad
- Privacy Review
- bug 773400 - Tracking bug
- bug 773404 - Security review KPI Dashboard
- bug 773399 - Legal review KPI Dashboard
- bug 746231 - First KPI Visualization
- bug 746233 - All Remaining KPI Visualizations
- instrument site in preparation for a/b testing of UX experiments