Identity/BrowserID/KPI Dashboard

From MozillaWiki
Jump to: navigation, search

KPI Dashboard

A set of visualizations for UX and Product to evaluate how well we're doing with improving user's experience of BrowserID.

Architecture

KPI Dashboard architecure.jpg

There are two main components, KPI Backend and KPI Frontend.

See https://wiki.mozilla.org/Privacy/Reviews/KPI_Backend

Data Glossary

During the users interaction with the dialog we capture various information:

  • timestamp: the time that the interaction started, rounded off to 10 minute intervals
  • event_stream: interesting events that occurred during the user's interaction, including both events initiated by the user (mouse clicks) as well as events originating from running javascript code (keypair generation). Each event is uniquely named and includes a time offset for when it occurred measured from when the dialog was displayed
  • email_type: In the event that the interaction results in the user selecting an email address to use to sign in, we include the type of email used: "primary" is an email address from a domain that has browserid support, "secondary" is an email address from a domain that does not directly support browserid
  • number_emails: When an interaction proceeds to the point where the user authenticates to the personaid service we include the number of emails that this user has verified with BrowserID
  • new_account: If during an interaction a new browserid account is created, this property is true (as opposed to an interaction which represents sign in using an existing account)
  • lang: The language that was displayed to this user during the interaction
  • number_sites_remembered: If the user is authenticated to the Persona servers at any point during the interaction, we include the number of distinct sites that the user has logged into recently using browserid.
  • number_sites_signed_in: A subset of "number_sites_remembered" is "number_sites_signed_in". If a site is remembered and the user is signed in at that site, when the user visits the site an assertion will be generated automatically. If a site is remembered but the user is not signed in at that site, an assertion will not be generated.
  • screen_size: The screen dimensions of the device used by the user, determined programatically with javascript
  • orphaned: If the dialog is opened but the user never logs into a site, the dialog is considered orphaned.
  • sample_rate: Rate at which the server is sampling clients data for KPI messages. 0.1 would be a 10% sample rate. We plan on shipping with 100% traffic, so 1.0
  • user_agent A generalized version of user agent which includes coarse grained details for Operating System, Browser, and Browser version. Does not contain original user agent string. Implementation will prefer non-fingerprintable values over technical accuracy (okay to encode 'unknown' for Operating System or 'Windows' if reg doesn't match a given version of Windows NT). No Operating system version is collected. Browser version will be as course grained as possible, while remaining useful. Examples: Firefox 13, Chrome 13, Safari 5.1 - no build number or other fine grained value is recorded.

Codebase

Environments

Environment Test Link Persona Login Backend Dashboard
Development http://dev.123done.org https://dev.diresworb.org https://kpiggybank.hacksign.in/wsapi/interaction_data http://dashboard.personatest.org/
Stage http://beta.123done.org/ https://diresworb.org TBD TBD
Production http://123done.org/ https://browserid.org TBD TBD

Domain Name

This lives on the BrowserID server, so no new public domain names.

New internal sub-domains for the backend and Dashboard will be created.

Project Management

Team

  • Crystal Beasley - UX
  • Dan Mills - Product
  • Annie Elliot - Metrics
  • Raymond Forbes - Security
  • Alina Hua - Privacy
  • Katie Parlante, Shane Tomlinson - Development
  • Gene Wood - Services
  • John Morrison - QA

Schedule

Phase I Backend

  • 4/23 - Security Review KPI Backend
  • 4/25 TBD - Data Safety Meeting
  • 4/25 - 5/9 Document one KPI
  • 4/25 - 5/9 Dev sprint for KPI Backend
  • 5/9 - 5/23 Document all KPIs

Phase II Dashboard

  • 5/9 - 6/30 Dev sprint for "implement first cut visualization of one KPI"
  • 5/23 - 6/5 - Stage Env for Backend
  • 5/23 - 7/6 Dev sprint for "implement visualization of all KPIs"
  • 7/7 - 7/20 Dev sprint for Dashboard polish
  •  ? - Collect test data
  •  ? - Prod Env for Backend
  •  ? - Security Review KPI Dashboard
  •  ? - Legal & Privacy Review KPI Dashboard
  •  ? - Stage Env for Frontend
  •  ? - Prod Env for Frontend
  •  ? - Deploy Backend and Collect data
  •  ? - Deploy Dashboard to display data
  • 8/24 - nmalkin's Internship ends

Bugs or Forms

Backend

Frontend

Get Involved

Retrieved from "https://wiki.mozilla.org/index.php?title=Identity/BrowserID/KPI_Dashboard&oldid=519932"