Identity/Metrics

Last updated: 2014/02/27

Firefox Accounts

User Stories

Metrics/Monitoring requirements/priorities as driven by user stories

• all metrics story bugs in bugzilla
• Nutshell Summary written for a Feb 2014 meeting

Infrastructure

1. general query access to nginx, server log files (via kibana)
2. facility for writing custom alerts, monitors (via heka/lua)

Metrics/Growth

1. One Metric That Matters! Total number of Firefox Accounts : https://bugzilla.mozilla.org/show_bug.cgi?id=972105
2. Accounts created per day : https://bugzilla.mozilla.org/show_bug.cgi?id=968973
   • Higher priority segmentations: https://bugzilla.mozilla.org/show_bug.cgi?id=968990
   • Lower priority segmentatons: https://bugzilla.mozilla.org/show_bug.cgi?id=969003
   • We'll get other auth server endpoints as well, in particular account/login and cert/sign
3. Active daily users : https://bugzilla.mozilla.org/show_bug.cgi?id=968983
4. UX flow analysis : https://bugzilla.mozilla.org/show_bug.cgi?id=969007 (lower priority)

Monitoring/Health

1. Users failing to create/sign in to service : https://bugzilla.mozilla.org/show_bug.cgi?id=974130
2. Timing information for create/login
3. Email queue and latency
4. mem/cpu/disk space/concurrent connections
5. # hitting max failed password attempts : https://bugzilla.mozilla.org/show_bug.cgi?id=977329

Fraud/Security

1. observe/alert mass account creation
2. observe/alert large # of brute force attempts (also observing any that succeed)

Dev/Debug

1. monitor V8 heap stats

Timeline

• Svcops Timeline Planning

Resources

• Notes from meetings with svcops: https://id.etherpad.mozilla.org/fxa-metrics-monitoring
• PM/UX related user stories: https://id.etherpad.mozilla.org/fxacct-metrics-user-stories
• QA/SE Monitoring user stories: https://id.etherpad.mozilla.org/fxa-monitoring-user-stories
• OpSec user stories: # https://mana.mozilla.org/wiki/display/SECURITY/Firefox+Accounts+Security+Design+Review#FirefoxAccountsSecurityDesignReview-SecurityConcepts
• Shared Services : Describes shared services for heka/elasticsearch/kibana deployments. FxA will likely use its own deployment, based on this one.
• FHR work happening separately (Desktop: https://bugzilla.mozilla.org/show_bug.cgi?id=958561, Android: https://bugzilla.mozilla.org/show_bug.cgi?id=891600)
• https://wiki.mozilla.org/ADI
• Minimum Viable Metrics : The original design for minimum set of metrics required before going live with real accounts. Original plan was WMF & Marketplace on FxOs, Sync actually went first (Desktop & Android).

Older Notes

• https://id.etherpad.mozilla.org/fxacct-metrics
• https://id.etherpad.mozilla.org/fxacct-mvm
• https://id.etherpad.mozilla.org/fxacct-mvm-details
• https://id.etherpad.mozilla.org/fxacct-metrics-fraud-detection

Persona

• Persona Metrics