TestEngineering/Services/FxA Sync QoS

Quality of Service Goals

Firefox Accounts

• 98% uptime
  • Defined as the all users ability to create or login to Firefox Accounts
• Less than three priority P1 bugs in the first month of deployment
  • includes inability to sign in for large number (>10%) of users for more than 2 hours.

Sync 1.5

• 95% uptime
  • Sync is a background service and short term downtime is within spec of the feature.
  • Defined as the all users ability to sync data
• Less than three priority P1 bugs in the first month of deployment
  • includes inability to sync for large number (>10%) of users for a significant for more than 24 hrs.

QoS metrics

4/29 - 5/6

• Uptime:
  • 99.25% (1.25 hrs of down time due to only create account impact)

• Response Time
  • Login and Create Response Time: 200-300ms
  • No Regressions
  • Current problems with dashboard DB making difficult to extract exact numbers

• Error and Status Responses
  • 98% success response (0.3% 5xx responses)
  • email latency: less than 3 sec, no reported drops other than outages

• Issues:
  • email outages prevent users from verifying account and reset password.
  • 30 min email verification delivery failure due to send limit incorrectly set
  • 2 - one hour SES amazon delivery failures for entire region

• P1 Issues: None

• General Issues:
  • one report of sync failure, investigating: https://bugzilla.mozilla.org/show_bug.cgi?id=1006360
  • No Sync or FxA availability issues as we approach 500k total accounts.

• Quality Influence:
  • Able to drive monitoring and alerting goals to track Quality of Service
  • Identify Fraud and Abuse monitoring requirements for Deployment
  • Deployed 11 trains to date, many hot patches