Firefox/Accounts/Progress Report/2017Q3: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Added objectives 2 and 3, and their respective key results.)
 
(completed updates for objective 3)
 
(27 intermediate revisions by the same user not shown)
Line 1: Line 1:
__NOTOC__
__NOTOC__
<big><big>'''June 28, 2017'''</big></big>
<big><big>'''September 25, 2017'''</big></big>


== OKR Progress ==
== Q3 OKR Progress ==


<!--Should explain WHY for the health/status reported above.  Items covered here may be covered in more detail further down in the report.  If senior management were to look at nothing but this section, they should be able to tell what is going on with the project.-->
<!--Should explain WHY for the health/status reported above.  Items covered here may be covered in more detail further down in the report.  If senior management were to look at nothing but this section, they should be able to tell what is going on with the project.-->
Line 13: Line 13:
|rowspan=2; style="text-align: center; background-color: #f9d9a8"| '''Notes'''
|rowspan=2; style="text-align: center; background-color: #f9d9a8"| '''Notes'''
|-
|-
|style="text-align: center; background-color: #f9d9a8"|Apr 4
|style="text-align: center; background-color: #f9d9a8"|Jul 10
|style="text-align: center; background-color: #f9d9a8"|Apr 18
|style="text-align: center; background-color: #f9d9a8"|Jul 24
|style="text-align: center; background-color: #f9d9a8"|May 2
|style="text-align: center; background-color: #f9d9a8"|Aug 8
|style="text-align: center; background-color: #f9d9a8"|May 15
|style="text-align: center; background-color: #f9d9a8"|Aug 21
|style="text-align: center; background-color: #f9d9a8"|May 30
|style="text-align: center; background-color: #f9d9a8"|Sep 5
|style="text-align: center; background-color: #f9d9a8"|June 12
|style="text-align: center; background-color: #f9d9a8"|Sep 18
|style="text-align: center; background-color: #f9d9a8"|Score
|style="text-align: center; background-color: #f9d9a8"|Score
|-
|-
|rowspan=2| 1. To enable webextension developers to provide value with Firefox Accounts on desktop.
|rowspan=4| 1. To have improved the quality for the Firefox Account experience on desktop.
| align="left"|1.1 To have created 1 mvp of a webextension API for FxA on desktop.  
| align="left"|1.1 Close ALL (100%) issues in the Quantum/Photon milestone. (Crouton)
|align="center"|60%
|align="center"|65%
|align="center"|70%
|align="center"|75%
|align="center"|90%
|align="center"|100%
|align="center"|1.0
|
Priorities for support of Quantum/Photon in 57:
*[https://waffle.io/mozilla/fxa?milestone=FxA-130:%20Crouton%20Flow Crouton Flow Issues for 57 release.]
|
*Done!
*[https://waffle.io/mozilla/fxa/metrics/burndown?milestone=FxA-130:%20Crouton%20Flow Crouton Flow Burndown Chart]
|-
|align="left"| 1.2 Close 50% of our Q3 quality backlog.
|align="center"|
|align="center"|50%
|align="center"|50%
|align="center"|50%
|align="center"|60%
|align="center"|70%
|align="center"|75%
|align="center"|75%
|align="center"|80%
|align="center"|75%
|align="center"|1
|align="center"|90%
|align="left"|
|align="center"|100%
Web Extension API for FxA on Desktop
|align="center"|1.0
*[https://github.com/mozilla/fxa-features/issues/121 Issue 121: OAuth Support for Web Extensions]  
|
Quality Priorities for Q3:
*[https://waffle.io/mozilla/fxa?milestone=FxA-131:%20Q3%20 Quality related P1s that we’ve identified to fix in Q3.]
|
|
*Will ship what we can for Hoverpad. Implementing Auth Server PKCE, the "Proof Key for Code Exchange" spec **[https://openid.net/2015/05/26/enhancing-oauth-security-for-mobile-applications-with-pkse/ Enhancing OAuth Security for Mobile Applications with PKCE]
*Done!
**[https://www.authlete.com/documents/article/pkce/index Proof Key for Code Exchange (RFC 7636)]
*Completed 13 out of 22, or 59%.
**[https://github.com/mozilla/fxa-oauth-server/pull/466/files#r121266395 Issue 466 - feat(pkce): add PKCE support to the oauth server]
*[https://waffle.io/mozilla/fxa/metrics/burndown?milestone=FxA-131:%20Q3%20Quality Quality P1s Burndown]
**[https://github.com/mozilla/fxa-oauth-server/issues/465 Issue 465 - Implement PKCE to support public clients using the code grant flow]
**[https://github.com/mozilla/fxa-content-server/pull/5126 Issue 5126 - feat(oauth): support PKCE oauth parameters]
|-
|-
|align="left"| 1.2 To have at least 1 internal project consuming our API on desktop.
|align="left"| 1.3 Increase by 15% the rate of users that become multi-device as a result of going through our first run flow.
|align="center"|50%
|align="center"|50%
|align="center"|50%
|align="center"|50%
|align="center"|60%  
|align="center"|50%
|align="center"|65%
|align="center"|40%
|align="center"|70%
|align="center"|0%
|align="center"|80%
|align="center"|0%
|align="center"|1
|align="center"|0.0
|
*Supports the onboarding team, marketing retention team and Mozilla.org team to improve user retention (through converting first run users to multi-device users).
*[https://sql.telemetry.mozilla.org/dashboard/fxa-q3-okr-1-3 Dashboards related to user retention]
*[https://github.com/mozilla/fxa-content-server/pull/5332 Issue 5332: ABC test for CAD phase 3 (deep link)]
|
*Didn't achieve this. Had a lot of questions about the metrics.
|-
|align="left"| 1.4 Fix newsletter subscription when email is confirmed on a 2nd device.
|align="center"|95%
|align="center"|95%
|align="center"|95%
|align="center"|95%
|align="center"|90%
|align="center"|90%
|align="center"|1.0
|
|
Hoverpad or ScreenShot integration
Fixes a problem with getting marketing newsletter opt-in data over to Salesforce.
|
|
*Actively collaborating.
*Done!
*Plan to meet with Screenshots team at SF Work Week.
|-
|-
|rowspan=2| 2. To have improved the quality of the Firefox Account experience on mobile.
|rowspan=2| 2. To have improved the quality of the Firefox Account experience on mobile.
   
   
| align="left"|2.1 Measure and A/B test best push opt-in screen on iOS.  
| align="left"|2.1 Measure and A/B test best push opt-in screen on iOS.  
|align="center"|50%
|align="center"|50%
|align="center"|
|align="center"|40%
|align="center"|
|align="center"|55%
|align="center"|
|align="center"|55%
|align="center"|
|align="center"|50%
|align="center"|
|align="center"|50%
|align="center"|
|align="center"|0.7
|rowspan=2; align="left"|
|align="left"|
*[https://docs.google.com/spreadsheets/d/1tPE4KSNd_RU-ahSAK7a5hMcP5EAKGMlHJHegna87xLg/edit#gid=0 FxA Sync prioritization backlog for iOS]
*[https://docs.google.com/spreadsheets/d/1tPE4KSNd_RU-ahSAK7a5hMcP5EAKGMlHJHegna87xLg/edit#gid=0 FxA Sync prioritization backlog for iOS]
*[https://mozilla.invisionapp.com/share/76CPM7AC2#/screens/240561291_Desktop-_Connect_Another_Device Proposed mock up]
|
|
*Still need to figure out what caused the numbers to go down on May 24th.
*PR reviewed but not merged.
*Good metrics were the result of users being incorrectly identified as being in the experiment.
*Review brought up some interesting points to address.
*[https://groups.google.com/a/mozilla.com/forum/?hl=en#!topic/fxa-core/Dg3HPIDItrY Current working theory documented in an email thread after confidence level was estimated]:
|-
|-
| align="left"|2.2 Complete X% of Q3 mobile bugs.
| align="left"|2.2 Complete 80% of Q3 mobile bugs.
|align="center"|X%
|align="center"|N/A
|align="center"|
|align="center"|60%
|align="center"|
|align="center"|60%
|align="center"|
|align="center"|60%
|align="center"|
|align="center"|65%
|align="center"|
|align="center"|90%
|align="center"|
|align="center"|0.8
|align="left"|  
|align="left"|  
*Need to finalize the list.
Identification and fixing of the most impactful bugs on iOS
|
*8 out of 13 are done - 80% of 80% done as of 9/25
*[https://waffle.io/mozilla/fxa/metrics/burndown?milestone=FxA-132:%20Q3%20Mobile Mobile Quality Burndown Chart]
|-
|-
|rowspan=4| 3. To have completed all features that are still in flight from Q1 and Q2.
|rowspan=4| 3. To have completed all features that are still in flight from Q1 and Q2.
| align="left"|3.1 Have a device pairing flow ready to A/B test.  
| align="left"|3.1 Have a device pairing flow ready to A/B test.  
|align="center"|75%
|align="center"|75%
|align="center"|
|align="center"|75%
|align="center"|  
|align="center"|50%
|align="center"|
|align="center"|25%
|align="center"|
|align="center"|0%
|align="center"|
|align="center"|0%
|align="center"|
|align="center"|0.3
|
|align="left"|
|
*Allow users to test out device pairing. This does not necessarily mean that QR codes are used.
*Device pairing is ready. Shane to open issues for the deliverables of this key result.
|
*Pushed forward the challenge with auto-focus.
*There are things we got done this quarter that we can roll forward.
*Great analysis.
|-
|-
| align="left"|3.2 Ship secondary email.
| align="left"|3.2 Ship secondary email.
|align="center"|80%
|align="center"|80%
|align="center"|
|align="center"|80%
|align="center"|
|align="center"|90%
|align="center"|
|align="center"|90%
|align="center"|
|align="center"|90%
|align="center"|
|align="center"|90%
|align="center"|
|align="center"|0.9
|
|
*We are allowing users to add a secondary email. Allows users to change primary email.
|
|
*Have only a little work to finish up from Q2.
*Caching bug fixed
*Client code is in
*Works for AMO, and they have an issue to fix one little thing
*Pocket was the only remaining thing.
|-
|-
| align="left"|3.3 Land support for relier encryption keys.  
| align="left"|3.3 Land support for relier encryption keys.  
|align="center"|70%
|align="center"|70%
|align="center"|
|align="center"|70%
|align="center"|
|align="center"|75%
|align="center"|
|align="center"|75%
|align="center"|
|align="center"|75%
|align="center"|
|align="center"|80%
|align="center"|
|align="center"|0.9
|  
|align="left"|
Relier encryption keys
|
|
*Awaiting 2 reviews for encryption protocol.
*Have done a lot of good work on it but it’s not totally closed out.
*Still working on the reviews.
*Currently under scrutiny by two crypto specialists.
*Can’t ship until we clear the security reviews.
*[https://github.com/mozilla/fxa/issues/245 Issue 245: Prototype key support api for Firefox Notes]
|-
|-
| align="left"|3.4 Integrate 100% of high priority Amplitude events.  
| align="left"|3.4 Integrate 100% of high priority Amplitude events.  
|align="center"|75%
|align="center"|75%
|align="center"|
|align="center"|75%
|align="center"|
|align="center"|75%
|align="center"|
|align="center"|75%
|align="center"|
|align="center"|75%
|align="center"|
|align="center"|90%
|align="center"|
|align="center"|0.9
|align="center"|
|align="left"|
Implementation of Amplitude
|
|
*Have gotten the ball rolling with coordination.
*Should be collecting real-time data this week.
*We’ve haven’t completely tied everything off, but it’s basically done.
*Working in stage and currently fine tuning.
|-
|-
|}
|}

Latest revision as of 01:53, 30 September 2017

September 25, 2017

Q3 OKR Progress

Objective Key Result Confidence Features Notes
Jul 10 Jul 24 Aug 8 Aug 21 Sep 5 Sep 18 Score
1. To have improved the quality for the Firefox Account experience on desktop. 1.1 Close ALL (100%) issues in the Quantum/Photon milestone. (Crouton) 60% 65% 70% 75% 90% 100% 1.0

Priorities for support of Quantum/Photon in 57:

1.2 Close 50% of our Q3 quality backlog. 50% 75% 75% 90% 100% 1.0

Quality Priorities for Q3:

1.3 Increase by 15% the rate of users that become multi-device as a result of going through our first run flow. 50% 50% 50% 40% 0% 0% 0.0
  • Didn't achieve this. Had a lot of questions about the metrics.
1.4 Fix newsletter subscription when email is confirmed on a 2nd device. 95% 95% 95% 95% 90% 90% 1.0

Fixes a problem with getting marketing newsletter opt-in data over to Salesforce.

  • Done!
2. To have improved the quality of the Firefox Account experience on mobile. 2.1 Measure and A/B test best push opt-in screen on iOS. 50% 40% 55% 55% 50% 50% 0.7
  • PR reviewed but not merged.
  • Review brought up some interesting points to address.
2.2 Complete 80% of Q3 mobile bugs. N/A 60% 60% 60% 65% 90% 0.8

Identification and fixing of the most impactful bugs on iOS

3. To have completed all features that are still in flight from Q1 and Q2. 3.1 Have a device pairing flow ready to A/B test. 75% 75% 50% 25% 0% 0% 0.3
  • Allow users to test out device pairing. This does not necessarily mean that QR codes are used.
  • Pushed forward the challenge with auto-focus.
  • There are things we got done this quarter that we can roll forward.
  • Great analysis.
3.2 Ship secondary email. 80% 80% 90% 90% 90% 90% 0.9
  • We are allowing users to add a secondary email. Allows users to change primary email.
  • Caching bug fixed
  • Client code is in
  • Works for AMO, and they have an issue to fix one little thing
  • Pocket was the only remaining thing.
3.3 Land support for relier encryption keys. 70% 70% 75% 75% 75% 80% 0.9

Relier encryption keys

3.4 Integrate 100% of high priority Amplitude events. 75% 75% 75% 75% 75% 90% 0.9

Implementation of Amplitude

  • Should be collecting real-time data this week.
  • We’ve haven’t completely tied everything off, but it’s basically done.
  • Working in stage and currently fine tuning.