Firefox/Activity Stream/OKRs/Desktop OKRs: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
No edit summary
 
(45 intermediate revisions by the same user not shown)
Line 1: Line 1:
__NOTOC__
__NOTOC__
<big><big><big>'''Activity Stream - Firefox Desktop - OKR Progress Report</big></big></big>
<big><big><big>'''Activity Stream - Firefox Desktop - OKR Progress Report </big></big></big><br>
 
<big><big>'''Q4-2017'''</big></big> <br>
<big><big>'''June 2, 2017'''</big></big>
<big> '''Last Updated: Jan 12th, 2018'''</big><br>
 
<big> '''Next Update: Feb 9th, 2018'''</big><br><br>
[https://wiki.mozilla.org/Firefox/Activity_Stream/Status/30-May-2017 Latest Activity Stream Program Status Report]
<big> '''Q4 Final Score: 36%'''</big>
 
 


== OKR Progress ==
== OKR Progress ==
[https://docs.google.com/document/d/1Ba8up2LQXgGkpfcT5x1ooDN6-XRWwA6q5xy5axXe2oE/edit#heading=h.asted8fbrqja OKR DOCUMENT HERE]<br>
[https://github.com/mozilla/activity-stream-okrs/projects/5 OKR Github Project HERE]<br>
[[File:Green-sm.jpg|frameless|15px]] On track, issues well managed - Confidence level 70% or higher<br> [[File:Yellow-sm.jpg|frameless|15px]] At risk, unknowns without clear mitigations - Confidence level 40-69% <br>
[[File:Red-sm.jpg|frameless|15px]] Blocked on deliverables, needs decision, escalation required - Confidence level below 40%<br>
<!--Should explain WHY for the health/status reported above.  Items covered here may be covered in more detail further down in the report.  If senior management were to look at nothing but this section, they should be able to tell what is going on with the project.-->


{| class="wikitable" style="color:#000000; background-color: #FFFFFF; padding: 10"
{| class="wikitable" style="color:#000000; background-color: #FFFFFF; padding: 10"
|rowspan=2; style="text-align: center; background-color: #f9d9a8"| '''Objective'''  
|rowspan=2; style="text-align: center; background-color: #f9d9a8"| '''Objective'''  
|rowspan=2; style="text-align: center; background-color: #f9d9a8"| '''Key Result'''
|rowspan=2; style="text-align: center; background-color: #f9d9a8"| '''Key Result'''
|colspan=7; style="text-align: center; background-color: #f9d9a8"| '''Confidence'''
|colspan=3; style="text-align: center; background-color: #f9d9a8"| '''Confidence'''
|rowspan=2; style="text-align: center; background-color: #f9d9a8"| '''State'''
|rowspan=2; style="text-align: center; background-color: #f9d9a8"| '''Notes'''
|rowspan=2; style="text-align: center; background-color: #f9d9a8"| '''Notes'''
|-
|-
|style="text-align: center; background-color: #f9d9a8"|May 2
|style="text-align: center; background-color: #f9d9a8"|Oct 6
|style="text-align: center; background-color: #f9d9a8"|May 16
|style="text-align: center; background-color: #f9d9a8"|Nov 3
|style="text-align: center; background-color: #f9d9a8"|June 2
|style="text-align: center; background-color: #f9d9a8"|Final Score<br>Jan 2018
|style="text-align: center; background-color: #f9d9a8"|June 13
|style="text-align: center; background-color: #f9d9a8"|June 30
|style="text-align: center; background-color: #f9d9a8"|July 11
|style="text-align: center; background-color: #f9d9a8"|July 28
|-
|-
|rowspan=1| 1. ACTIVITY STREAM ENGAGEMENT - Activity Stream newtab presents users with a more engaging experience
|rowspan=7| 1. Support Activity Stream Release <br>
|align="left"|1.1 Activity Stream has feature parity with existing about:newtab and about:home
-<br>
|align="center"|75%
-  
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/37 75%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/37 40%]
|align="center"|
|align="center"|
|align="center"|
|align="center"|
| Currently at 14 Open and 11 Closed [https://github.com/mozilla/activity-stream/issues?q=is%3Aissue+label%3AParity+is%3Aopen Parity Bugs Here].  Remaining Open are part of MVP Scope and still considered on track for FF57. There is work that doesn't make sense to do on the Test Pilot side, and should only be implemented in the system add on, and all of this likely won't be completed in the system add-on by the end of June.
|-
|-
|align="left"|1.1 Weighed Highlights
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/60 50%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/60 0%]
|align="center"|0%
|align="center"|De-prioritized
|Extend highlights section to include X of the following: bookmarks, downloads, 'pockets', screenshots, synced tabs, watched videos.<br>
11/17 - De-prioritized in favor of activities necessary to ship Activity Stream in FX57
|-
|align="left"|1.2 Implement Tippytop Sites Databases
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/57 80%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/57 90 %]
|align="center"| 100%
|align="center"| Complete
|50% complete.  Partially completed earlier.  Remaining work expected to wrap up in the coming weeks. <br> 01/18 - Completely Successfully.


|rowspan=4| 2. ACTIVITY STREAM PERFORMANCE - AS will not negatively impact Firefox desktop performance.  AS will comply with all Quantum perf targets wrt Smoothness, Responsiveness and Page Load.
| align="left"|2.1 Activity Stream is continuously profiled and optimized on a shitty Windows laptop
|align="center"|60%
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/30 60%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/30 60%]
|align="center"|
|align="center"|
|align="center"|
|align="center"|
| This is not happening continuously, as most people do not have access to such hardware on consistent basis, and also other items are taking priority. We need to get 'shitty windows laptops' in more peoples hands (mostly the Graduation team).  A short demonstration on how to do the profiling will be given in Lightning Talks next week.
|-
|-
| align="left"|2.2 The Activity Stream system add-on collects and reports key (well defined) performance metrics through telemetry
|align="left"|1.3 Customization
|align="center"|75%
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/63 50%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/29 75%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/63 10%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/29 85%]
|align="center"|70%
|align="center"|
|align="center"|Satisfactory Progress Made
|align="center"|
|De-prioritized in favor of work required to launch Activity Stream in FX57
|align="center"|
|align="center"|
| Key telemetry infrastructure has landed and investigation into key metrics has been completed.  The remaining piece is implementation which is currently considered on track.  Expect to begin landing code early next week.
|-
|-
| align="left"|2.3  The Activity Stream system add-on does not regress Page Load, Smoothness or Responsiveness in Firefox relative to Tiles
|align="left"|1.4 Improve average Top Site, Pocket and Highlights post click load performance by 50%  
|align="center"|75%
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/70 50%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/28 75%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/70 100%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/28 75%]
|align="center"|100%
|align="center"|
|align="center"| Done
|align="center"|
|Work Completed.
|align="center"|
|align="center"|
| Currently Activity Stream in Nightly meets and/or exceeds all performance expectations.  Will continue to monitor as we move through to the Release channel. Not noticing any regressions on Pine, perfherder. Confidence remains unchanged until we pref on.
|-
|-
| align="left"|2.4  The Activity Stream system add-on does not cause appreciable main thread hangs
|align="left"|1.5 Preferences -> Firefox Main Preferences
|align="center"|75%
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/65 50%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/27 75%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/65 10%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/27 78%]
|align="center"|0%
|align="center"|
|align="center"|De-prioritized
|align="center"|
|De-prioritized in favor of work necessary to ship Activity Stream in FX57
|align="center"|
|align="center"|
| Activity Stream has been instrumented with a BackgroundHangReporter such that we will receive reports if any notable hangs occur, and thus can be investigated and resolved as necessary.  No hangs reported on nightly to date. There are still ongoing optimizations. Confidence changes only slightly until we pref on in Nightly, at while point we'll have more information. 
|-
|-
|rowspan=6| 3.  GRADUATION - Activity Stream experience is available to a meaningful percentage of the general Firefox Nightly population
|align="left"|1.6 Defects, Regressions and Measurement and Optimization thereof.
| align="left"|3.1 Activity Stream has landed in m-c and is pref'd on for at least 10% of Nightly users
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/66 50%]
|align="center"|75%
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/66 70%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/38 75%]
|align="center"|70%
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/38 75%]  
|align="center"| Satisfactory Progress Made
|align="center"|
|Target: keep the number of open critical regressions against released AS to < 5
|align="center"|
|align="center"|
|align="center"|
| Requirements for this KR are expected to be met at the end of the current sprint (Apocalypse Now). The main element of risk here, based on previous experience, is the tests
|-
|-
| align="left"|3.2 Activity Stream has completed all privacy and security reviews/audits
 
|align="center"|75%
|rowspan=4| 2. Experiment Services <br>
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/36 75%]
-<br>
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/36 75%]
-  
|align="center"|
|align="center"|
|align="center"|
|align="center"|
| Making progress and on track.  Both groups have been engaged.  Fairly confident that we are not doing anything (at least obviously) that would cause us to fail either review. 
|-
|-
| align="left"|3.3 Activity Stream has passed all Accessibility reviews
|align="left"|2.1 Run at least 8 experiments using Experimenter / Shield / Normandy / Ping Centre / Redash Client stack
|align="center"|60%
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/71 90%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/35 60%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/71 100%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/35 70%]
|align="center"|100%
|align="center"|
|align="center"|Done
|align="center"|
| 11/17 - Target number of experiments met for the quarter
|align="center"|
|align="center"|
| Probably don't need to wait until we are pref'd on in nightly.  Everything required is implemented.  Tim will engage Marco Zehe to get this kicked off. 
|-
|-
| align="left"|3.4 Ability to store and query Activity Stream specific metadata in PlacesDB module
|align="left"|2.2 Run at least one 'full-stack' experiment for a non-AS group
|align="center"|75%
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/68 50%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/32 75%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/68 0%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/32 80%]
|align="center"|70%
|align="center"|
|align="center"|Satisfactory Progress Made
|align="center"|
| De-prioritized in favor of other activities required to launch Activity Stream in FX57<br>
|align="center"|
01/18 - While not 'run', work was done to support Pocket Spoc experimentation going forward.
|align="center"|
| Patch was reviewed.  Waiting until after freeze/merge to land this.  Further details can be found in the bug [https://bugzilla.mozilla.org/show_bug.cgi?id=1352502 HERE].
|-
|-
| align="left"|3.5 The Activity Stream system add-on sends all event telemetry to ping-centre
|align="left"|2.3 Begin ping-centre / Event UT integration for Activity Stream
|align="center"|75%
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/69 50%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/31 75%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/69 10%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/31 90%]
|align="center"|0%
|align="center"|
|align="center"|De-prioritized
|align="center"|
| De-prioritized in favor of other activities required to launch Activity Stream in FX57
|align="center"|
|align="center"|
| This now exists in central. We thought this was complete, but not quite. The backend infrastructure needs to be updated, and needs to be pref'd on.
|-
|-
| align="left"|3.6 Sentry
 
|align="center"|75%
|rowspan=6| 3. Performance & Telemetry<br>
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/41 75%]
-<br>
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/41 75%]
-  
|align="center"|
|align="center"|
|align="center"|
|align="center"|
| There is a patch for this already. Approval to land this in Nightly has been given. Once landed further evaluation will take place, which may impact confidence.
|-
|-
|rowspan=4| 4. EXPERIMENTATION & ANALYSIS - Our capability to rapidly and accurately perform data based product experimentation is standardized and automated.
|align="left"|3.1 Release AS Web Extension API (Newtab Content Extension) to Firefox 
| align="left"|4.1 Launched at least one experiment using production Experimenter deployment
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/44 30%]
|align="center"|75%
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/44 10%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/39 75%]
|align="center"|0%
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/41 85%]
|align="center"|De-prioritized
|align="center"|
| Written but not landed. Needs input from other groups <br>
|align="center"|
11/17 - De-prioritized in favor of other activities required to launch Activity Stream in FX57
|align="center"|
|align="center"|
| Working on auth.  Should be getting close to ready to deploy.  Does generate the JSON file.  Need signoff from Ops.
|-
|-
| align="left"|4.2 Run Shield Study to prove AS is as engaging as existing about:newtab
|align="left"|3.2 Release 1 Go Faster System update of AS (Nightly -> Beta)
|align="center"|75%
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/43 60%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/34 75%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/43 50%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/34 80%]
|align="center"|0%
|align="center"|
|align="center"|De-prioritized
|align="center"|
| 11/17 - We may still have the opportunity to do this before the end of the quarter, but has until now been de-prioritized. <br>
|align="center"|
1/18 - Did not fit this into the end of the quarter as mentioned above.
|align="center"|
| Parameters for the study have been defined. Will have pocket and 2 rows of top sites as one of the variants. Other will be Tiles. Targeting end of June/early July.
|-
|-
| align="left"|4.3 Build out experimentation infrastructure in Nightly and run at least one experiment
|align="left"|3.3 Notification Centre
|align="center"|60%
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/61 50%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/33 60%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/61 10%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/33 60%]
|align="center"|0%
|align="center"|
|align="center"|De-prioritized
|align="center"|
| De-prioritized in favor of other activities required to launch Activity Stream in FX57
|align="center"|
|-
|align="center"|
|align="left"|3.4 Signals Library
| The requirements here have not been clear and thus nobody has been driving it forward.  Dependent on being pref'd on for some percentage of users first so we have data. Not considered a priority by the team for this quarter.
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/62 50%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/62 0%]
|align="center"|0%
|align="center"|De-prioritized
| De-prioritized in favor of other activities required to launch Activity Stream in FX57
|-
|align="left"|3.5 Sentry, Operational Health and Measurement thereof
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/67 50%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/67 50%]
|align="center"|0%
|align="center"|De-prioritized
| 11/17 - Attempting to fit in landing a Sentry service to AS over then next few weeks.<br>
1/18 - Did not fit in to plans at the end of the year after all.
|-
|-
| align="left"|4.4 Deploy automated dashboard generation
|align="center"|75%
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/40 75%]
|align="center"|[https://github.com/mozilla/activity-stream-okrs/issues/40 85%]
|align="center"|
|align="center"|
|align="center"|
|align="center"|
| This is getting very close.  100% Test coverage. Redash client reads and parses experiments.JSON definition in format on S3 sent from Experimenter and generates update-able dashboards from it.  Working on adding logging and deploy package for lambda


|-
 
|}
|}


[[Category:Wikipage templates]]
[[Category:Wikipage templates]]

Latest revision as of 21:45, 19 January 2018

Activity Stream - Firefox Desktop - OKR Progress Report
Q4-2017
Last Updated: Jan 12th, 2018
Next Update: Feb 9th, 2018

Q4 Final Score: 36%

OKR Progress

OKR Github Project HERE

Objective Key Result Confidence State Notes
Oct 6 Nov 3 Final Score
Jan 2018
1. Support Activity Stream Release

-
-

1.1 Weighed Highlights 50% 0% 0% De-prioritized Extend highlights section to include X of the following: bookmarks, downloads, 'pockets', screenshots, synced tabs, watched videos.

11/17 - De-prioritized in favor of activities necessary to ship Activity Stream in FX57

1.2 Implement Tippytop Sites Databases 80% 90 % 100% Complete 50% complete. Partially completed earlier. Remaining work expected to wrap up in the coming weeks.
01/18 - Completely Successfully.
1.3 Customization 50% 10% 70% Satisfactory Progress Made De-prioritized in favor of work required to launch Activity Stream in FX57
1.4 Improve average Top Site, Pocket and Highlights post click load performance by 50% 50% 100% 100% Done Work Completed.
1.5 Preferences -> Firefox Main Preferences 50% 10% 0% De-prioritized De-prioritized in favor of work necessary to ship Activity Stream in FX57
1.6 Defects, Regressions and Measurement and Optimization thereof. 50% 70% 70% Satisfactory Progress Made Target: keep the number of open critical regressions against released AS to < 5
2. Experiment Services

-
-

2.1 Run at least 8 experiments using Experimenter / Shield / Normandy / Ping Centre / Redash Client stack 90% 100% 100% Done 11/17 - Target number of experiments met for the quarter
2.2 Run at least one 'full-stack' experiment for a non-AS group 50% 0% 70% Satisfactory Progress Made De-prioritized in favor of other activities required to launch Activity Stream in FX57

01/18 - While not 'run', work was done to support Pocket Spoc experimentation going forward.

2.3 Begin ping-centre / Event UT integration for Activity Stream 50% 10% 0% De-prioritized De-prioritized in favor of other activities required to launch Activity Stream in FX57
3. Performance & Telemetry

-
-

3.1 Release AS Web Extension API (Newtab Content Extension) to Firefox 30% 10% 0% De-prioritized Written but not landed. Needs input from other groups

11/17 - De-prioritized in favor of other activities required to launch Activity Stream in FX57

3.2 Release 1 Go Faster System update of AS (Nightly -> Beta) 60% 50% 0% De-prioritized 11/17 - We may still have the opportunity to do this before the end of the quarter, but has until now been de-prioritized.

1/18 - Did not fit this into the end of the quarter as mentioned above.

3.3 Notification Centre 50% 10% 0% De-prioritized De-prioritized in favor of other activities required to launch Activity Stream in FX57
3.4 Signals Library 50% 0% 0% De-prioritized De-prioritized in favor of other activities required to launch Activity Stream in FX57
3.5 Sentry, Operational Health and Measurement thereof 50% 50% 0% De-prioritized 11/17 - Attempting to fit in landing a Sentry service to AS over then next few weeks.

1/18 - Did not fit in to plans at the end of the year after all.

Retrieved from "https://wiki.mozilla.org/index.php?title=Firefox/Activity_Stream/OKRs/Desktop_OKRs&oldid=1187136"