Unified Telemetry/Status reports/July 10 2015: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(→‎Risks/Issues: owner tweak)
m (Kparlante moved page Status reports/July 10 2015 to Unified Telemetry/Status reports/July 10 2015: move to namespace)
 
(12 intermediate revisions by 2 users not shown)
Line 1: Line 1:
[https://mana.mozilla.org/wiki/display/PM/Unified+Telemetry+Status+report+July+10 previous weeks report]
[https://mana.mozilla.org/wiki/display/PM/Unified+Telemetry+Status+report+July+6 previous weeks report]


== Unified Telemetry status report July 10, 2015 ==
== Unified Telemetry status report July 10, 2015 ==
Line 5: Line 5:
=== Overall Project Health ===
=== Overall Project Health ===


Red - Development work is near completion (expected next week). The validation work required to be confident enough to turn off the FHR mechanism will exceed the amount of time remaining in the cycle. Unified telemetry's target for turning off FHR and collecting opt-out telemetry information from the release population is now r41.
Red - Development work is near completion (expected next week). The validation work required to be confident enough to turn off the FHR mechanism will exceed the amount of time remaining in the cycle. Unified telemetry's target for turning off FHR and collecting opt-out telemetry information from the release population is now r41. The pipeline has been in production and collecting pre-release UT data; it's ready for other types of production traffic (e.g. FxOS pings and cloud service log data).


=== Exec Summary ===
=== Exec Summary ===
Line 18: Line 18:
** https://bugzilla.mozilla.org/show_bug.cgi?id=1160636 (Allow query of "how many users of type X")
** https://bugzilla.mozilla.org/show_bug.cgi?id=1160636 (Allow query of "how many users of type X")
* Ongoing planning on FHR V2/V3 historic pipeline migration link to status [https://mana.mozilla.org/wiki/display/PM/FHR+historic+pipeline+update+July+6 here].
* Ongoing planning on FHR V2/V3 historic pipeline migration link to status [https://mana.mozilla.org/wiki/display/PM/FHR+historic+pipeline+update+July+6 here].
* creation of milestones and plan for r41 delivery begins
* Creation of milestones and plan for r41 delivery begins


=== Risks/Issues ===
=== Risks/Issues ===
Line 25: Line 25:
! Description of Risks/Issues !! State !! Owner !! Plan to Resolve/Mitigation !! Target Date
! Description of Risks/Issues !! State !! Owner !! Plan to Resolve/Mitigation !! Target Date
|-
|-
| Data integrity between V2/V4 and V4 internal data consistency || Open || Brendan/Sam || Investigation in progress. Added resources (Sam). https://etherpad.mozilla.org/fhr-v4-validation || Example
| Data integrity between V2/V4 and V4 internal data consistency || Open || Brendan/Sam || Investigation in progress. Added resources (Sam). https://etherpad.mozilla.org/fhr-v4-validation || 7/15
|-
|-
| Data continuity across V2/V4 || Open || Katie/Mark/Trink || Mark writing up plan from Whistler; metrics team specifying data sets and reviewing "executive" data set. https://bugzilla.mozilla.org/show_bug.cgi?id=1182684 || Example
| Data continuity across V2/V4 || Open || Katie/Mark/Trink || Mark writing up plan from Whistler; metrics team specifying data sets and reviewing "executive" data set. https://bugzilla.mozilla.org/show_bug.cgi?id=1182684 || 7/15
|-
|-
| Legal review || Open || BDS/Legal || Meeting between groups || Example
| Legal review || Open || BDS/Legal || Meeting between groups || 8/04
|-
|-
| QA sign off (functional, load) || Open || Stuart || Working with QA on creating test cases/test plans || Example
| QA sign off (functional, load) || Open || Stuart || Working with QA on creating test cases/test plans || 8/04
|-
|-
| Operations - data retention requirements || Open || Travis/Katie || Eng team owes ops a doc defining ping types and data retention requirements || Example
| Operations - data retention requirements || Open || Travis/Katie || Eng team owes ops a doc defining ping types and data retention requirements || 8/04
|-
|-
| Operations - analysis tools & microservices || Open || Travis/Katie || Eng team to provide architecture and data flow, Ops figures out micro services needed || Example
| Operations - analysis tools & microservices || Open || Travis/Mark/Roberto || [https://docs.google.com/a/mozilla.com/document/d/1KoLtIFV-aZtxruSVNmcc26F22MfqWjDynKgZ6adYk54/edit?usp=sharing%20 Architecture/Data flow diagram]; meeting next Monday (7/13) || 8/04
|-
|-
| Data loss issue last week || Open || mreid/whd/trink || [https://bugzilla.mozilla.org/show_bug.cgi?id=1179128 Tee server needs to return error status from old or new]. Added Ops resources (Daniel Thornton). || Example
| Data loss incident || Open || mreid/whd/trink || [https://bugzilla.mozilla.org/show_bug.cgi?id=1179128 Tee server needs to return error status from old or new]. Added Ops resources (Daniel Thornton). || 7/15
|-
|-
| Remote about:healthreport content || Open || Katie/Georg || Made a request to Laura Thomson for help || Example
| Remote about:healthreport content || Open || Katie/Georg || Made a request to Laura Thomson for help || 8/04
|-
|-
| Example || Example || Example || Example || Example
| Budget, size of UT pings || Open || Mark/BDS || https://bugzilla.mozilla.org/show_bug.cgi?id=1182693 || 8/04
|-
| Analysis difficulty || Open || Katie/tbd || No plan yet, aside from ongoing work on tools || 8/04
|}
|}


=== Accomplished for Last Period ===
=== Accomplished for Last Period ===


* Data validation from beta stream
* Client work: [https://docs.google.com/spreadsheets/d/1yAJmgCGYyk1d7A41DZa653Z3u2AbH-kDWsO1vPSgbfE/edit?usp=sharing Spreadsheet]
* Tagged/sorted validation tickets
* Updates to the unified telemetry decoder and executive report
* [https://docs.google.com/a/mozilla.com/document/d/1KoLtIFV-aZtxruSVNmcc26F22MfqWjDynKgZ6adYk54/edit?usp=sharing Architecture flow diagram] in preparation for meeting with ops
* Progress on data validation
**  Compare FHR v2 and FHR v4 search, crash, and other fields: https://bugzilla.mozilla.org/show_bug.cgi?id=1179376 -- close agreement for search counts
** Saved-session vs main pings: https://bugzilla.mozilla.org/show_bug.cgi?id=1147395 -- mismatch in about 7% of sessions for one of the metrics investigated


=== Planned for Upcoming Period ===
=== Planned for Upcoming Period ===


Engineering
Engineering
* Python notebooks
* Uplift final client changes for r40: [https://docs.google.com/spreadsheets/d/1yAJmgCGYyk1d7A41DZa653Z3u2AbH-kDWsO1vPSgbfE/edit?usp=sharing spreadsheet]
* Data validation
* Data validation: https://etherpad.mozilla.org/fhr-v4-validation
* compare v2/v4 rollups for dashboards
* Continue working on work in "b5" milestone: http://mzl.la/1FPWuJG
Ops
Ops
* Tooling
* Meeting to go over Telemetry tools/microservices production deployment
* microservices
* Continued work on scaling for release loads
Performance
* items
QA
* test cases and testing
Project Management
Project Management
* create meeting for legal review
* create meeting for legal review
* follow up with ops and qa
* follow up with ops and qa
* mitigation plan for projects depending on UT
* reassess milestones given schedule adjustment


=== Outstanding requests not yet road mapped into a release ===
=== Outstanding requests not yet road mapped into a release ===

Latest revision as of 17:20, 16 July 2015

previous weeks report

Unified Telemetry status report July 10, 2015

Overall Project Health

Red - Development work is near completion (expected next week). The validation work required to be confident enough to turn off the FHR mechanism will exceed the amount of time remaining in the cycle. Unified telemetry's target for turning off FHR and collecting opt-out telemetry information from the release population is now r41. The pipeline has been in production and collecting pre-release UT data; it's ready for other types of production traffic (e.g. FxOS pings and cloud service log data).

Exec Summary

  • Ongoing effort to validate data from nightly, aurora and beta channels
  • Ongoing effort to prepare pipeline to scale to release traffic
  • Ongoing effort to make telemetry tools and APIs work with v4 data
  • Working on a mitigation plan for projects that were hoping to analyze release population data in r40
    • Data available from nightly, aurora and beta channels now; analysis can begin
    • Create python notebooks with example code for these projects
  • Re-prioritizing two visualization projects that make use of pre-release data:
  • Ongoing planning on FHR V2/V3 historic pipeline migration link to status here.
  • Creation of milestones and plan for r41 delivery begins

Risks/Issues

Description of Risks/Issues State Owner Plan to Resolve/Mitigation Target Date
Data integrity between V2/V4 and V4 internal data consistency Open Brendan/Sam Investigation in progress. Added resources (Sam). https://etherpad.mozilla.org/fhr-v4-validation 7/15
Data continuity across V2/V4 Open Katie/Mark/Trink Mark writing up plan from Whistler; metrics team specifying data sets and reviewing "executive" data set. https://bugzilla.mozilla.org/show_bug.cgi?id=1182684 7/15
Legal review Open BDS/Legal Meeting between groups 8/04
QA sign off (functional, load) Open Stuart Working with QA on creating test cases/test plans 8/04
Operations - data retention requirements Open Travis/Katie Eng team owes ops a doc defining ping types and data retention requirements 8/04
Operations - analysis tools & microservices Open Travis/Mark/Roberto Architecture/Data flow diagram; meeting next Monday (7/13) 8/04
Data loss incident Open mreid/whd/trink Tee server needs to return error status from old or new. Added Ops resources (Daniel Thornton). 7/15
Remote about:healthreport content Open Katie/Georg Made a request to Laura Thomson for help 8/04
Budget, size of UT pings Open Mark/BDS https://bugzilla.mozilla.org/show_bug.cgi?id=1182693 8/04
Analysis difficulty Open Katie/tbd No plan yet, aside from ongoing work on tools 8/04

Accomplished for Last Period

Planned for Upcoming Period

Engineering

Ops

  • Meeting to go over Telemetry tools/microservices production deployment
  • Continued work on scaling for release loads

Project Management

  • create meeting for legal review
  • follow up with ops and qa
  • mitigation plan for projects depending on UT
  • reassess milestones given schedule adjustment

Outstanding requests not yet road mapped into a release

Description State Owner Plan to Resolve/Mitigation Target Date
FireFox OS - app pings Open Katie Need to schedule and understand impact on project TBD
histograms for loop/hello Open Katie Need to schedule and understand impact on project TBD

Important Links/References

Retrieved from "https://wiki.mozilla.org/index.php?title=Unified_Telemetry/Status_reports/July_10_2015&oldid=1085186"