CloudServices/Sync/OKRs 2016Q3

From MozillaWiki
Jump to: navigation, search

1. Build user trust by improving structural data integrity for bookmarks.

Description Rating Evaluation
1.1 Measure structural integrity defects in users' bookmark trees. 1 Instrumentation for bookmark defects complete.
1.2 Land fixes to the underlying defects related to the disabled, failing bookmark validation tests in TPS and enable these tests in TPS. 0.7 We haven't enabled mobile root validation in TPS, because bug 1302901 is awaiting review. We also landed other work to support this, even though we haven't enabled the tests yet.
1.3 Deploy sync server API for batch atomic writes to production and land client integrations on Desktop. (iOS and Android need coordination) 0.5 Client work is 100% complete, but the server is awaiting a production deployment plan. Nodes are currently replaced only by attrition, and we also don't want to replace all nodes at once with the new code. Outcome from this is a clear separation of responsibilities between Desktop and server teams: the Desktop team doesn't write server code; the server offers an API that we consume.
1.4** Land improvements to the sync change tracker to ensure that users' bookmark changes are tracked before Sync has initialized or during a sync. 0.5 We have a plan in place. This objective was appropriately ambitious; turns out it was just hard.
1.5* Land repair code that uploads missing bookmarks. 0 We don't have a plan for this, as there are a lot of issues that came up in the design. Overly ambitious; not as simple as we thought.
1.6 Land validators to measure the integrity of 3 other sync data types. Integrate the validators into TPS and report defect rates via telemetry. 0.9 / avg 0.6 This isn't enabled yet, since we're not sure of the risk of enabling validation for all users. Enabling is a one-line change. Apart from that, all validator work is complete and in TPS.

*Won't achieve ** At risk

2 Create value by helping users with task continuity between mobile and desktop.

Description Rating Evaluation
2.1 Add affordance to Desktop context menu to send tab to connected sync devices. 1 Shipped.
2.4 Add measurement to the sync ping for send tab usage. 0.1 This became more ambitious than just measuring clicks on the "Send Tab" button. We have a plan, but it's not at all close to landing. This key result should have been more precise; it's not sufficiently clear to evaluate.
2.2** Improve timeliness of sending tabs to devices using push events on Nightly Desktop, Android, and iOS. 0.8 Shipped on Desktop and Android, which are big-impact platforms. Not shipped on iOS, but less impact.
2.3 Land messaging to inform users of incoming tabs in Desktop (bug 1244597). 1 / avg 0.725 Shipped. UX is working on a second iteration to make this better.

3. Use data to help prioritize new features and fix defects in Sync

Description Rating Evaluation
3.1** Create a bookmark structural integrity dashboard on https://sql.telemetry.mozilla.org/. Dependent on 1.1 0 Structural integrity data is included in the Sync ping, but needs additional work to show in re:dash.
3.2** Create a integrity of 3 sync data types (1.6) dashboard on https://sql.telemetry.mozilla.org/ . Dependent on 1.6 0 We've done a lot of work toward getting data in the pipeline. We also don't have an OKR for a general Sync health dashboard, which did land.
3.3** Visualize send tab usage from 2.4 on https://sql.telemetry.mozilla.org/. 0
3.4** Measure lost data occurrences due to single device password resets, visualize on https://sql.telemetry.mozilla.org/ 0.5 / avg 0.125 We have an estimate: 6% of people signing in reset their passwords and lose their data. 85% of password resets are for users on a single device. The visualization exists, but we're not confident it's answering the question we want to ask.

**At risk but we believe we can get data in Presto or Redshift in order to make Re:Dash dashboards

4. Validate an experience that improves user trust by retaining sync data through a password reset

Description Rating
4.1 Validate with users cross-platform alternatives that persists user data through a password reset.
4.2 ** Instrument the current reset experience as a baseline for future improvements.
4.3 ** Plan measurement of the new experience.

5. Increase multi-device usage by improving in-context messaging

Description Rating
5.1 ** Document options for creating in-context messages reinforcing value of sync features.
5.2 ** Create mocks detailing the user experience through a chosen design.
5.3 ** Test and create report of feature viability through testing on small user sample.