This doc is work in progress

OKRs for Push

This document tracks the Objectives and Key Results that inform the work of the Push team on a quarterly basis.

Q3 2016

This quarter’s focus is on our partners and the adoption of push in general by promoting the value of push to developers and content producers.

Overall Score for Q3 2016: (0.0 - 1.0; average of scores below)

Objective #1: Show how Push is valuable

(Primarily Arcadio & Bryan)

Key Result Score Develop key push metrics document[a][b] (aha)

• What metrics are we using to identify funnel and engagement

(0.0 - 1.0) Create and keep updated slide for metrics and product news on Firefox Product Cross-Functional Weekly (9am PST Wed) (aha) (0.0 - 1.0) Work with Hacks blog to install and use wp-web-push (aha) (0.0 - 1.0) Write up Hacks post on implementation lessons of wp-web-push (aha) (0.0 - 1.0) Write follow article up on the value of desktop push in general (aha) (0.0 - 1.0) Write follow up article detailing Hack team experiments with the UX of when to request push and increase conversions (aha) (0.0 - 1.0) Overall Score: (0.0 - 1.0) (avg of above)

Objective #2: Push service is ready for traffic at scale

Key Result Score 1. Successful rollout of Push with Partner

• Create endpoint specifically for Partner use
• Load test of FB endpoint can scale to 10 million users

(0.0 - 1.0) 2. Develop a test system that is reliable, easy to use, and run regularly.

• Stabilize load testing system code for reliable runs
• Automate load-testing system runs on check-in
• Regularly run scaled load-test to 10 million target

(0.0 ¢- 1.0) 3. Define and create partner engineer ‘run book’ to have team transparency with quality of service.

• Clear communication channels, expectation of bugs logged, and ownership of issue.
• Define requirements for response times and contact points for tier 1 partners.

(0.0 - 1.0) 4. Getting monitoring up to the level of quality needed for FB scale

• Documented set of Push system monitors, alerts, and alert levels
• Develop and deploy new metrics and logging
• Work with Metrics/Ops to get new alerting systems in place

(0.0 - 1.0) 5. Debugging Dashboard

• Ability to lookup endpoints for log traces, histogram of activity
• Ability to lookup uaid/uaid_hash/channelID for log activity
• Forceful expiration of a given endpoint/uaid (integration testing ability)

(0.0 - 1.0) 6. Create “Push at a Glance” dashboard on Mana that embeds KPIs from re:dash and Datadog.

• Integrate major KPI’s for Executive view with metric descriptions

(0.0 - 1.0) Overall Score: (0.0 - 1.0) (avg of above)

Objective #3: Improved developer experience and competitiveness

Key Result Score 1. Plan experiment[c][d] to test Firefox (push) continuing to run in the background even after Firefox is closed. (aha) (0.0 - 1.0) 2. requireInteraction

• Implement and land requireInteraction on nightly[e][f] (platform)
• Develop product release requirements plan for shipping in release

(0.0 - 1.0) 3. Create a roadmap and decision criteria for implementing spec and non-spec future Push features:

• Topics
• Custom actions
• Receipts

(0.0 - 1.0) 4. Create a product plan for a push status page that gathers all relevant information for partners and implementers.

• Uptime
• Release version
• Compat Matrix Information
• Current Load
• Contact Points

(0.0 - 1.0) 5. Publish VAPID and data encryption article to services blog and MDN

• blog draft[g]
• Get review sign-off from Chris Mills, Bryan Clark, and Martin Thomson

(0.0 - 1.0) 6. Validate integration testing solutions for Push

• Proposals of options (Docker, etc) written up
• Shared proposal with 3 partners

(0.0 - 1.0) Overall Score: (0.0 - 1.0) (avg of above)

Objective #4: Support Push for Context Graph mobile efforts

Key Result Score 1. Deploy APNS support to production

• APNS deployed in dev/stage with separate iOS key for prod

(0.0 -1.0) 2. Regularly run test suites for GCM and APNS bridges

• Documented run-script for Push Bridge testing and devices to use
• Regularly scheduled run-tests with logged/historic results

(0.0 - 1.0) 3. Create dashboards for GCM and APNS usage

• Define KPI’s for GCM/APNS
• Implement logging/metrics and deploy to prod
• Aggregated dashboard view in Mana or re:dash of Bridge KPI’s

(0.0 - 1.0) Overall Score: (0.0 - 1.0) (avg of above)

[a]Do we need more Gecko-based telemetry here? [b]bryan says we'll need some probes to verify server and client usage funnel. [c]+overholt@mozilla.com I'll need someone from platform to help create this plan. [d]I'm happy to help and get input from others. [e]+overholt@mozilla.com I'd like to hold this on nightly until we have a plan for the UX. Not necessarily stacking. [f]OK. [g]Published a draft to my personal blog: https://blog.unitedheroes.net/archives/p/4973/sending-webpush-notifications-via-mozilla-push-service-autopush/