Test Pilot/2017Q3

From MozillaWiki
Jump to: navigation, search

Test Pilot 2017 Q3 Objectives and Key Results

Final Status (Sep 30) Comments
Objective 1 Improve storytelling so that we support Firefox and grow community .48
KR1 Bi-weekly updates for each new experiment published on the blog or experiment pages. .66 We didn't publish updates at this cadence for every experiment, but we did publish a lot of things, including specific posts on the experiments. We decided this was about 2/3rds the spirit of this KR.
KR2 Get 1000 combined reads on all blog posts 1 So many reads. Over 6000 near the end of September.
KR3 Have 15 people participate in user research on design prototypes .27 Only 4 participants in the Notes optimization study
KR4 30 attendees at a Brown Bag about storyboarding 0 This was planned for Taipei but that trip was cancelled. We've made a new plan to host a meeting in Austin with the ET team.
Objective 2 Grow Screenshots usage and expand offered features so that people find Screenshots to be a useful tool. .73
KR1 Ship screenshots to 100% of release users 1 🎉 Shipping by default in 56
KR2 10% of Screenshots users sign in to FxA 0 A patch to integrate FxA exists, but we didn't ship it to production.
KR3 .5% of non-owner, non-Firefox visitors click a branded CTA 1 (total click events)/((unique non-ff visits to the home page)+(unique non-ff shot page views)) = 1.04%
KR4 All pages and API endpoints load in under 2s (minus network time) .9 All the endpoints are well under 2s now, except we found the "delete all data" endpoint near the end and it's still pretty slow.
Objective 3 Improve Test Pilot so that the Test Pilot program is a pleasure to use, conforms to the Firefox 57 plan, and improves engagement with experiments. 1
KR1 Ship SDK-free Test Pilot to 100% of users 1
KR2 Use storybook for all newly implemented feature cards 1
KR3 Implement 4 Test Pilot feature cards 1 2 shipped, 2 waiting on L10n freeze and will ship next sprint
KR4 Decrease home page load time by 25% 1 Beat this by a decent amount, but need to get the actual numbers from Donovan.
KR5 Send a notification through the Test Pilot add-on using a WebExtension experiment API 1 This code exists and works, but didn't ship to production yet.
Objective 4 Find, guide, and graduate ideas and experiments into features so that Test Pilot is a proven valuable input to the future direction of Firefox .17
KR1 Publish conclusions from three A/B tests for Firefox Send .33 We shipped one A/B test and published the results.
KR2 Create a document with a plan to ship a partner-repack experiment 0