Test Pilot/2017Q1
From MozillaWiki
Test Pilot 2017 Q1 Objectives and Key Results
| Final Status (Mar 31) | Comments | ||
|---|---|---|---|
| Objective 1 | Demonstrate Test Pilot experiments can result in changes to Firefox | 0.5 | |
| KR1 | Two former Test Pilot experiments in active SHIELD studies | 0.5 | This was supposed to be Activity Stream and Tab Center. Tab Center hit a snag landing the system add-on and is delayed until early/mid Q2. In retrospect, we shouldn't have had this on our goal list since we don't own it. |
| Objective 2 | Be company role models for community involvement | 0.59 | |
| KR1 | Test 3 contributor value hypotheses and publish the results | 0.3 | Lots of planning towards this. Three finalists:
|
| KR2 | Publish conclusions from three experiments as they graduate from Test Pilot | 0.67 | No More 404s and Tracking Protection |
| KR3 | Create a draft curriculum for an upcoming class with the Open Innovation team | 0.8 | Lots of planning with the Ravensbourne group. We have two Design Prompts written which will be the foundation of the course. |
| Objective 3 | Grow Test Pilot's user base and capabilities | 0.45 | |
| KR1 | At the end of Q1 double the percentage of new sessions enabling an experiment on intial Test Pilot install | 0.61 | Q4 2016 was 6.6% . source; page 3, "% of New Sessions Enabling an Exp". There is a new dashboard which puts this number at 10.9%, with some qualifiers: If you look at the performance by traffic medium, you'll see 65% of our traffic was from snippets, but snippets have the lowest percentage of sessions enabling experiments (7.4%). That means any time we run a snippet campaign we tank this number, so basing it off of a percentage of new visitors was a bad idea and not a great OKR. The proper way to do this would have been an A/B test. .61 credit for the increase in the number, just realize that it's heavily weighted by other forces. |
| KR2 | Improve Smoothed Test Pilot Engagement Ratio by 6% | 0.17 | Average of the entirety of Q4 is .53 so our target was .59. Average for Q1 turned out to be .54, up 1% and 1/6th of our goal. From: source; second graph down "TxP Active Users / TxP Smoothed Engagement Ratio". Click View Source, change the dates to >90 days to include the whole quarter, download the CSV, ensure a quarter's worth of data for TxP add-on, run Average() on the smoothed ratio. |
| KR3 | Plan a mobile GTM for 2017 | 0.0 | This deliverable is a document outlining the plan. |
| KR4 | Create 4 feature cards ready for UX inspired by user testing reports | 1.0 | Cards on trello |
Updates
- On google drive
- Mid-Quarter we got the request for Firefox Screenshots (formerly Page Shot) to be landed directly in Firefox for our June launch. This made some waves in our schedule and disrupted some OKRs. Overall, we still have a lot to be proud of that isn't represented above: First experiment landing in Firefox (Activity Stream landing soon after), first web extension as a system add-on landing in Firefox, first use of Google Analytics in Firefox, coordination for Screenshots between a dozen teams (and still looking like it's on time), 2/3rds of our successfully graduated experiments happened this quarter, the Test Pilot codebase continues to be simplified and improved (added Flow)
