Test Pilot/2017Q1: Difference between revisions
Jump to navigation
Jump to search
mNo edit summary |
mNo edit summary |
||
| Line 30: | Line 30: | ||
| KR1 | | KR1 | ||
| Test 3 contributor value hypotheses and publish the results | | Test 3 contributor value hypotheses and publish the results | ||
| | | <span style="color:red;font-weight:bold;">0.15</span> | ||
| Lots of [https://public.etherpad-mozilla.org/p/testpilot-participation planning] towards this. Three finalists: <ul><li>Voting up/down possible new experiments to help us choose and help people feel more involved</li><li>Writing up ideas for other users on how they are using experiments (e.g. snooze tabs seems to be something that people could develop productivity tips around)</li><li>Do a better job telling the story of Test Pilot. There's something here about content that TP is generating in terms of an approach to ideation and experimenting</li></ul> | | Lots of [https://public.etherpad-mozilla.org/p/testpilot-participation planning] towards this. Three finalists: <ul><li>Voting up/down possible new experiments to help us choose and help people feel more involved</li><li>Writing up ideas for other users on how they are using experiments (e.g. snooze tabs seems to be something that people could develop productivity tips around)</li><li>Do a better job telling the story of Test Pilot. There's something here about content that TP is generating in terms of an approach to ideation and experimenting</li></ul> | ||
|- | |- | ||
| Line 40: | Line 40: | ||
| KR3 | | KR3 | ||
| Create a draft curriculum for an upcoming class with the Open Innovation team | | Create a draft curriculum for an upcoming class with the Open Innovation team | ||
| | | <span style="color:orange;font-weight:bold;">0.8</span> | ||
| | | Lots of planning with the Ravensbourne group. We have two Design Prompts written which will be the foundation of the course. | ||
|- | |- | ||
| | | | ||
| Line 65: | Line 65: | ||
| KR3 | | KR3 | ||
| Plan a mobile GTM for 2017 | | Plan a mobile GTM for 2017 | ||
| | |<span style="color:red;font-weight:bold;">0.0</span> | ||
| This deliverable is a document outlining the plan. | | This deliverable is a document outlining the plan. | ||
|- | |- | ||
Revision as of 18:18, 29 March 2017
Test Pilot 2017 Q1 Objectives and Key Results
| Final Status (Mar 31) | Comments | ||
|---|---|---|---|
| Objective 1 | Demonstrate Test Pilot experiments can result in changes to Firefox | ||
| KR1 | Two former Test Pilot experiments in active SHIELD studies | 0.5 | This was supposed to be Activity Stream and Tab Center. Tab Center hit a snag landing the system add-on and is delayed until early/mid Q2. In retrospect, we shouldn't have had this on our goal list since we don't own it. |
| Objective 2 | Be company role models for community involvement | ||
| KR1 | Test 3 contributor value hypotheses and publish the results | 0.15 | Lots of planning towards this. Three finalists:
|
| KR2 | Publish conclusions from three experiments as they graduate from Test Pilot | 0.67 | No More 404s and Tracking Protection |
| KR3 | Create a draft curriculum for an upcoming class with the Open Innovation team | 0.8 | Lots of planning with the Ravensbourne group. We have two Design Prompts written which will be the foundation of the course. |
| Objective 3 | Grow Test Pilot's user base and capabilities | ||
| KR1 | At the end of Q1 double the percentage of new sessions enabling an experiment on intial Test Pilot install | 0.61 | Q4 2016 was 6.6% . source; page 3, "% of New Sessions Enabling an Exp". There is a new dashboard which puts this number at 10.9%, with some qualifiers: If you look at the performance by traffic medium, you'll see 65% of our traffic was from snippets, but snippets have the lowest percentage of sessions enabling experiments (7.4%). That means any time we run a snippet campaign we tank this number, so basing it off of a percentage of new visitors was a bad idea and not a great OKR. The proper way to do this would have been an A/B test. .61 credit for the increase in the number, just realize that it's heavily weighted by other forces. |
| KR2 | Improve Smoothed Test Pilot Engagement Ratio by 6% | 0.17 | Average of the entirety of Q4 is .53 so our target was .59. Average for Q1 turned out to be .54, up 1% and 1/6th of our goal. From: source; second graph down "TxP Active Users / TxP Smoothed Engagement Ratio". Click View Source, change the dates to >90 days to include the whole quarter, download the CSV, ensure a quarter's worth of data for TxP add-on, run Average() on the smoothed ratio. |
| KR3 | Plan a mobile GTM for 2017 | 0.0 | This deliverable is a document outlining the plan. |
| KR4 | Create 4 feature cards ready for UX inspired by user testing reports | Ready for UX cards listed here |