Test Pilot/2017Q4: Difference between revisions
Jump to navigation
Jump to search
(Created page with " Test Pilot 2017 Q4 Objectives and Key Results {| class="wikitable" ! ! ! Final Status (Dec 31) ! Comments |- ! Objective 1 ! Improve Test Pilot so that the Test Pilot prog...") |
mNo edit summary |
||
| Line 10: | Line 10: | ||
! Objective 1 | ! Objective 1 | ||
! Improve Test Pilot so that the Test Pilot program is a pleasure to use and improves engagement with experiments. | ! Improve Test Pilot so that the Test Pilot program is a pleasure to use and improves engagement with experiments. | ||
! | ! <span style="color:green;font-weight:bold;">1</span> | ||
! | ! | ||
|- | |- | ||
| KR1 | | KR1 | ||
| Implement 3 new features inspired by user feedback or x-functional need | | Implement 3 new features inspired by user feedback or x-functional need | ||
| | | <span style="color:green;font-weight:bold;">1</span> | ||
| | | Done: Photonization, Componetization, and the Featured Experiment Hero area. | ||
|- | |- | ||
| KR2 | | KR2 | ||
| Produce a document defining process and interactions with teams | | Produce a document defining process and interactions with teams | ||
| | | <span style="color:green;font-weight:bold;">1</span> | ||
| | | [https://docs.google.com/spreadsheets/d/1ma8o8Qi436q1hjV3YaB1otmBdQCe5zdG29WsHbuOL9c/edit giant RACI document] | ||
|- | |- | ||
| KR3 | | KR3 | ||
| Publish 12 blog posts on the Test Pilot blog | | Publish 12 blog posts on the [https://medium.com/firefox-test-pilot Test Pilot blog] | ||
| | | <span style="color:green;font-weight:bold;">1</span> | ||
| | | 12 done. Might even get 13! | ||
|- | |- | ||
| | | | ||
| Line 35: | Line 35: | ||
! Objective 2 | ! Objective 2 | ||
! Grow Screenshots usage and expand offered features so that people find Screenshots to be a useful tool. | ! Grow Screenshots usage and expand offered features so that people find Screenshots to be a useful tool. | ||
! | ! <span style="color:red;font-weight:bold;">.43</span> | ||
! | ! | ||
|- | |- | ||
| KR1 | | KR1 | ||
| Implement 3 new features inspired by user feedback | | Implement 3 new features inspired by user feedback | ||
| | | <span style="color:orange;font-weight:bold;">.67</span> | ||
| | | Finished highly requested features: copy-image-to-clipboard and download-only mode. Didn't quite get Annotations shipped. | ||
|- | |- | ||
| KR2 | | KR2 | ||
| Publish results from 3 A/B tests | | Publish results from 3 A/B tests | ||
| | | <span style="color:red;font-weight:bold;">0</span> | ||
| | | A/B tests written, but ran out of time to ship them. No results published yet. | ||
|- | |- | ||
| KR3 | | KR3 | ||
| Get 5000 total pieces of feedback from survey | | Get 5000 total pieces of feedback from survey | ||
| | | <span style="color:red;font-weight:bold;">0</span> | ||
| | | Survey written, but ran out of time to ship them. | ||
|- | |- | ||
| KR4 | | KR4 | ||
| Deliver a document planning Screenshots interaction with Firefox | | Deliver a document planning Screenshots interaction with Firefox | ||
| | | <span style="color:green;font-weight:bold;">1</span> | ||
| | | [https://docs.google.com/document/d/1WA8yQPhcVK8hdrQD0hV_e0AqBOwd2Q9Nzv882QGjzng/edit Document finished] although the plan will evolve with the committee that came out of the all hands github+relman meeting. | ||
|- | |- | ||
| KR5 | | KR5 | ||
| Demonstrate via a dashboard that Screenshots is within RAIL guidelines | | Demonstrate via a dashboard that Screenshots is within RAIL guidelines | ||
| | | <span style="color:orange;font-weight:bold;">.5</span> | ||
| | | We instrumented the add-on in Firefox and [https://datastudio.google.com/org/Ra9O2QIYT8We-1pOmnS_6w/reporting/0ByaR4oZsXEGSVFRzVVA4MEpwR0U/page/TmtK made a dashboard] but are not within RAIL guidelines. Calling this halfway. | ||
|- | |- | ||
| | | | ||
| Line 70: | Line 70: | ||
! Objective 3 | ! Objective 3 | ||
! Find, guide, and graduate ideas and experiments into features so that Test Pilot is a proven valuable input to the future direction of Firefox | ! Find, guide, and graduate ideas and experiments into features so that Test Pilot is a proven valuable input to the future direction of Firefox | ||
! | ! <span style="color:green;font-weight:bold;">.93</span> | ||
! | ! | ||
|- | |- | ||
| KR1 | | KR1 | ||
| In Firefox Send, do a total of four new features or new A/B tests | | In Firefox Send, do a total of four new features or new A/B tests | ||
| | | <span style="color:green;font-weight:bold;">1</span> | ||
| | | Shipped password support, and multi-downloads. Two A/B tests around Firefox promotions. | ||
|- | |- | ||
| KR2 | | KR2 | ||
| Complete UX spec for Tab Split | | Complete UX spec for Tab Split | ||
| | | <span style="color:green;font-weight:bold;">1</span> | ||
| | | [https://mozilla.invisionapp.com/share/7FE2XCUCP#/screens/252448093_A-1_TabSplit done] | ||
|- | |- | ||
| KR3 | | KR3 | ||
| Deliver a working prototype of the Theme Experiment | | Deliver a working prototype of the Theme Experiment | ||
| | | <span style="color:green;font-weight:bold;">1</span> | ||
| | | [https://lmorchard.github.io/ThemesRFun/ done] | ||
|- | |- | ||
| KR4 | | KR4 | ||
| Deliver a document planning a design sprint in Q1 | | Deliver a document planning a design sprint in Q1 | ||
| | | <span style="color:green;font-weight:bold;">1</span> | ||
| | | [https://docs.google.com/document/d/1QZPRWKifN-0Nq8bZsXoGkYhS46VQtAeId71JoOjUY14/edit done] | ||
|- | |- | ||
| KR5 | | KR5 | ||
| Publish graduation reports for four experiments | | Publish graduation reports for four experiments | ||
| | | <span style="color:orange;font-weight:bold;">.65</span> | ||
| | | [https://medium.com/firefox-test-pilot/firefox-test-pilot-pulse-graduation-a3dc2212cce0 Pulse], Activity Stream (going live tomorrow morning. Add a link if you're reading this plz). Sharon has a .15 worth of a draft written for Snooze Tabs also which will be published soon. | ||
|- | |- | ||
| | | | ||
Latest revision as of 00:15, 22 December 2017
Test Pilot 2017 Q4 Objectives and Key Results
| Final Status (Dec 31) | Comments | ||
|---|---|---|---|
| Objective 1 | Improve Test Pilot so that the Test Pilot program is a pleasure to use and improves engagement with experiments. | 1 | |
| KR1 | Implement 3 new features inspired by user feedback or x-functional need | 1 | Done: Photonization, Componetization, and the Featured Experiment Hero area. |
| KR2 | Produce a document defining process and interactions with teams | 1 | giant RACI document |
| KR3 | Publish 12 blog posts on the Test Pilot blog | 1 | 12 done. Might even get 13! |
| Objective 2 | Grow Screenshots usage and expand offered features so that people find Screenshots to be a useful tool. | .43 | |
| KR1 | Implement 3 new features inspired by user feedback | .67 | Finished highly requested features: copy-image-to-clipboard and download-only mode. Didn't quite get Annotations shipped. |
| KR2 | Publish results from 3 A/B tests | 0 | A/B tests written, but ran out of time to ship them. No results published yet. |
| KR3 | Get 5000 total pieces of feedback from survey | 0 | Survey written, but ran out of time to ship them. |
| KR4 | Deliver a document planning Screenshots interaction with Firefox | 1 | Document finished although the plan will evolve with the committee that came out of the all hands github+relman meeting. |
| KR5 | Demonstrate via a dashboard that Screenshots is within RAIL guidelines | .5 | We instrumented the add-on in Firefox and made a dashboard but are not within RAIL guidelines. Calling this halfway. |
| Objective 3 | Find, guide, and graduate ideas and experiments into features so that Test Pilot is a proven valuable input to the future direction of Firefox | .93 | |
| KR1 | In Firefox Send, do a total of four new features or new A/B tests | 1 | Shipped password support, and multi-downloads. Two A/B tests around Firefox promotions. |
| KR2 | Complete UX spec for Tab Split | 1 | done |
| KR3 | Deliver a working prototype of the Theme Experiment | 1 | done |
| KR4 | Deliver a document planning a design sprint in Q1 | 1 | done |
| KR5 | Publish graduation reports for four experiments | .65 | Pulse, Activity Stream (going live tomorrow morning. Add a link if you're reading this plz). Sharon has a .15 worth of a draft written for Snooze Tabs also which will be published soon. |
Updates
Non-Goals
- We elected not to have any retention related goals because it didn't seem like we have enough time in a quarter to decide on a change, implement it, and measure it for any meaningful amount of time.