Confirmed users, Bureaucrats and Sysops emeriti
1,737
edits
Test Pilot/2016Q4: Difference between revisions
Jump to navigation
Jump to search
m
no edit summary
m (→Updates) |
mNo edit summary |
||
| Line 15: | Line 15: | ||
| KR1 | | KR1 | ||
| 30% improvement in Test Pilot experiment enable rate for new users | | 30% improvement in Test Pilot experiment enable rate for new users | ||
| | | <span style="color:orange;font-weight:bold;">0.50</span> | ||
| Starting rate is 8.02% which makes our target 10.42%. | | Starting rate is 8.02% which makes our target 10.42%. On Dec 31 the average for the quarter was 8.27% although the average for just December was 12.60% so we definitely did some good, we just didn't have long to measure the change. I'm going to call this 50% due to the short measurement period but the positive longterm outlook going into Q1. | ||
|- | |- | ||
| KR2 | | KR2 | ||
| Line 35: | Line 35: | ||
! Objective 2 | ! Objective 2 | ||
! Provide evidence that Test Pilot is useful in helping to make product decisions | ! Provide evidence that Test Pilot is useful in helping to make product decisions | ||
! | ! .44 | ||
! | ! | ||
|- | |- | ||
| KR1 | | KR1 | ||
| Publish conclusions from three experiments and graduate them from Test Pilot | | Publish conclusions from three experiments and graduate them from Test Pilot | ||
| | | <span style="color:orange;font-weight:bold;">0.33</span> | ||
| | | [https://testpilot.firefox.com/experiments/universal-search Universal Search] | ||
|- | |- | ||
| KR2 | | KR2 | ||
| Launch two new experiments | | Launch two new experiments | ||
| | | <span style="color:red;font-weight:bold;">0.0</span> | ||
| | | Closest was Cliqz, but that got bumped to Jan. Snoozetabs was functional, but quite a bit left on polish. | ||
|- | |- | ||
| KR3 | | KR3 | ||
| Build an offboarding process | | Build an offboarding process | ||
| | | <span style="color:green;font-weight:bold;">1.0</span> | ||
| | | Universal Search graduated. | ||
|- | |- | ||
| | | | ||
| Line 60: | Line 60: | ||
! Objective 3 | ! Objective 3 | ||
! Have the ability to run experiments outside of the US Market | ! Have the ability to run experiments outside of the US Market | ||
! | ! .5 | ||
! | ! | ||
|- | |- | ||
| KR1 | | KR1 | ||
| Add localization support to Test Pilot and get 80% of users using Test Pilot in their first choice language. | | Add localization support to Test Pilot and get 80% of users using Test Pilot in their first choice language. | ||
| | | <span style="color:green;font-weight:bold;">1.0</span> | ||
| | | Averaged over Q4 en-US is 85% of requests so calling this green. Note that at the end of December en-US is down to 74.71% of requests. | ||
|- | |- | ||
| KR2 | | KR2 | ||
| Launch at least one locale-specific experiment in one non-English locale | | Launch at least one locale-specific experiment in one non-English locale | ||
| | | <span style="color:red;font-weight:bold;">0.0</span> | ||
| | | This was Cliqz which got bumped to Jan. | ||
|} | |} | ||