11
edits
Changes
First version for the document
= Leanplum Contextual Hints Test Plan (iOS & Android) - July 2017 =
; '''Goal:'''
With the initial implementation of the Leanplum SDK into our mobile browser products, we want to start A/B testing contextual hints to increase retention.
; '''Overarching Hypothesis:'''
We believe that by promoting contextual hints that helps users explore and change settings within the browser, we will drive DAUs and retention.
==iOS==
; '''iOS Hypothesis 1:'''
We believe that by promoting Focus to our privacy conscious users inside Firefox, we will increase installs and DAUs in Focus, and increase retention on both apps (We know that users who have Focus and Firefox installed retain better (Adjust)).
; '''iOS Hypothesis 2:'''
We believe that by promoting Sync to our users inside Firefox, we will increase DAUs and retention as Firefox Account data shows that users with 2 synced devices retain longer.
{| class="wikitable"
|-
! Hyp. # !! Message Name !! Trigger !! Deep Link !! Message Type !! Event & Timing !! Targets !! Result
|-
| H1 || Clear Private Data - Focus Promotion || Clear private data || Focus Adjust link || Banner || Clear Private Data once || Test: en-USPost-test: en, en-gb, de, zh-tw || N/A
|-
| H1 || Opened New Tab - Focus Promotion - Promote Default Browser || Open new tab || Focus Adjust link || Banner || Open new tab once || Test: en-USPost-test: en, en-gb, de, zh-tw || N/A
|-
| H2 || Open Bookmark - Sync Promotion || Open bookmark || firefox://open-url?deep-link=settings/fxa || Banner || Open bookmark once || Test: en-USPost-test: en, en-gb, de, zh-tw || N/A
|-
| H3 || Save Bookmark - Sync Promotion || Save bookmark || firefox://sign_up || Dialog/Banner || Save bookmark once || Test: en-USPost-test: en, en-gb, de, zh-tw || N/A
|}
==Android==
; '''Android Hypothesis 1:'''
We believe that by prompting users to change their default browser to Firefox, we will increase DAUs and retention.
; '''Android Hypothesis 2:'''
We believe that by promoting add-ons to our users inside Firefox, we will increase DAUs and retention.
; '''Android Hypothesis 3: '''
We believe that by promoting Sync to our users inside Firefox, we will increase DAUs and retention as Firefox Account data shows that users with 2 synced devices retain longer.
{| class="wikitable"
|-
! Hyp. # !! Message Name !! Trigger !! Deep Link !! Message Type !! Event & Timing !! Targets !! Result
|-
| H1 || Launch Promote - Default Browser || App Start || firefox://default_browser || Dialog || User starts app second time (only once)VsUser starts app third time (only once)One event a week or 2 weeks - ask Evan || Test: en-USPost-test: en, en-gb, en-ca, de, zh-tw inUS, CA, AU, UK, DE, TW, HK, SG || N/A
|-
| H1 || Load Bookmark - Promote Default Browser || Load bookmark || firefox://default_browser || Dialog || Load Bookmark (1st time) oncevsLoad Bookmark (2 times) onceVsLoad Bookmark (5 times) once || Test: en-USPost-test: en, en-gb, en-ca, de, zh-tw inUS, CA, AU, UK, DE, TW, HK, SG || N/A
|-
| H2 || Firefox launches - add-ons promotion || App start || http://addons.mozilla.org || Dialog || User starts app 2nd time (only once)One event a week or 2 weeks || Test: en-US Post-test: en, en-gb, en-ca, de, zh-tw in US, CA, AU, UK, DE, TW, HK, SG || N/A
|-
| H3 || Save Password - Promote Sync || Save password || firefox://sign_up || Dialog/Banner || Save Password once, banner shows up the next time you launch app || Test: en-USPost-test: en, en-gb, en-ca, de, zh-tw inUS, CA, AU, UK, DE, TW, HK, SG || N/A
|}
; '''Goal:'''
With the initial implementation of the Leanplum SDK into our mobile browser products, we want to start A/B testing contextual hints to increase retention.
; '''Overarching Hypothesis:'''
We believe that by promoting contextual hints that helps users explore and change settings within the browser, we will drive DAUs and retention.
==iOS==
; '''iOS Hypothesis 1:'''
We believe that by promoting Focus to our privacy conscious users inside Firefox, we will increase installs and DAUs in Focus, and increase retention on both apps (We know that users who have Focus and Firefox installed retain better (Adjust)).
; '''iOS Hypothesis 2:'''
We believe that by promoting Sync to our users inside Firefox, we will increase DAUs and retention as Firefox Account data shows that users with 2 synced devices retain longer.
{| class="wikitable"
|-
! Hyp. # !! Message Name !! Trigger !! Deep Link !! Message Type !! Event & Timing !! Targets !! Result
|-
| H1 || Clear Private Data - Focus Promotion || Clear private data || Focus Adjust link || Banner || Clear Private Data once || Test: en-USPost-test: en, en-gb, de, zh-tw || N/A
|-
| H1 || Opened New Tab - Focus Promotion - Promote Default Browser || Open new tab || Focus Adjust link || Banner || Open new tab once || Test: en-USPost-test: en, en-gb, de, zh-tw || N/A
|-
| H2 || Open Bookmark - Sync Promotion || Open bookmark || firefox://open-url?deep-link=settings/fxa || Banner || Open bookmark once || Test: en-USPost-test: en, en-gb, de, zh-tw || N/A
|-
| H3 || Save Bookmark - Sync Promotion || Save bookmark || firefox://sign_up || Dialog/Banner || Save bookmark once || Test: en-USPost-test: en, en-gb, de, zh-tw || N/A
|}
==Android==
; '''Android Hypothesis 1:'''
We believe that by prompting users to change their default browser to Firefox, we will increase DAUs and retention.
; '''Android Hypothesis 2:'''
We believe that by promoting add-ons to our users inside Firefox, we will increase DAUs and retention.
; '''Android Hypothesis 3: '''
We believe that by promoting Sync to our users inside Firefox, we will increase DAUs and retention as Firefox Account data shows that users with 2 synced devices retain longer.
{| class="wikitable"
|-
! Hyp. # !! Message Name !! Trigger !! Deep Link !! Message Type !! Event & Timing !! Targets !! Result
|-
| H1 || Launch Promote - Default Browser || App Start || firefox://default_browser || Dialog || User starts app second time (only once)VsUser starts app third time (only once)One event a week or 2 weeks - ask Evan || Test: en-USPost-test: en, en-gb, en-ca, de, zh-tw inUS, CA, AU, UK, DE, TW, HK, SG || N/A
|-
| H1 || Load Bookmark - Promote Default Browser || Load bookmark || firefox://default_browser || Dialog || Load Bookmark (1st time) oncevsLoad Bookmark (2 times) onceVsLoad Bookmark (5 times) once || Test: en-USPost-test: en, en-gb, en-ca, de, zh-tw inUS, CA, AU, UK, DE, TW, HK, SG || N/A
|-
| H2 || Firefox launches - add-ons promotion || App start || http://addons.mozilla.org || Dialog || User starts app 2nd time (only once)One event a week or 2 weeks || Test: en-US Post-test: en, en-gb, en-ca, de, zh-tw in US, CA, AU, UK, DE, TW, HK, SG || N/A
|-
| H3 || Save Password - Promote Sync || Save password || firefox://sign_up || Dialog/Banner || Save Password once, banner shows up the next time you launch app || Test: en-USPost-test: en, en-gb, en-ca, de, zh-tw inUS, CA, AU, UK, DE, TW, HK, SG || N/A
|}
