Leanplum Contextual Hints

From MozillaWiki
Jump to: navigation, search

Test Plan (iOS & Android) - July 2017

Goal:

With the initial implementation of the Leanplum SDK into our mobile browser products, we want to start A/B testing contextual hints to increase retention.

Overarching Hypothesis:

We believe that by promoting contextual hints that helps users explore and change settings within the browser, we will drive DAUs and retention.

iOS

iOS Hypothesis 1:

We believe that by promoting Focus to our privacy conscious users inside Firefox, we will increase installs and DAUs in Focus, and increase retention on both apps (We know that users who have Focus and Firefox installed retain better (Adjust)).

iOS Hypothesis 2:

We believe that by promoting Sync to our users inside Firefox, we will increase DAUs and retention as Firefox Account data shows that users with 2 synced devices retain longer.

iOS Hypothesis 3:

We believe that by promoting tips and tricks such as customizing settings to our users inside Firefox, we will increase DAUs and retention.

iOS Hypothesis 4:

We believe that by promoting Pocket to our users inside Firefox, we will increase overall DAUs and retention of the Pocket app.

Hyp. # Message Name Trigger Targets
H1 Opened New Tab - Focus Promotion Open new tab Test: en-US 10% of release population, Post-test: en, en-gb, de, zh-tw
H2 Open Bookmark - Sync Promotion Open bookmark Test: en-US 10% of release population, Post-test: en, en-gb, de, zh-tw
H2 Save Bookmark - Sync Promotion Save bookmark Test: en-US 10% of release population, Post-test: en, en-gb, de, zh-tw
H2 Save Login and Password - Sync Promotion Save bookmark Test: en-US 10% of release population, Post-test: en, en-gb, de, zh-tw
H3 Search URL Area - Default Search Engine Promotion Interacts with search url area Test: en-US 10% of release population, Post-test: en, en-gb, de, zh-tw
H3 Search URL Area - Homepage Setting Promotion Interacts with search url area Test: en-US 10% of release population, Post-test: en, en-gb, de, zh-tw
H3 Open New Tab - New Tab Setting Promotion Open new tab Test: en-US 10% of release population, Post-test: en, en-gb, de, zh-tw
H4 Open Bookmark - Pocket Promotion Open bookmark Test: en-US 10% of release population, Post-test: en, en-gb, de, zh-tw

Android

Android Hypothesis 1:

We believe that by prompting users to change their default browser to Firefox, we will increase DAUs and retention.

Android Hypothesis 2:

We believe that by promoting add-ons to our users inside Firefox, we will increase DAUs and retention.

Android Hypothesis 3:

We believe that by promoting Sync to our users inside Firefox, we will increase DAUs and retention as Firefox Account data shows that users with 2 synced devices retain longer.

Android Hypothesis 4:

We believe that by promoting tips and tricks like the bookmark list or search engine settings to our users inside Firefox, we will increase DAUs and retention.

Android Hypothesis 5:

We believe that by promoting Focus to our privacy conscious users inside Firefox, we will increase installs and DAUs in Focus, and increase retention on both apps (We know that iOS users who have Focus and Firefox installed retain better (Adjust)).


Hyp. # Message Name Trigger Targets
H1 Launch app - Default Browser promotion App Start Test: en-US 10% of release population, Post-test: en, en-gb, en-ca, de, zh-tw inUS, CA, AU, UK, DE, TW, HK, SG
H1 Load Bookmark - Default Browser promotion Load bookmark Test: en-US 10% of release population, Post-test: en, en-gb, en-ca, de, zh-tw inUS, CA, AU, UK, DE, TW, HK, SG
H2 Launch app - Add-ons promotion App start Test: en-US 10% of release population, Post-test: en, en-gb, en-ca, de, zh-tw in US, CA, AU, UK, DE, TW, HK, SG
H5 Open New Tab - Focus promotion Open new tab Test: en-US 10% of release population, Post-test: en, en-gb, en-ca, de, zh-tw inUS, CA, AU, UK, DE, TW, HK, SG
H4 Save Bookmark - Bookmark List Promotion Save bookmark Test: en-US 10% of release population, Post-test: en, en-gb, en-ca, de, zh-tw inUS, CA, AU, UK, DE, TW, HK, SG
H1 Log in to FxA - Default Browser Promotion Log in to FxA Test: en-US 10% of release population, Post-test: en, en-gb, en-ca, de, zh-tw inUS, CA, AU, UK, DE, TW, HK, SG
H1 Add Bookmark 5 times - Default Browser Promotion Add Bookmark 5 times Test: en-US 10% of release population, Post-test: en, en-gb, en-ca, de, zh-tw inUS, CA, AU, UK, DE, TW, HK, SG
H3 Save Bookmark - Sync Promotion Save bookmark Test: en-US 10% of release population, Post-test: en, en-gb, en-ca, de, zh-tw inUS, CA, AU, UK, DE, TW, HK, SG
H4 Search URL Area - Default Search Engine Promotion Interact with search url area Test: en-US 10% of release population, Post-test: en, en-gb, en-ca, de, zh-tw inUS, CA, AU, UK, DE, TW, HK, SG

For more detail about the triggers and deep links in Android, please go to this page[1]