Latest revision as of 16:42, 21 January 2016

Goals

The FxOS Metrics project enables making data driven decisions for the following:

Understanding how FxOS is doing in the market versus product/business goals
Gaining insights from quantitative and qualitative data, to enable improve the product/business goals
Understanding usage models, to improve future product development and opportunities
Determining effectiveness of marketing campaigns.

Metrics

Customer lifecycle	Metrics to be collected	Dashboard
Acquiring Customers	Partners/Channels though which devices are sold	Dashboard: Choose Operator/OEM/Country combination of channel
Activation of Devices	Total number of Activated devices	Dashboard: Choose 'All' in SW/Operator/OEM/ to get a count
Retention Of Customers	MAU (Monthly Active Users) Apps usage per month Performance (App Startup time, Power consumption)	Internal
Revenue	Number of searches done Paid App Downloads	Internal
Referral	Net Promoter Score Recommendations from Users User Satisfaction Factor	Future

Roadmap

FxOS Release	Metrics to be collected	Bugs
1.3	Acquisition, Activations: Activation Ping:	Engg: 986701,997960 Legal: 980920 Privacy: 992487
2.1	Retention App Usage (Default Off)	Client: 982663 Legal: 1021259, 1109429
2.2	Retention App Usage (Default On) AppUsage, MAU Acquisition: Report Preinstalled Apps Revenue Num of Searches/Revenue	AppUsage, MAU: 1109422 Client side work for enabling AppUsage , 1119009 Marketplace Apps Only 1109426 Server side dashboard for Most used apps: 1021173 Report Preinstalled Apps, 1126524 Search
2.5	Retention: OTA collection of App Startup time (1179004). Activation Accuracy of activation: FTU Ping: Avoid counting repeat pings from a known device. (1171685). Privacy: Converged UX for opt-in, opt-out of metrics. 1181295. Scaling: Framework (Metabug: 1152000) to collect custom Metrics and histogram Non Phone devices support: Using TV serial number to avoid counting repeat pings. 1171685. B2GDroid Support (B2GDroid Metric Collection 1198338) AddOns Support (AddOns Metric collection 1198346)
2.6 (May 2016)	(In Planning Meta Bug for 2.6 (1226182)) Activation Optimizing FTU: % of signups to Moz services: FxAccounts, FMD Accuracy of Retention/MAU: FTU ping should include info on whether User opt-in for metrics (1198164) Retention Support for progressive web apps (1226458) Make uuid available to Foxfooder app (1226274) Inclusion of device metrics in A/B testing (likely to be moved beyond 2.5) Engagement with device: Number of voice calls, data (wifi and cellular), number of text messages. 1193225. Device Startup time: OTA collection of device dtartup time (1177226) Advanced A/B testing Churn Rate & User Lifecycle Analysis (In planning) Framework Improvements Enum Type and Boolean Type histograms Support for New Security model 1225960 FoTA Ping Engg bug: 1202970, Legal bug: 1202987. Referral Net Promoter Score (User Satisfaction Factor) Privacy User ability to delete collected info. 1126513. Server Graphing of Addon Histograms Graphing of Keyed Histograms
End-2016 (In Planning)	Retention Content Recommendation Engine based on usage (User Benefit)

User Stories

User Stories

FoxFooding

Data gathered for the FoxFooding program

Virtual Team

Device Engg: Tamara Hills, Gregor Wagner, Russ Nicoletti
Device QA: Shing Lyu, Iris Hsiao, Gerry Chang
Server Metrics: Dave Zeber, John Jensen
Planning: Dominik Strohmeier, Maria Oteo
Product Mgmt: Peter Dolanjski, Maria Oteo
Alumni: Marshall Culpepper, Hema Koka

@@ Line 27: / Line 27: @@
 *MAU (Monthly Active Users)
 *Apps usage per month
-|In Progress
+*Performance (App Startup time, Power consumption)
+|Internal
 |-
 |Revenue
@@ Line 33: / Line 34: @@
 *Number of searches done
 *Paid App Downloads
-|In Progress
+|Internal
 |-
 |Referral
 |
-*Recommendations from Users
+*Net Promoter Score
+**Recommendations from Users
+**User Satisfaction Factor
 |Future
 |}
@@ Line 48: / Line 51: @@
 |1.3
 |
-*[https://wiki.mozilla.org/Firefox_OS/Metrics/activationping Activation Ping]:
+*Acquisition, Activations:
-**Acquisition
+**[https://wiki.mozilla.org/Firefox_OS/Metrics/activationping Activation Ping]:
-**Activations
 |
-Engg: 986701,997960
+*Engg: 986701,997960
-Legal: 980920
+*Legal: 980920
-Privacy: 992487
+*Privacy: 992487
 |-
 |2.1
 |
-*[https://wiki.mozilla.org/FirefoxOS/Metrics/App_Usage App Usage (Default Off)]
+*Retention
+**[https://wiki.mozilla.org/FirefoxOS/Metrics/App_Usage App Usage (Default Off)]
 |
 *Client: 982663
@@ Line 65: / Line 68: @@
 |2.2
 |
-*[https://wiki.mozilla.org/FirefoxOS/Metrics/App_Usage App Usage (Default On)]
+*Retention
-**[https://wiki.mozilla.org/FirefoxOS/Metrics/App_Usage AppUsage],
+**[https://wiki.mozilla.org/FirefoxOS/Metrics/App_Usage App Usage (Default On)]
-**[https://docs.google.com/a/mozilla.com/document/d/1i9CwaeLsOM-XS1Z8UZ88-b03CkPyxHI07sLhcZ9z4EY/edit#heading=h.lqni56rfu030 MAU]
+***[https://wiki.mozilla.org/FirefoxOS/Metrics/App_Usage AppUsage],
-*[https://bugzilla.mozilla.org/1021173 Report Preinstalled Apps, Acquisition]
+***[https://docs.google.com/a/mozilla.com/document/d/1i9CwaeLsOM-XS1Z8UZ88-b03CkPyxHI07sLhcZ9z4EY/edit#heading=h.lqni56rfu030 MAU]
-*[https://bugzilla.mozilla.org/show_bug.cgi?id=1126524 Num of Searches/Revenue ]
+*Acquisition:
+**[https://bugzilla.mozilla.org/1021173 Report Preinstalled Apps]
+*Revenue
+**[https://bugzilla.mozilla.org/show_bug.cgi?id=1126524 Num of Searches/Revenue ]
 |
 *[https://wiki.mozilla.org/FirefoxOS/Metrics/App_Usage AppUsage], MAU:
@@ Line 78: / Line 84: @@
 *[https://bugzilla.mozilla.org/1126524 1126524 Search ]
 |-
-|3.0 & Future
+|2.5
 |
-*FxOS Health Report:
+*Retention:
-**Usage of data, calls, text
+**[https://bugzilla.mozilla.org/1179004 OTA collection of App Startup time (1179004)].
-**Performance
+*Activation
-**Number of Signups for FMD, FxAccounts
+**Accuracy of activation: [https://bugzilla.mozilla.org/1171685 FTU Ping: Avoid counting repeat pings from a known device. (1171685).]
-**User Behavior API exposed to some Apps
+*Privacy:
-**[https://bugzilla.mozilla.org/show_bug.cgi?id=126513 126513: User ability to delete collected info]
+**[https://bugzilla.mozilla.org/1181295 Converged UX for opt-in, opt-out of metrics. 1181295.]
+*Scaling:
+**[https://bugzilla.mozilla.org/1152000 Framework (Metabug: 1152000) to collect custom Metrics and histogram]
+**Non Phone devices support: [https://bugzilla.mozilla.org/1171685 Using TV serial number to avoid counting repeat pings. 1171685.]
+**B2GDroid Support ([https://bugzilla.mozilla.org/show_bug.cgi?id=1198338 B2GDroid Metric Collection 1198338])
+**AddOns Support ([https://bugzilla.mozilla.org/1198346: AddOns Metric collection 1198346])
+|
+|-
+|2.6 (May 2016)
+| (In Planning [https://bugzilla.mozilla.org/show_bug.cgi?id=1226182 Meta Bug for 2.6 (1226182)])
+*Activation
+**Optimizing FTU: % of signups to Moz services: FxAccounts, FMD
+**Accuracy of Retention/MAU: [https://bugzilla.mozilla.org/1198164 FTU ping should include info on whether User opt-in for metrics (1198164)]
+*Retention
+**[https://bugzilla.mozilla.org/show_bug.cgi?id=1226458 Support for progressive web apps (1226458)]
+**[https://bugzilla.mozilla.org/show_bug.cgi?id=1226274 Make uuid available to Foxfooder app (1226274)]
+**Inclusion of device metrics in A/B testing (likely to be moved beyond 2.5)
+**Engagement with device: [https://bugzilla.mozilla.org/1193225 Number of voice calls, data (wifi and cellular), number of text messages. 1193225.]
+** Device Startup time: [https://bugzilla.mozilla.org/1177226 OTA collection of device dtartup time (1177226)]
+**Advanced A/B testing
+**Churn Rate & User Lifecycle Analysis (In planning)
+**Framework Improvements
+***Enum Type and Boolean Type histograms
+**Support for New Security model [https://bugzilla.mozilla.org/show_bug.cgi?id=1225960 1225960]
+**FoTA Ping [https://bugzilla.mozilla.org/show_bug.cgi?id=1202970 Engg bug: 1202970], [https://bugzilla.mozilla.org/show_bug.cgi?id=1202987. Legal bug: 1202987].
+*Referral
+**Net Promoter Score (User Satisfaction Factor)
+*Privacy
+**[https://bugzilla.mozilla.org/1126513 User ability to delete collected info. 1126513.]
+*Server
+**[https://bugzilla.mozilla.org/show_bug.cgi?id=1226062 Graphing of Addon Histograms]
+**[https://bugzilla.mozilla.org/show_bug.cgi?id=1226064 Graphing of Keyed Histograms]
+|-
+|End-2016 (In Planning)
+|
+*Retention
+**Content Recommendation Engine based on usage (User Benefit)
 |
 |}
@@ Line 91: / Line 133: @@
 =User Stories=
 *[https://docs.google.com/a/mozilla.com/spreadsheet/ccc?key=0AmVPziKCVEUEdGFoczVCaHBzOGxQcXNxWWs3aWRSalE#gid=0 User Stories]
+= FoxFooding =
+[https://wiki.mozilla.org/FirefoxOS/Metrics/FoxFooding Data gathered for the FoxFooding program]
 =Virtual Team=
-*Device Engg: Marshall Culpepper, Tamara Hills, Hema Koka, Gregor
+*Device Engg:  Tamara Hills, Gregor Wagner, Russ Nicoletti
-*Device QA: Shing Lyu
+*Device QA: Shing Lyu, Iris Hsiao, Gerry Chang
 *Server Metrics: Dave Zeber, John Jensen
-*Engg Program Mgmt: Michael Treese
+*Planning: Dominik Strohmeier, Maria Oteo
-*Planning: Dominik Strohmeier
+*Product Mgmt: Peter Dolanjski, Maria Oteo
-*Product Mgmt: Ravi Dandu
+*Alumni: Marshall Culpepper, Hema Koka
-*Marketplace Product Mgmt: David Bialer
-*Marketplace Program Mgmt: Stephanie Turner
-*Marketplace Engg: Andy McKay, Rob Hudson
 =Links=
+*[https://wiki.mozilla.org/FirefoxOS/Metrics/CustomMetrics CustomMetrics]
+*[https://wiki.mozilla.org/FirefoxOS/Metrics/EngineeringMetrics EngineeringMetrics]
 *[https://wiki.mozilla.org/FirefoxOS/Metrics/activationping Activation Ping]
 *[https://wiki.mozilla.org/FirefoxOS/Metrics/App_Usage AppUsage Data]
@@ Line 109: / Line 154: @@
 *[https://wiki.mozilla.org/FirefoxOS/Metrics Main Metrics Page (this page)]
 *[https://wiki.mozilla.org/B2G/QA/Ping_Testing Testing Guide]
+*[https://wiki.mozilla.org/Firefox_OS/Metrics/FAQ Frequently Asked Questions]
+*[https://docs.google.com/document/d/1CxpSQzjvV-N3owFHpOWcki9dmNZa874-x2MJp4NxG-g/edit#heading=h.jgf7qkensd70 Performance Goals]

Firefox OS/Metrics: Difference between revisions

Latest revision as of 16:42, 21 January 2016

Contents

Goals

Metrics

Roadmap

User Stories

FoxFooding

Virtual Team

Links

Navigation menu