Firefox/Input/Feedback Strategy: Difference between revisions

From MozillaWiki
< Firefox‎ | Input
Jump to navigation Jump to search
 
(32 intermediate revisions by the same user not shown)
Line 7: Line 7:
{|
{|
|-
|-
| style="width: 100%; background-color: #efefef;" | Source
| style="width: 20%; background-color: #efefef;" | Source
| style="width: 100%; background-color: #efefef;" | Type
| style="width: 60%; background-color: #efefef;" | Type
| style="width: 100%; background-color: #efefef;" | Options
| style="width: 20%; background-color: #efefef;" | Options
|-
|-
| SUMO
| SUMO
Line 84: Line 84:


== Needs ==
== Needs ==
* Central - The value of this channel is to shows bleeding edge features that may or may not end up in the release, but give a glimpse into Firefox's strategy/vision.
* '''Central''' - The value of this channel is to shows bleeding edge features that may or may not end up in the release, but give a glimpse into Firefox's strategy/vision.
* Aurora - Users here will be able to experiment with new features and determine if they match quality criteria to be graduated to a release of Firefox proper.
* '''Aurora''' - Users here will be able to experiment with new features and determine if they match quality criteria to be graduated to a release of Firefox proper.
* Beta - Users will be able to get the inside track on new features landing in Firefox proper and increase the overall quality of the new release by finding compatibility  
* '''Beta''' - Users will be able to get the inside track on new features landing in Firefox proper and increase the overall quality of the new release by finding compatibility  
* Release - We'll look for possible security and  Top Issues and possible Chemspills on this channel.
* '''Release''' - We'll look for possible security and  Top Issues and possible Chemspills on this channel.


{| class="standard-table" style="width: 100%;"
{| class="standard-table" style="width: 100%;"
|-
|-
| class="header" |  
| style="background-color: #efefef;" |  
| class="header" | Regressions
| style="background-color: #efefef;" | Regressions
| class="header" | Stability
| style="background-color: #efefef;" | Stability
| class="header" | Feature Quality
| style="background-color: #efefef;" | Feature Quality
| class="header" | UX Feedback
| style="background-color: #efefef;" | UX Feedback
| class="header" | Website Compat.
| style="background-color: #efefef;" | Website Compat.
| class="header" | Plugin Compat.
| style="background-color: #efefef;" | Plugin Compat.
| class="header" | Addon Compat.
| style="background-color: #efefef;" | Addon Compat.
| class="header" | Chemspills
| style="background-color: #efefef;" | Chemspills
|-
|-
| Central
| style="background-color: #FEF1B5;"| Central
| X
| P2
| X
| P1
|  
|  
|
|
Line 111: Line 111:
|
|
|-
|-
| Aurora
| style="background-color: #FEF1B5;"| Aurora
| X
| P1
| X
| P1
| X
| P1
| X
| P1
| X
| P3
| X
| P2
|
|
|
|
|-
|-
| Beta
| style="background-color: #FEF1B5;"| Beta
| X
| P3
| X
| P1
| X
| P1
| X
| P2
| X
| P1
| X
| P1
| X
| P1
|
|
|-
|-
| Release
| style="background-color: #FEF1B5;"| Release
|  
|  
|  
|  
|  
|  
|
|
| X
| P2
| X
| P1
| X
| P3
| X
| P1
|}
|}


Line 150: Line 150:


=== Central ===
=== Central ===
'''Goal'''
* Show the health (i.e. general stability and features) of builds on the Central channel on a weekly-basis.
'''Stakeholders'''
'''Stakeholders'''
* Product Drivers
* Product Drivers
Line 161: Line 164:
| style="background-color: #efefef;"| '''Time'''
| style="background-color: #efefef;"| '''Time'''
|-
|-
| Firefox Development Meeting
| Firefox Development Meeting ('''reporting''')
| Tues 11:00AM Pacific
| Tues 11:00AM Pacific
|}
|}


<br>'''What to Report'''
<br>'''What to Gather'''


{| width="100%" cellpadding="3"
{| width="100%" cellpadding="3"
Line 171: Line 174:
| style="background-color: #efefef;" | '''Report'''
| style="background-color: #efefef;" | '''Report'''
| style="background-color: #efefef;" | '''Source'''
| style="background-color: #efefef;" | '''Source'''
| style="background-color: #efefef;" | '''Responsibility'''
|-
|-
|  
|  
* Daily triage of happy/sad/ideas per feature
* happy/sad/ideas trending over the past week
|
* [http://input.mozilla.com/en-US/search?q=&product=firefox&version=6.0a1&date_start=2011-05-03&date_end= Input]
|-
|
|
* Input
* Daily top crashers
* Weekly top crashers
|
|
* Feature Manager
* [https://crash-stats.mozilla.com/daily?form_selection=by_version&v&#91;&#93;=6.0a1&throttle&#91;&#93;=100.00 Socorro]
|-
|-
|
|
* Daily triage of top crashers
* Daily number of major (and higher) severity bugs in a UNCO/NEW state with no assignee filed  (with query link)
* Weekly regressions filed in a UNCO/NEW state with no assignee
|
|
* Socorro
* Bugzilla
|
* QA Services
|-
|-
|
|
Line 191: Line 196:
* Weekly change in number of ADU's
* Weekly change in number of ADU's
|
|
* Metrics Dashboard
* [https://metrics.mozilla.com/pentaho/content/pentaho-cdf-dd/Render?solution=metrics&path=%2Fblocklist%2FChannels&file=channelBreakdown.wcdf Metrics Dashboard]
|-
|
* Daily performance numbers and trends
* Weekly performance trend
|
* [http://graphs.mozilla.org Performance Graph]
|-
|
* Startup time
* Miscellaneous Information
|
|
* Metrics/PMM
* Telemetry
|}
|}


=== Aurora ===
=== Aurora ===
'''Goal'''
* Display the user satisfaction and quality levels of specific features on the Aurora channel over the past week.
'''Stakeholders'''
'''Stakeholders'''
* Product Drivers
* Product Drivers
Line 216: Line 234:
|}
|}


<br>'''What to Report'''
<br>'''What to Gather'''


{| width="100%" cellpadding="3"
{| width="100%" cellpadding="3"
Line 222: Line 240:
| style="background-color: #efefef;" | '''Report'''
| style="background-color: #efefef;" | '''Report'''
| style="background-color: #efefef;" | '''Source'''
| style="background-color: #efefef;" | '''Source'''
| style="background-color: #efefef;" | '''Responsibility'''
|-
|-
|  
|  
* Weekly per-new-feature happy/sad/ideas per feature
* Daily triage of happy/sad/ideas per feature (Searchable or Automatic?)
* happy/sad/ideas trending over the past week
|
|
* Input
* [http://input.mozilla.com/en-US/search?q=&product=firefox&version=5.0a2&date_start=2011-05-03&date_end= Input]
|
* Aakash/Cheng
|-
|-
|
|
* Weekly top crashers (crashes per user)
* Daily top crashers
* Weekly top crashers
* Daily Crashes per user
|
|
* Socorro
* [https://crash-stats.mozilla.com/daily?form_selection=by_version&v&#91;&#93;=5.0a2&throttle&#91;&#93;=100.00 Socorro]
|
* QA Services/Sheila Mooney
|-
|-
|
|
* Weekly per-new-feature social media feedback
* Daily number of major (and higher) severity bugs in a UNCO/NEW state with no assignee filed  (with query link)
* Weekly regressions filed in a UNCO/NEW state with no assignee
|
|
* Twitter
* Bugzilla
* Facebook
|
* Metrics
|-
|-
|
|
* Weekly newly found blocker bugs
* Weekly number of ADU's
* Weekly change in number of ADU's
* Channel Switcher ADU change information
|
|
* Bugzilla (flagged "+" for a specific release)
* [https://metrics.mozilla.com/pentaho/content/pentaho-cdf-dd/Render?solution=metrics&path=%2Fblocklist%2FChannels&file=channelBreakdown.wcdf Metrics Dashboard]
|
* Legneato/Release Manager
|-
|-
|
|
* Weekly number of ADU's
* Startup time
* Weekly change in number of ADU's
* Miscellaneous Information
|
|
* Metrics Dashboard
* Telemetry
|
* Metrics/PMM
|}
|}


=== Beta ===
=== Beta ===
'''Goal'''
* Display the compatibility and quality levels of addons, websites and stability on the Beta channel over each week.
'''Stakeholders'''
'''Stakeholders'''
* Product Drivers
* Product Drivers
Line 280: Line 295:
|}
|}


<br>'''What to Report'''
<br>'''What to Gather'''
{| width="100%" cellpadding="3"
{| width="100%" cellpadding="3"
|-
|-
| style="background-color: #efefef;" | '''Report'''
| style="background-color: #efefef;" | '''Report'''
| style="background-color: #efefef;" | '''Source'''
| style="background-color: #efefef;" | '''Source'''
| style="background-color: #efefef;" | '''Responsibility'''
|-
|-
|  
|  
* Weekly per-new-feature happy/sad/ideas
* Weekly per-new-feature happy/sad/ideas
* Daily happy/sad/ideas stats and top insight per feature (Searchable or Automatic?)
* domains-specific feedback stats and insights compared to the alexa/google top 1000 sites
|
|
* Input
* [http://input.mozilla.com/en-US/search?q=&product=firefox&version=4.0b12&date_start=2011-05-03&date_end= Input]
|
* ?
|-
|-
|
|
* Weekly top crashers (crashes per user)
* Daily top crashers
* Weekly top crashers
* Daily Crashes per user
|
|
* Socorro
* [https://crash-stats.mozilla.com/daily?form_selection=by_version&v&#91;&#93;=4.0a12&throttle&#91;&#93;=100.00 Socorro]
|
* ?
|-
|-
|
|
* Weekly per-new-feature social media feedback
* Weekly per-new-feature social media feedback
|
|
* Twitter
* [http://search.twitter.com/search?q=firefox Twitter]
* Facebook
|
* ?
|-
|-
|
|
Line 313: Line 324:
|
|
* SUMO forums
* SUMO forums
|
* ?
|-
|-
|
|
Line 320: Line 329:
|
|
* Test Pilot User Study (if available)
* Test Pilot User Study (if available)
|
* ?
|-
|-
|
|
* Weekly newly found blocker bugs
* Daily number of major (and higher) severity bugs in a UNCO/NEW state with no assignee filed  (with query link)
* Weekly regressions filed in a UNCO/NEW state with no assignee
|
|
* Bugzilla
* Bugzilla
|
* ?
|-
|-
|
|
* Weekly number of ADU's
* Weekly number of ADU's
* Weekly change in number of ADU's
* Weekly change in number of ADU's
* Channel Switcher ADU change information
|
* [https://metrics.mozilla.com/pentaho/content/pentaho-cdf-dd/Render?solution=metrics&path=%2Fblocklist%2FChannels&file=channelBreakdown.wcdf Metrics Dashboard]
|-
|
* Add-ons Compatible
|
* [https://addons.mozilla.org/en-US/firefox/compatibility Compatibility Dashboard]
|-
|
|
* Metrics Dashboard
* Startup time
* Miscellaneous Information
|
|
* ?
* Telemetry
|}
|}


=== Release ===
=== Release ===
'''Goal'''
* Display the compatibility, security and quality levels of websites, stability and user experience on the Release channel over each week.
'''Stakeholders'''
'''Stakeholders'''
* Product Drivers
* Product Drivers
Line 354: Line 373:
| style="background-color: #efefef;"| '''Time'''
| style="background-color: #efefef;"| '''Time'''
|-
|-
| Firefox Product Meeting
| Firefox Product Meeting ('''reporting''')
| Wed 11:00AM Pacific
| Wed 11:00AM Pacific
|}
|}


<br>'''What to Report'''
<br>'''What to Gather'''


{| width="100%" cellpadding="3"
{| width="100%" cellpadding="3"
Line 364: Line 383:
| style="background-color: #efefef;" | '''Report'''
| style="background-color: #efefef;" | '''Report'''
| style="background-color: #efefef;" | '''Source'''
| style="background-color: #efefef;" | '''Source'''
| style="background-color: #efefef;" | '''Responsibility'''
|-
|-
|  
|  
* Weekly per-new-feature happy/sad/ideas
* Weekly per-new-feature happy/sad/ideas
* Daily happy/sad/ideas stats and top insight per feature (Searchable or Automatic?)
* domains-specific feedback stats and insights compared to the alexa/google top 1000 sites
|
|
* Input
* [http://input.mozilla.com/en-US/search?q=&product=firefox&version=4.0.1&date_start=2011-05-03&date_end= Input]
|
* ?
|-
|-
|
|
* Weekly top crashers (crashes per user)
* Daily top crashers
|
* Weekly top crashers
* Socorro
* Daily Crashes per user
|
|
* ?
* [https://crash-stats.mozilla.com/daily?form_selection=by_version&v&#91;&#93;=4.0.1&throttle&#91;&#93;=100.00 Socorro]
|-
|-
|
|
* Weekly per-new-feature social media feedback
* Weekly per-new-feature social media feedback
|
|
* Twitter
* [http://search.twitter.com/search?q=firefox Twitter]
* Facebook
|
* ?
|-
|-
|
|
Line 392: Line 407:
|
|
* SUMO forums
* SUMO forums
|
* ?
|-
|-
|
|
Line 399: Line 412:
|
|
* Test Pilot User Study (if available)
* Test Pilot User Study (if available)
|
* ?
|-
|-
|
|
* Weekly newly found chemspill bugs filed
* chemspill keyword bugs
|
|
* Bugzilla
* Bugzilla
|
* ?
|-
|-
|
|
* Weekly number of ADU's
* Weekly number of ADU's
* Weekly change in number of ADU's
* Weekly change in number of ADU's
* Channel Switcher ADU change information
|
|
* Metrics Dashboard
* [https://metrics.mozilla.com/pentaho/content/pentaho-cdf-dd/Render?solution=metrics&path=%2Fblocklist%2FChannels&file=channelBreakdown.wcdf Metrics Dashboard]
|-
|
|
* ?
* ('''Mobile only''') Reviews and Ratings
|
* Android Marketplace
|-
|-
|
|
* Weekly press articles/notable highlights
* Add-ons Compatible
|
|
* Press/PR
* [https://addons.mozilla.org/en-US/firefox/compatibility Compatibility Dashboard]
|
* ?
|-
|-
|
|
* ('''Mobile only''') Android Marketplace Reviews and Ratings
* Startup time
|
* Miscellaneous Information
* PMM
|
|
* ?
* Telemetry
|}
|}


Latest revision as of 15:42, 24 May 2011

Draft-template-image.png THIS PAGE IS A WORKING DRAFT Pencil-emoji U270F-gray.png
The page may be difficult to navigate, and some information on its subject might be incomplete and/or evolving rapidly.
If you have any questions or ideas, please add them as a new topic on the discussion page.

Background

Firefox 4 was a success for user feedback with tens of thousands support threads and millions of pieces of feedback on Input. Unfortunately, we didn't integrate many other feedback mediums that offer more information about key aspects of the browser (i.e. stability, features, responsiveness, usability and performance). Mozilla encompasses many different mediums such as Socorro, Test Pilot, news sources, Bugs, Social Networks (i.e. facebook, twitter) and ADU's. The new rapid release process will require new feedback strategies to quickly enlighten drivers on the status quo of the browser across multiple channels

Current Feedback Offerings

Source Type Options
SUMO
  • Help requests from users regarding a breakage in functionality or usability
  • docs metrics
  • forums
  • live chat
Socorro
  • Crash reports with opt-in user comments/urls/e-mails
Input
  • Text-based comments via an in-product mechanism and categorized by happy/sad/ideas
  • feeds
  • search
  • daily clusters
  • domain-specific site feedback
Bugzilla
  • Mozilla's primary bug tracking system
  • bug list of issues not fixed for specified release
Test Pilot
  • Platform that collects structured user feedback through Firefox
  • user studies
Press/PR
  • A general gauge of how Firefox is perceived by news outlets.
  • news articles/opinion pieces/walkthroughs/competitor news
Social Media
  • Text-based comments collected from 3rd-party sources.
ADUs
  • The number of actively daily users for a given build of Firefox.
(Mobile only)

Reviews and Ratings

  • User-submitted reviews and ratings
  • android marketplace
Forum/Newsgroup Comments
  • Free-form text comments in a threaded format

Needs

  • Central - The value of this channel is to shows bleeding edge features that may or may not end up in the release, but give a glimpse into Firefox's strategy/vision.
  • Aurora - Users here will be able to experiment with new features and determine if they match quality criteria to be graduated to a release of Firefox proper.
  • Beta - Users will be able to get the inside track on new features landing in Firefox proper and increase the overall quality of the new release by finding compatibility
  • Release - We'll look for possible security and Top Issues and possible Chemspills on this channel.
Regressions Stability Feature Quality UX Feedback Website Compat. Plugin Compat. Addon Compat. Chemspills
Central P2 P1
Aurora P1 P1 P1 P1 P3 P2
Beta P3 P1 P1 P2 P1 P1 P1
Release P2 P1 P3 P1

Objectives

  • To form a group of feedback leads throughout the organization
  • Create a comprehensive picture of what users are saying on a regular basis

Plan

Initially, we want to meet once a week to coalesce our findings for the needs over each release channel. The final output of each meeting will be a comprehensive report of all feedback summaries.

Central

Goal

  • Show the health (i.e. general stability and features) of builds on the Central channel on a weekly-basis.

Stakeholders

  • Product Drivers
  • Feature Teams
  • QA

When

Meeting Time
Firefox Development Meeting (reporting) Tues 11:00AM Pacific


What to Gather

Report Source
  • happy/sad/ideas trending over the past week
  • Daily top crashers
  • Weekly top crashers
  • Daily number of major (and higher) severity bugs in a UNCO/NEW state with no assignee filed (with query link)
  • Weekly regressions filed in a UNCO/NEW state with no assignee
  • Bugzilla
  • Weekly number of ADU's
  • Weekly change in number of ADU's
  • Daily performance numbers and trends
  • Weekly performance trend
  • Startup time
  • Miscellaneous Information
  • Telemetry

Aurora

Goal

  • Display the user satisfaction and quality levels of specific features on the Aurora channel over the past week.

Stakeholders

  • Product Drivers
  • Feature Teams
  • Marketing
  • QA

When

Meeting Time
Feedback meeting (gathering) Tues 3:00PM Pacific
Aurora Channel Meeting (reporting) Wed 2:00PM Pacific


What to Gather

Report Source
  • Daily triage of happy/sad/ideas per feature (Searchable or Automatic?)
  • happy/sad/ideas trending over the past week
  • Daily top crashers
  • Weekly top crashers
  • Daily Crashes per user
  • Daily number of major (and higher) severity bugs in a UNCO/NEW state with no assignee filed (with query link)
  • Weekly regressions filed in a UNCO/NEW state with no assignee
  • Bugzilla
  • Weekly number of ADU's
  • Weekly change in number of ADU's
  • Channel Switcher ADU change information
  • Startup time
  • Miscellaneous Information
  • Telemetry

Beta

Goal

  • Display the compatibility and quality levels of addons, websites and stability on the Beta channel over each week.

Stakeholders

  • Product Drivers
  • Feature Teams
  • Marketing
  • Support
  • QA

When

Meeting Time
?? ??


What to Gather

Report Source
  • Weekly per-new-feature happy/sad/ideas
  • Daily happy/sad/ideas stats and top insight per feature (Searchable or Automatic?)
  • domains-specific feedback stats and insights compared to the alexa/google top 1000 sites
  • Daily top crashers
  • Weekly top crashers
  • Daily Crashes per user
  • Weekly per-new-feature social media feedback
  • Weekly top issues
  • SUMO forums
  • Weekly user study findings
  • Test Pilot User Study (if available)
  • Daily number of major (and higher) severity bugs in a UNCO/NEW state with no assignee filed (with query link)
  • Weekly regressions filed in a UNCO/NEW state with no assignee
  • Bugzilla
  • Weekly number of ADU's
  • Weekly change in number of ADU's
  • Channel Switcher ADU change information
  • Add-ons Compatible
  • Startup time
  • Miscellaneous Information
  • Telemetry

Release

Goal

  • Display the compatibility, security and quality levels of websites, stability and user experience on the Release channel over each week.

Stakeholders

  • Product Drivers
  • Feature Teams
  • Marketing
  • Support
  • Security
  • RRRT

When

Meeting Time
Firefox Product Meeting (reporting) Wed 11:00AM Pacific


What to Gather

Report Source
  • Weekly per-new-feature happy/sad/ideas
  • Daily happy/sad/ideas stats and top insight per feature (Searchable or Automatic?)
  • domains-specific feedback stats and insights compared to the alexa/google top 1000 sites
  • Daily top crashers
  • Weekly top crashers
  • Daily Crashes per user
  • Weekly per-new-feature social media feedback
  • Weekly top issues
  • SUMO forums
  • Weekly user study findings
  • Test Pilot User Study (if available)
  • chemspill keyword bugs
  • Bugzilla
  • Weekly number of ADU's
  • Weekly change in number of ADU's
  • Channel Switcher ADU change information
  • (Mobile only) Reviews and Ratings
  • Android Marketplace
  • Add-ons Compatible
  • Startup time
  • Miscellaneous Information
  • Telemetry

Open Questions

  • Normalize counts across feedback channels?
  • Reporting rates across feedback channels?
  • How does press coverage work into this reporting system?
  • What about bugs filed which are the most critical, but are not fixed?
Retrieved from "https://wiki.mozilla.org/index.php?title=Firefox/Input/Feedback_Strategy&oldid=311771"