Support/FeedbackProcess

Overview

This document describes how we collect feedback, issues and concerns about Firefox Desktop from users and communicate it back to the rest of Mozilla. There are two main sections here: Data sources and Outputs. The Data sources are where we're currently gathering information. The Outputs are the various places that we're reporting this out.

How this affects you

If you're interested in what our users are saying about desktop Firefox (in particular, what problems they are having with the product) you can sign up for the appropriate report. If there's a focus that we're missing that you're particularly interested in, please let Cheng (cww@mozilla) know and we'll work on customizing something that meets your needs. If you're collecting feedback from a data source or you know of a data source that we should integrate, please let us know.

Data sources

SUMO (forums)

• Best for: release channel issues (beta/aurora channel currently doesn't really account for much traffic)
• Pluses: Provides pretty good quality information: We get plugins and useragents, often troubleshooting info and can work closely with cooperating users to figure out what's going on.
• Minuses: There is nothing positive on SUMO (it's for getting help with problems) so it's not useful for getting overall user sentitment. Lower total amount of reports than input. Takes a long time to go through reports.
• Frequency of when we look at it: We look at SUMO forums thoroughly (reading hundreds of posts) once a week (usually Friday) as well as check in every 1-2 days reading maybe a couple dozen posts. Major issues that come up are also flagged by the community.

Input

• Best for: overview of beta/aurora channel. "First impressions" for release channel but not really issues.
• Pluses: We get a lot of feedback and it's fast to search and read through lots of it. Some user sentiment information.
• Minuses: The main drawback is we basically only get a sentence from each user and can't contact them back unless they choose to tweet their problem so it's hard to tell how reproducible something is. Issues/problems aren't reported here as much as in SUMO.
• Frequency of when we look at it: We look at Input formally once a week on Mondays, trying to read as much as we can at once.

Social media

• Best for: getting an overview of user sentitment, especially on release channels.
• Pluses: We get both positive and negative comments from an audience that is perhaps a little more technical -- we can really see what people like. It's also really fast; major issues (like an update breaking a large number of users or a site going down) can show up first on social networking before they show up on SUMO.
• Minuses: it requires a lot of manual sifting through, we don't get any technical information (user agents, extension information) and often the comments are short and emotional rather than informative.
• Frequency of when we look at it: Right now, various people read through twitter/Facebook reports but we haven't integrated this feedback into our regular reporting process.

Issue investigations

This isn't a separate input really but it's something that provides a different kind of information. We go through and really dig into a specific issue, looking all all the reports we can find (from the above sources, bugzilla, crash stats), contacting users back, talking with developers etc.

• Best for: specific issues that are urgent or on request.
• Pluses: getting STRs and detailed descriptions so work can be made towards a fix.
• Minuses: Not for seeing how a problem fits into the bigger picture since you're specifically digging for one issue. Also, it's very very time consuming.
• Frequency of when we look at it: This is done on an on-demand basis but usually once or twice a week (on bad weeks, easily could be an entire week).

Outputs

Weekly report in planning meeting

• When: Every Wednesday.
• Focus: New issues on all channels, especially actionable ones. This means that after releases, it's mostly major reports of things we should fix and outside releases it's mostly extensions/sites breaking or malware reports.
• How do I get on this list: Go to the Planning meetings [1]

Weekly reports in Aurora/Beta meetings

• When: In the appropriate meeting as needed.
• Focus: If channel-specific issues show up in feedback.
• How do I get on this list: Go to the Aurora/Beta meetings [2] [3]

Support report

• When: As needed, definitely after every new release.
• Focus: Presenting overall user concerns and issues with a version of Firefox to a larger base. The main reason that these take a while to do as we try to have fully investigated issues and all the details when we post.
• How do I get on this list: Sign up here: [4]

Weekly desktop report

• When: Every week, in the weekly desktop report, we'll start putting a quick summary of support trends we're noticing.
• Focus: It's not meant to be fully comprehensive but just provide an overview of the kinds of issues users are more likely than usual to be seeing each week.
• How do I get on this list: Email Laura Mesa (lmesa@mozilla.com)

RRRT list

• When: As needed.
• Focus: If an issue suddenly spikes (an extension update broke a major site for a lot of people), we'll email the RRRT list where it'll get immediate QA/driver attention. We do this for issues that can't wait until the next major report. We also notify this list of anything that shows up post releases.
• How do I get on this list: Sign up here: https://mail.mozilla.org/listinfo/rrrt

Known gaps and actions

We know that the above data sources and reports aren't comprehensive. Here are some known gaps:

• We don't have a way to cover reporting of "general" issues that aren't specific to a single Firefox version and aren't spiking but should be addressed. In particular, these issues are usually very generic and lack concrete steps that developers can act on. As a Q3 goal, we'd like to report on these regularly and focus on one or two issues to drive forward.
• We don't have a strategy for collecting the feedback that we get from social media and incorporating that information into our reports. This is something we'd like to improve over the next quarter as well.