PostCrash

DRAFT
The content of this page is a work in progress intended for review.

Please help improve the draft!

Ask questions or make suggestions in the discussion
or add your suggestions directly to this page.

The goal of this project is to improve the user post-crash experience by:

• Developing about:crashes into a more useful resource
• Contacting the user by email after a crash is resolved

The project will require changes to about:crashes, SUMO, and Socorro in order to supply the required functionality.

about:crashes

We want to make about:crashes more useful. Ideally, for each crash we would display:

• the crash id
• the crash signature (source: Socorro)
• associated bugzilla ids (source: Socorro) and status of these bugs (source: Bugzilla or Socorro).
• If a bug is resolved fixed, show the version it was fixed in.
• if a crash is associated with a plugin, link to plugincheck (source: Socorro)
• If a canonical SUMO article exists, a link to that article, otherwise a link to SUMO search for that signature (as is currently done in Socorro)

The client should obtain the extra information from Socorro and SUMO via XHR. This data should be stored locally once final. Final data includes:

• crash signature
• bugzilla ids for RESOLVED bugs
• plugin association
• canonical SUMO links

Data that should be polled each time about:crashes is loaded includes:

• Status of bugzilla bugs that were unresolved last time about:crashes was loaded
• Search for a canonical SUMO article where one previously didn't exist

In order to minimize load on Socorro and SUMO, we will be default only show an expanded view for the most recent crash. Users may individually expand other crashes (something like "check crash status" on each previous crash).

Question:

• Should we show both hangs and crashes? Users may load the page and see nothing but hangs. Does it make more sense to (perhaps by default) just show crashes?

Changes to SUMO

• Add a webservice call that, given a signature and locale, searches for relevant content on SUMO and returns a URI.
• The web service should work as follows:
  • 1. If a canonical article exists for crash_signature (top search result), link directly to it, and display the title of the article
    2. Else if search results exist for crash_signature, show search results
    3. Else link directly to a single generic crash article

Changes to Socorro

• Add a webservice call. Given crash id, return signature, associated plugin, associated bugzilla id and bug status.

User email contact

After discussion (see chofmann's comments below for background), we think the best way to solve this is with a manual process rather than an automated one., something like the following:

• During the release process, a Support team member should collect a set of bugs fixed in the release and matching crash signatures.
• Socorro should provide the ability for an admin user to email the set of users who have submitted crashes matching a particular signature.
• Support team email would be something along the lines of:

"On date X/Y/Z you submitted a crash report to Mozilla. Crashes with this signature were addressed in bug xyz which has been resolved in version A.B.C."

Issues with regard to authenticity of email / phishing can be addressed by encouraging users to "visit mozilla.com and click on blah for more information" (taking a lead from Paypal/eBay here). We should in general make sure that emails come from a mozilla.com address and use whatever we have in place for email authentication.

Questions

• Some crashes have more than one associated bug. Should we email the user each time a bug is fixed or when all the bugs are fixed (or some other model)?
  • the period of when bugs get fixed to when the crash happens might be very long. bugs being fixed on the trunk now might not be available in a final release until end of q3 or q4. It doesn't make much sense to e-mail users now about fixes that won't appear in a final firefox 4.0 until then, or e-mail them at the end of the year about that that crash they had 6 months ago. A single signature with many bugs makes it nearly impossible to correlate if any one of the bugs might have fixed the specific bug that the user just crashed on. we would need to start storing the full stack trace(s) of the bugs that we have fixed and map them to the full stack trace that the user experienced. -chofmann

• What volume of users supply emails? (Check: For each of the topcrashers, how many associated user emails do we have?) The architecture of the solution for sending email depends on the volume.
  • Also depend on what events we send e-mails for. (see below: expect to send about 12k e-mails per day for just incoming reports where we are auto responding when a user sends us an e-mail address, and around 5k per day if we e-mail when there is a bug on file associated with the signature) -chofmann
  • Do repeated crashes mean repeated e-mails to the users with the same message, or do we track so that we don't keep telling users there is no fix for their crash yet? if we don't track it the e-mails start to look like spam. --chofmann

• we could be over engineering things here. the system that we had with talkback served a good purpose. when we found a specific crash that we had something specific to tell users about, we constructed the message, then we queued up the system to watch for that signature and send the message. This was for a very small number of the total overall crashes, but had useful information as opposed to just auto-responding. e.g.

"from your recent crash we detected that you need to upgrade to Skype X.X to fix the crash."  

At any particular point in time we might have something this specific to say in an extremely low pct. of cases. Numbers below indicate this might be lower than 1% of the time a user reports a crash to us.

• sending e-mails with technical instructions on how to fix or mitigate crashes, and espcially instructions that involve downloading and installing software is open to impersonation, and phishing. we need build in ways for users to authenticate that the message came from mozilla and that the same instructions are available on an "offical" mozilla hosted website.

some numbers

 
 14k   -count of reports that have e-mail address, 
378k   -total crashes per day, 
  3.6% -ratio of e-mails to total crashes
  5k   -signature has e-mail and an assigned bug 
  1%   -ratio of signatures with emails and assigned bugs to total crashes
 35%   -ratio of reports with e-mails to reports e-mails with bugs.

                                             email& email&
date           email=yes #crash email/crash   bugs  bugs/tl   email/email&bug
20100530-crashdata.csv 12596 341229 0.03691   4319  0.0126572   0.342887
20100531-crashdata.csv 13567 372037 0.03646   4784  0.0128589   0.35262
20100601-crashdata.csv 13941 378282 0.03685   5053  0.0133578   0.362456
20100602-crashdata.csv 13806 379287 0.03639   4793  0.0126369   0.347168
20100603-crashdata.csv 12353 332924 0.03710   3855  0.0115792   0.31207

-chofmann

Mockups

Early idea on a revamped about:crashes page

• The latest crash automatically queries SUMO and Socorro for status info.
• Older crashes might not query the servers automatically to reduce server load.
• To reduce information overload, the crash ID isn't shown, but clicking on the signature takes you to the full report on Socorro.

Some additional thoughts/ideas:

• Would be even better if the page could query SUMO for the full article title and show that instead of "View solution" (the SUMO service should also support l10n to make sure that a localized article title appears if it exists).
• The latest crash could be more prominently highlighted -- perhaps even have a separate section above the "Recent Crash Reports".
• The generic support link at the bottom could be expanded into its own section. The target link could take you to a start page specializing on crash/hang problems, with prominent links for how to upgrade plugins, etc.