QA/Talkback/Topcrash Analysis

Introduction

How often does Firefox or Thunderbird crash? In what particular features (or during which tasks) do we see frequent crashes? Talkback, our crash reporting tool, can show us data to help answer these questions.

Publically accessible Talkback database: http://talkback-public.mozilla.org/talkback/fastfind.jsp

What needs to be done?

Mozilla QA periodically analyzes the Talkback data to see if new topcrash bugs need filing (and ultimately developer attention). Ideally this would occur on a regular basis:

• For the trunk products (Firefox and Thunderbird), check Talkback data approximately 3x a week.
• For branch Firefox and Thunderbird products, check Talkback during the candidate process and immediately after release. Then about once or twice a month thereafter.
• For branch (1.7.x) Mozilla, , check Talkback immediately after release. Then about once or twice a month thereafter.

How to view topcrash reports

1. Go to http://talkback-public.mozilla.org/reports/
2. Click on the product: Firefox, Thunderbird, Mozilla or Camino.
3. Choose the branch, trunk, release, etc. from the links under these three or two columns:
   • Development: trunk builds are here.
   • Major releases (Firefox and Thunderbird only): 1.0, 1.0.x, etc.
   • Milestone releases: also includes final Mozilla releases.
   • Beta releases: Mozilla and Camino only.
4. In the left side, a list of the reports available should appear, such as:
   • Topcrashers by Build: this is the main report to start with.
   • Simple Reports: this groups crashes by signature, and skimming this helps with getting additional comments/feedback.
   • Smart Analysis: also grouped by signature, but also lists bug number info (where available). You can add info found here to new topcrash bugs filed, especially if it helps with finding a reproducible test. Note that you should skim first to see that there's no sensitive data there (eg, user accidentally typed in a passwd!) and remove that before pasting into a bug report.

How to file a topcrash bug

1. Look at the topcrash report, and scan the top 10-15 ranked Stack Signatures.
2. Scan the Open Bugs column for stack signatures lacking bugs (empty cells).
3. Also look at the Total column which shows the total crash count per stack signature, for the range of build ID's displayed in the table.
   • Note that during the early stages of trunk development, the total crash number might be rather low (1-5), hence might not be significant enough to be filed as a topcrash bug (yet!). These shouldn't be completely ignored, however, so might deserve their own bug filed if Smart Analysis displays user comments with enough information to reproduce.
4. To figure out the best Bugzilla product and component for the topcrash bug, click (best open in a new tab) the Smart Analysis: All Platforms link on the left side of the page.
5. While filing the bug in Bugzilla, make sure the summary has the following format: branch [@ Stack:Signature()][...] - bug_summary
   • branch is the specific branch (e.g., FX102, TB10x, Trunk, M176, etc.).
   • [@ Stack:Signature()] is the stack signature from Talkback, enclosed in [] with @ followed by a whitespace preceding the signature. You can also have multiple signatures in the summary. Note that you need not include the following in the signature:
     • parenthesis in functions ()
     • call line numbers from functions
     • + signs
     • 0x00000000 or other memory offsets
   • bug_summary is, where applicable, the actual summary of the bug.
6. The bug should also include the crash and topcrash keywords. If any of the comments have reproducible test steps, go ahead and use topcrash+.