QA/Talkback/Reproducing Crash Bugs

From MozillaWiki
< QA‎ | Talkback
Jump to: navigation, search

« QA/Talkback

Objective

One of the toughest challenges with Talkback data is finding smart ways to use all the information available to help reproduce crashes that others have experienced.

There might be a few "Topcrashers" that some people will discover easily, but many of the crash bugs in Bugzilla are very difficult to understand without actually seeing it yourself. Therefore, we need to find specific steps to reproduce crashes or develop simplified testcases to help ourselves and the developers debug those crashes.

With the community feedback from bugs and in Talkback data, we hope to use Bug Day to get more eyes on those tricky bugs and see if a few people can find ways to easily reproduce those crashes.

Goals for Bug Day

  • Familiarize yourself with the various Talkback tools and reports.
  • Find UNCONFIRMED crash bugs, try to reproduce them, and update the bug as needed.
    • If there is missing information or no steps to reproduce, try to understand what the reporter might have been seeing and ask for clarification and testcase.
    • If you are able to reproduce the crash, mark the bug NEW and add your steps to reproduce or testcase to the bug.
    • If enough users are unable to reproduce the bug, mark it WORKSFORME.
  • Find older bugs that might be good candidates to mark WORKSFORME.
    • Try to find bugs with testcases and/or steps to reproduce that have not seen any action in a while
    • Try to crash with a recent nightly or major release.
    • If it is no longer an issue and you cannot find any similar crashes in Talkback data, you can mark it WORKSFORME.

NOTE: Usually when I mark a bug WORKSFORME, I also make sure to say that the bug should be REOPENED if anyone is able reproduce it with a recent nightly or major release.

Common Bug Queries

Higher Priority These bugs often need more investigation and/or testing. They are a great place to find bugs that people can use some help with. Either by trying to reproduce the crash, creating testcases, or verifying whether it still exists in recent releases.

Difficult To Reproduce Any bugs that you come across that are difficult to understand, debug, or reproduce are most likely bugs that don't have steps or a testcase.

  • If you find a bug like that and it seems like something we need to investigate further, please add [need testcase] in the status whiteboard.

Low Hanging Fruit These bugs are all known crash bugs that have testcases already. They are a great place to look to see what types of testcases have been created for testing crash bugs:

There are also a few things the community can do to help clean up these bugs lists:

  • Find out if older bugs still exist with the latest versions of the Mozilla Suite 1.7.13, Firefox 1.5.0.2, and/or Thunderbird 1.5.0.2.
    • If you can reproduce a crash with recent builds/releases, it might help bring back some interest in fixing it. If you find steps to reproduce or develop a testcase, please add the "testcase" keyword.
    • If it is no longer an issue in the latest code, it might be a good candidate for marking "Worksforme" to get it off the radar.
  • Find out if any of the crash bugs are a "topcrash" for a recent Mozilla Suite, Firefox and/or Thunderbird release.
    • If a unique stack signature in the bug returns a lot of crashes in Talkback data, it might be worth noting and investigating further.
    • If you notice a top website mentioned in the bug, it might be worth looking it up in Talkback data to see if the same crash has been reported by a lot of users.
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Talkback/Reproducing_Crash_Bugs&oldid=74568"