Confirm, emeritus
722
edits
Changes
m
no edit summary
across many organizations. The great benefit is that the various
products are extremely thoroughly tested on a day to day basis and
everyone benefitis benefits from the resulting quality. New regressions are
often noticed right away before other new fixes are built on top of
the problematic code. A diversity of opinions often allows the
valid problems, putting these issues on track to be addressed.
In addition, new Mozilla QA's QAs are coming into a very complex
project which may not operate in a way similar to projects they've
worked in before.
Do the following steps in batches, if possible. Each time you submit changes to
the bug it creates extra Bugmail, so it's better to do the changes in as few rounds
as possible. These techniques really apply to bugs QA's QAs are filing as well.
# <u>Understand the bug:</u> Read very carefully to try and understand what they're talking about.
# <u>Look for duplicates:</u> A lot of bug reporters mistakenly assume their bug won't have been filed yet. Search for keywords in the bug, and try alternate spellings and synonyms. For example, if the bug is about the control key, try spelling it ctrl as well. Look through fixed bugs when you search for duplicates.
# <u>Simplify the bug testcase</u>. You're the lead investigator -- so try to get enough information to turn intermittent or hard to reproduce bugs into something that is easy to reproduce. See more on this in "How to Really, Really Help Developers on Bugs" below.
# <u>Ensure appropriate hardware/platform tests</u>: most bug filers will only have tested on one platform. If bug was filed for a particular platform, see if it's a bug on another platform. If it's been reproduced on at least 2 platforms that aren't both Unix/Linux, then go ahead and mark it All/All ((Hardware/OS). Most bugs are actually All/All. It's useful to get a feeling of when to do extra tests and when not to waste your time, but this comes with experience. For example, OS X uses many native Carbon form controls and menus whereas Windows and Linux do not, so form control bugs on Mac are probably just Mac-only. However, a bug in how HTML links are dealt with is almost certainly a bug on all platforms. Also, it's useful to know the QA community so that you can ask for help when you don't have the right hardware/OS to test a particular bug with.
# <u>Ensure a consise concise and precise summary</u>: It is reasonable to edit the summary a bit if it will help people understand the bug more quickly or will clear up something that's misspelled or incorrect. Check to see that the summary is clear, concise and describes the problem as fully and correctly as is reasonable in a short space. Make sure it contains the words people would probably search for before filing a duplicate bug. To risk stating the obvious, the spelling in the summary matters because searches will for "Escape" will fail if the reporter wrote "Escpae".
# <u>Security sensitive bugs</u>: [TBD]
# <u>Ensure Proper Severity</u> (but don't change Priority field, which is for the developer). Here's a quick guide, but you can't get more help from Bugzilla itself
#* Critical: crash and security bugs (should also be marked with keyword crash).
#* Major: Bugs that affect the main screen, cause dataloss, loss of ability to operate the software or are just plain awful
#* Normal/Minor/TrivalTrivial: Most other bugs
# <u>Change Product, Component and owner</u>: Many bugs get marked "Firefox" when they are really bugs in the core engine. If it's a basic XUL, HTML or layout bug that affects multiple products send it to Core. After you submit a Product change to the bug, Bugzilla will allow you to change the Component as well. When changing Product and/or Component you usually want to select the radio button "Reassign bug to default assignee and QA contact of selected component ". The exception to this is when the bug is a regression and already assigned to a particular developer who caused the regression. You can spot those situations via a comment like "Joe, this appears to be from your checkin from bug 32412".
# <u>Add Keywords</u> -- here are just a few of the important ones [https://bugzilla.mozilla.org/describekeywords.cgi the full list of keywords is here]).
#* "dataloss": this bug will cause users to lose data
#* "hang": freezes or locks up the application
#* "testcase": indicates that a simplifed simplified testcase is included (see below)
# <u>List related meta bugs</u> in "this bug blocks". If an important meta bug has a difficult to remember number, give it a memorable alias. For example, here are some important accessibility-related meta bugs:
#* focusnav: related to focus and tab navigation
One really important thing that QAs can do for developers is to narrow down to the actual markup
causing the bug -- this applies for QA-filed bugs as well. Many testcases
filed by inexperienced bug reporters come from a website, which unforunately unfortunately maychange and thus the testcase is lost. Or, theHTML from the given testcase might be huge, which makes debugging extra difficult.
Here are some steps for using the process of elimination to create a minimal testcase from a webpage that displays a consistent bug.
# In the end, make sure that the bug is marked with the keyword "regression", and add a very clear comment something like: "March 19, 2006 trunk: works, March 20, 2006 trunk: broken".<li>An experienced QA can pinpoint the actual checkin with the problem, and assign the bug to the developer who was responsible.
If you are very technical, it is possible to go even further. Some QA's QAs who know
how to apply patches and build their own Mozilla have even been known to find which
part of a patch caused the bug, by building with or without each piece in a patch.
# Bugs that have a patch which was almost correct, and has been commented on (perhaps minused or not), but no one finished the work. Perhaps the original developer just forgot about it.
# Bugs that have been awaiting review for over 1 month
# Bugs that have had review requested, but not from a specific person. This often happens by mistake of the reviewer name was entered incorrectly, in which case Bugzilla will flag the patch for review but not assign that to someone. These review requests are almost always ignored. If you think this is a mistake, try to assign to the review request to someone appropriate (this takes experience). If you're not sure, ping the patch author and ask them if it was intentiionalintentional.
# Bugs where the patch author asked the reviewer a question, and received no answer
# Patches which are reviewed but no one checked them in. Believe it or not, this happens all the time. Either the developer forgot, or did not have the correct CVS permissions to make a checkin. In this case they're supposed to request help from a developer with permission to check in. If they don't know that, a perfectly good fix may just sit there rotting away.
