1
edit
QA/Triage: Difference between revisions
Jump to navigation
Jump to search
→How to Clean Incoming Bugs: copyedit
(→How to Track Relevant Incoming Bugs: Going into more detail on how to track a given component by e.g. QA contact) |
(→How to Clean Incoming Bugs: copyedit) |
||
| Line 33: | Line 33: | ||
Ensuring the bugs are properly marked up helps make sure they don't get ignored. | Ensuring the bugs are properly marked up helps make sure they don't get ignored. | ||
For example, if a bug that really affects all platforms is marked "Solaris" because | For example, if a bug that really affects all platforms is marked "Solaris" because | ||
the original bug author was on a Solaris machine, most developers will ignore it | the original bug author was on a Solaris machine, most developers will ignore it because | ||
they need to focus on the bugs that affect the most people. If a bug is in the wrong | they need to focus on the bugs that affect the most people. If a bug is in the wrong | ||
component it can also get stuck without any attention. Another common issue is if | component it can also get stuck without any attention. Another common issue is if | ||
truly major bugs are marked normal, or have a confusing summary. | truly major bugs are marked as normal, or if the bugs have a confusing summary. Ensuring that | ||
the bug is properly marked up is very helpful for getting it attention as well as | the bug is properly marked up is very helpful for getting it attention as well as | ||
keeping it from being ignored by appropriate queries that developers and other QAs | keeping it from being ignored by appropriate queries that developers and other QAs | ||
| Line 44: | Line 44: | ||
as possible. These techniques really apply to bugs QAs are filing as well. | as possible. These techniques really apply to bugs QAs are filing as well. | ||
# <u>Understand the bug:</u> Read very carefully to try and understand what they're talking about. | # <u>Understand the bug:</u> Read very carefully to try and understand what they're talking about. | ||
# <u>Look for duplicates:</u> A lot of bug reporters mistakenly assume their bug | # <u>Look for duplicates:</u> A lot of bug reporters mistakenly assume their bug hasn't been filed yet. Search for previously filed bugs by searching for keywords in the summary in the bug, and try alternate spellings and synonyms. For example, if the bug is about the control key, try spelling it "Ctrl" as well. Look through fixed and duplicate bugs when you search for duplicates. | ||
# <u>Ask for more information from the reporter</u>: If you don't understand the bug or can't reproduce the bug, ask the reporter for more information, until you have enough to try it yourself. If they did not provide build information, get that right away, because many bug reports are for older versions that have long been fixed.<br>Some other information you could ask for: | # <u>Ask for more information from the reporter</u>: If you don't understand the bug or can't reproduce the bug, ask the reporter for more information, until you have enough to try it yourself. If they did not provide build information, get that right away, because many bug reports are for older versions that have long been fixed.<br>Some other information you could ask for: | ||
#* Does the bug occur in [http://kb.mozillazine.org/Safe_mode Safe Mode]? With a [http://kb.mozillazine.org/Profile_Manager clean profile]? | #* Does the bug occur in [http://kb.mozillazine.org/Safe_mode Safe Mode]? With a [http://kb.mozillazine.org/Profile_Manager clean profile]? | ||
#* If it's a crash, ask for [http://kb.mozillazine.org/Talkback Talkback data]. ([[Mozilla_QA_Community:Talkback|more info on Talkback]]) | #* If it's a crash, ask for [http://kb.mozillazine.org/Talkback Talkback data]. ([[Mozilla_QA_Community:Talkback|more info on Talkback]]) | ||
#* If the reporter still can reproduce but you can't, ask him nicely if he can test a nightly build of [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-trunk/ Firefox] or [http://ftp.mozilla.org/pub/mozilla.org/seamonkey/nightly/latest-trunk/ SeaMonkey]. | #* If the reporter still can reproduce but you can't, ask him nicely if he can test a nightly build of [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-trunk/ Firefox] or [http://ftp.mozilla.org/pub/mozilla.org/seamonkey/nightly/latest-trunk/ SeaMonkey]. | ||
# <u>Reproduce the bug</u>: Change the status from Unconfirmed to New if appropriate. For instance, for many Core bugs, like layout or DOM bugs, | # <u>Reproduce the bug</u>: Change the status from Unconfirmed to New if appropriate. For instance, for many Core bugs, like layout or DOM bugs, a minimal testcase is often needed before a bug can be confirmed. Many times, bug reports are confusing and difficult to reproduce, so read the bug description over very carefully. It will reveal clues, and bring up questions. If the described problem is in fact the desired behavior then mark the bug INVALID. If the bug report does not have enough information to be of any value, and requests for clarification don't net results, then resolve the bug as INCOMPLETE. When marking INVALID, be sensitive to the fact that the bug reporter may get offended. While changing the status it's often good to write something like "This appears to work as designed, please reopen and provide more information if you believe this is in error."<br>If you're convinced you understand the bug, and you can't reproduce the bug, you could resolve the bug to WORKSFORME. But be careful with this, the bug can be a duplicate of a bug that has been fixed, so try to search for that bug. Also, you need to test on the same platform as the reporter, because it could be a platform dependent bug. The best resolution is if the original reporter marks a bug WORKSFORME, after he tried with the latest nightly build, tried with a clean profile, etc. | ||
# <u>Simplify the bug testcase</u>. You're the lead investigator -- so try to get enough information to turn intermittent or hard to reproduce bugs into something that is easy to reproduce. See more on this in [http://wiki.mozilla.org/MozillaQualityAssurance:Triage#How_to_Really.2C_Really_Help_Developers_on_Bugs_--_Minimal_Testcases "How to Really, Really Help Developers on Bugs -- Minimal Testcases"] below. | # <u>Simplify the bug testcase</u>. You're the lead investigator -- so try to get enough information to turn intermittent or hard to reproduce bugs into something that is easy to reproduce. See more on this in [http://wiki.mozilla.org/MozillaQualityAssurance:Triage#How_to_Really.2C_Really_Help_Developers_on_Bugs_--_Minimal_Testcases "How to Really, Really Help Developers on Bugs -- Minimal Testcases"] below. | ||
# <u>Ensure appropriate hardware/platform tests</u>: most bug filers will only have tested on one platform. If bug was filed for a particular platform, see if it's a bug on another platform. If it's been reproduced on at least 2 platforms that aren't both Unix/Linux, then go ahead and mark it All/All ((Hardware/OS). Most bugs are actually All/All. It's useful to get a feeling of when to do extra tests and when not to waste your time, but this comes with experience. For example, OS X uses many native Carbon form controls and menus whereas Windows and Linux do not, so form control bugs on Mac are probably just Mac-only. However, a bug in how HTML links are dealt with is almost certainly a bug on all platforms. Also, it's useful to know the QA community so that you can ask for help when you don't have the right hardware/OS to test a particular bug with. | # <u>Ensure appropriate hardware/platform tests</u>: most bug filers will only have tested on one platform. If bug was filed for a particular platform, see if it's a bug on another platform. If it's been reproduced on at least 2 platforms that aren't both Unix/Linux, then go ahead and mark it All/All ((Hardware/OS). Most bugs are actually All/All. It's useful to get a feeling of when to do extra tests and when not to waste your time, but this comes with experience. For example, OS X uses many native Carbon form controls and menus whereas Windows and Linux do not, so form control bugs on Mac are probably just Mac-only. However, a bug in how HTML links are dealt with is almost certainly a bug on all platforms. Also, it's useful to know the QA community so that you can ask for help when you don't have the right hardware/OS to test a particular bug with. | ||
| Line 61: | Line 61: | ||
# <u>Change Product, Component and owner</u>: Many bugs get marked "Firefox" when they are really bugs in the core engine. If it's a basic XUL, HTML or layout bug that affects multiple products send it to Core. After you submit a Product change to the bug, Bugzilla will allow you to change the Component as well. When changing Product and/or Component you usually want to select the radio button "Reassign bug to default assignee and QA contact of selected component ". The exception to this is when the bug is a regression and already assigned to a particular developer who caused the regression. You can spot those situations via a comment like "Joe, this appears to be from your checkin from bug 32412". | # <u>Change Product, Component and owner</u>: Many bugs get marked "Firefox" when they are really bugs in the core engine. If it's a basic XUL, HTML or layout bug that affects multiple products send it to Core. After you submit a Product change to the bug, Bugzilla will allow you to change the Component as well. When changing Product and/or Component you usually want to select the radio button "Reassign bug to default assignee and QA contact of selected component ". The exception to this is when the bug is a regression and already assigned to a particular developer who caused the regression. You can spot those situations via a comment like "Joe, this appears to be from your checkin from bug 32412". | ||
# <u>Add Keywords</u> -- here are just a few of the important ones [https://bugzilla.mozilla.org/describekeywords.cgi the full list of keywords is here]). | # <u>Add Keywords</u> -- here are just a few of the important ones [https://bugzilla.mozilla.org/describekeywords.cgi the full list of keywords is here]). | ||
#* "access": mark accessibility bugs with the keyword "access". This allows accessibility bugs to not be tied to the "Disability Access" or "Keyboard navigation" components, but to the actual component with the problem. For example, the developers for Places should really be responsible for accessibility bugs in Places, but adding the keyword "access" makes sure | #* "access": mark accessibility bugs with the keyword "access". This allows accessibility bugs to not be tied to the "Disability Access" or "Keyboard navigation" components, but to the actual component with the problem. For example, the developers for Places should really be responsible for accessibility bugs in Places, but adding the keyword "access" makes sure that those looking for accessibility bugs will also find the bug report. | ||
#* "regression": mark regressions with "regression". Was this ever not a bug? For example, did it work in Firefox 1.5? See below for more important information on how to deal with regressions. | #* "regression": mark regressions with "regression". Was this ever not a bug? For example, did it work in Firefox 1.5? See below for more important information on how to deal with regressions. | ||
#* "crash": pretty obvious :) | #* "crash": pretty obvious :) | ||
#* "dataloss": | #* "dataloss": causes users to lose data | ||
#* "hang": freezes or locks up the application | #* "hang": freezes or locks up the application | ||
#* "testcase": indicates that a simplified testcase is included (see below) | #* "testcase": indicates that a simplified testcase is included (see below) | ||