124
edits
Changes
→How to Help with Regressions -- Finding Regression Windows: add some more build id and bonsai info
== How to Help with Regressions -- Finding Regression Windows ==
Another great thing that QAs sometimes help developers with is finding a regression window for regressions. A regression window is the day that a bug broke. With this information we can take a look at the checkin list and through the process of eliminationsee what change caused the problem. It's always important to find the regressionwindow, because without it a developer may apply a bandaid fix, instead of findingthe true source of a problem.
Here are some steps for finding a regressions window:
# Use a separate dedicated profile, without any extensions installed and start with -profilemanager.
# Test the build and write down the results. It's common to get confused since you are testing for the absence of a bug, so good notes are important. If the bug occurs, then you know you will have to search back further in time. Otherwise you will need to search forward in time. The optimal algorithm is to test the date exactly between the last known date when the bug didn't occur and the last known date when the bug did occur. Keep dividing by two and you'll find the day.
# In the end, make sure that the bug is marked with the keyword "regression", and add a very clear comment something like: "March 19, 2006 trunk: works, March 20, 2006 trunk: broken". When you have found a regression window within a day, then the actual hour the product was built becomes important. You can give this information with the [http://wiki.mozilla.org/[MozillaQualityAssurance:Build_Ids |build ID]]. Another option is to look at the time when the "firefox.exe" or "thunderbird.exe" was last modified. Then take a larger margin (by the hour) of that time. A third option, if you're paying attention when downloading builds in step 2 above, is to note the sets of numbers in the directory name; stripping the dashes from that string gives you the build ID in most cases (e.g., a build from <tt>/pub/firefox/nightly/2005-08-27-19-trunk/</tt> is a trunk build with a build ID of 2005082719). # An experienced QA can pinpoint the actual checkin with the problem, and CC the developer who was responsible to the bug. Finding out which checkin was responsible for the regression is possible with the [http://bonsai.mozilla.org/cvsqueryform.cgi Bonsai tool]. At the bottom of that page you can add dates ("between ... and ..."). When submitting the form, you get the list of checkins between those dates. When you know the build IDs of the last working and first broken builds, expand them back into dates and times to get a more precise set of results (e.g., a build ID of 2005082719 becomes 2005-08-27 19:00). # If you are very technical, it is possible to go even further. Some QAs who knowhow to apply patches and build their own Mozilla have even been known to find whichpart of a patch caused the bug, by building with or without each piece in a patch.However, this is really not necessary, so don't worry if you're not comfortabledoing that.
Again, from all Mozilla developers to the QAs who find regression windows, "Thank You!"
