QA/Minimal Test Cases

From MozillaWiki
< QA
Jump to: navigation, search

What are Minimal Test Cases?

Finding the inherent issue of a bug report and providing an excellent reproduction scenario is more of an art form than a simple process. It takes reading comprehension, technical savvy and whole lot of effort to get through the process. So, most of the Bug Triagers in the MozQA community are excellent prolem solvers. So, it's really important to create a quality reproduction scenario to help out the community members who will be patching your issue.

A Simple Example of Bug Deconstruction

In order to do this, let's deconstruct a bug. Firstly, here are some steps for using the process of elimination to create a minimal test-case from a web page that displays a consistent bug.

  1. Save the web page on your local hard drive. Make a second copy.
  2. From the first copy, cut out big pieces of CSS, JS or HTML -- start with the pieces that are most likely to not have the bug. For example, at first just try to strip out plain paragraphs as these are relatively simple. Try to make as much as possible inline. Try to strip out all images, frames. If any image is required to get the bug, try changing the src to something that isn't likely to expire, such as: [[1]]
  3. If the new version of the file still has the bug, save it as your second copy and repeat step 2.
  4. If the new version does not have the bug, then your second copy still contains the bug. In this case, repeat step 2 but avoid cutting out the same piece that you did last time.
  5. When you can no longer cut out any HTML without destroying the ability of the file to show the bug, you have a minimal testcase.

Hopefully you'll be left with a file that has only 1-5 HTML elements in it, but still reproduces the bug. At this point you can actually try to start stripping out irrelevant attributes as well You may end up with something like <label for="myfile">Enter file name:</label><input type="file" disabled="true" id="myfile"> instead of 50k worth of HTML. From all the developers you do this for, "Thank You!".

When finished, attach the minimal testcase, mark the bug with keyword testcase, and change the bug summary if necessary. For example, "No borders in tables at http://www.sillystatsandstuff.com" can become something much more precise such as "tables with rowspans do not get borders". If it doesn't affect the test - add a title so it can be referred to, e.g <title>Testcase #1 for bug 123456</title>. Also, add the bug number in the filename of the testcase.

You need to use your brain throughout this process. For example, if the bug involves line wrapping, then the width of the window and the font size may impact the results. In this case, you can ensure that your reduced testcase will work for everyone by using "font-family: monospace;" and a width specified in "ch" units.

What do I take away from this?

Now, this can all be rather intimidating for people who haven't been playing around with different web technologies such as AJAX, JS, or XML for a long time, but the most important thing to take away from this page is that there's a back-end to the bugs you find when interacting with any software, hardware, or general product. Make sure to take that into account when writing your next bug or triaging a host of incoming bugs on Bugzilla!