FirefoxOS/Test Case Writing Guidelines

Test Case Writing Guidelines

Background

After a few years of Firefox OS development and testing, we (the Firefox OS QA team) found that the style of writing test cases in MozTrap is hard to maintain. Therefore we would like to come up with a test case writing guideline that promotes readability, maintainability and boosts QA productivity.

The test cases we refer to are functional/regression cases written in MozTrap. They may potentially be written into automation scripts.

Guidelines

1. Keep multiple conditions in one test case

   Example 1:

   Don't write two cases, one for testing turning on WiFi, one for testing turning off WiFi

   Do write one test case only like so: 1. Turn on WiFi, verify the Internet connection is working 2. Turn off WiFi, verify the Internet connection is not working.

   Example 2

   Put conditions, like connection type (WiFi, mobile data, etc.) for an app that requires Internet access, in one single case.

2. Don't write multiple cases that only verifies the UI matches the UX spec, instead write a single case that says "Verify App X's UI matches the UX spec (https://link.to.ux.spec.pdf)"
3. Avoid hard-coding UI texts in steps, otherwise you might need to update the case frequently when UI changes.

   Example:

   Don't write "A popup box saying 'No connection found' appears, click the "OK" button to dismiss it" (The "No connection found" and "OK" button may change, and it's not l10n-friendly.)

   Do provide the UX spec and l10n documentation, and write something like "A warning message appears, dismiss it".