QA/TCM/BDD

Summary

The new Test Case Management (TCM) system is broken into two main parts:

• Back-end REST interface written in Java
• Front-end GUI interface written in Python and Django

We chose Behavior Driven Development (BDD) as the general approach to our automation. For REST endpoint tests, we chose a Python Framework called Lettuce. For UI automation, we will use Selenium in one fashion or another, ideally controlled by Lettuce as well.

Mission

Goals

1. Automate as much functionality as possible and practical
2. Automate in a way that it's easy to determine what each test case covers

Requirements

• Git
• Python 2.6 or later
• Python packages
  • Lettuce 0.1.15
  • Numpy 1.2.1
• Jetty to run the MockServlet

Setup

I use a custom installation of Lettuce.  The main version of Lettuce doesn't support HTML output yet.  If you don't care about HTML output, you can just install the standard Lettuce with pip.

If you DO want the HTML output, I have a custom fork of Lettuce that provides this support.  It is available here:
https://github.com/camd/lettuce/tree/html_output

To install this package, check out the source code locally, then do the following:

$ cd lettuce
$ python setup.py install

Next, install the required package numpy:

$ pip install numpy

Running the tests

There are two parts to the tests, currently:

1. Mock Java Servlet
2. Tests written in Lettuce

Both are checked into Github here:
https://github.com/mozilla/tcm

Mock Servlet

The mock Servlet is written in Java, and is checked out with the tests. The source is checked in, but since this servlet is very simple and stable, I just checked in the latest .war file here to make it easier on you:

/tcm/tests/mockservlet/TcmMockServlet.war

Copy this file to the /webapps of your servlet container. For example:

/jetty-6.1.14/webapps

Then when you start or restart your servlet container, you'll be in business.

Tests

First: cd to the parent of the/features directory

$ cd tcm/tests/bdd

One last bit of configuration to check. The /features/Terrain.py file contains values that specify your host and port among other things. If you have different values than these defaults, you'll need to modify them:

world.hostname = "localhost"
world.port = 8080

Finally, you're ready to run the tests. Here's how:

$ lettuce

That will produce your standard text output to the command line. If you want to run and build an HTML file of the result, do this: Example

$ lettuce -v 5 > tcmout.html

Then load that file into the browser of your choice. However, if you don't use Firefox or Minefield, then you may experience the Helvetica Scenario, so please choose wisely.