QA/Mozmill Test Automation/On Demand Update Testing/Documentation

From MozillaWiki
Jump to navigation Jump to search

Prep

Things in this section should be done before a release has landed.

  1. Check for a functioning heartbeat emitter. Run one if unsure.
  2. Set up listeners on all desired platforms, grouped with a common cluster name.
  3. Create a configuration file for release staging.
  4. If update testing, pre-stage the release.

Heartbeat Emitter

  • Generates traffic to keep Pulse connections from being automatically closed.
  • Must be running -somewhere- on the internet. Can be anywhere.
  • Fine to run more than one. If you're not sure, boot one.

Usage

./heartbeat_emitter.py

Options/Behavior

No other requirements. It will print out local time every time a heartbeat is sent, so you can diagnose time of any server failures or freezes.

Release Listener

  • Listens for a release request and automatically runs the correct script
  • Run one per platform. Make sure a heartbeat has been set up.

Usage

./release_listener.py --help
Usage: release_listener.py [options] cluster platform

Launches a listener for the on-demand testing system. The listener will only 
respond to requests for the specified cluster. The listener will respond to
requests for either the specified platform or for all platforms.

Options:
  -h, --help            show this help message and exit
  -u UPDATE_SCRIPT, --update=UPDATE_SCRIPT
                        update test script to run [default:
                        ./release_update.py]
  -f FUNCTIONAL_SCRIPT, --functional=FUNCTIONAL_SCRIPT
                        functional test script to run [default:
                        ./release_bft.py]
  -d, --debug           print out extra debug information on ignored messages

Sample command lines

./release_listener.py qa-foobar mac

This runs a listener in the qa-foobar cluster, platform set to mac.

Options/Behavior

  • Internally, our clusters are the hostnames of the Mac Pro that groups the platform VMs.
  • Platform must exactly match one of platform specifiers used in the staging (and thus, usually the config file, see blow). Otherwise, it's an arbitrary string.
  • --update will redefine which test script is called for update requests. The default accomodates our internal systems.
  • --functional will redefine which test script is called for functional requests. Again, the default accomodates our internal systems.
  • --debug lets you know when things are ignored due to platform/cluster mismatches. This can be very noisy, but it's useful when you have no idea why something isn't responding.
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Mozmill_Test_Automation/On_Demand_Update_Testing/Documentation&oldid=339254"