QA/TDAI/Test Farm Roadmap

Q4 2006

• (DONE) create Windows XP SP2 test-specific reference platform
• (DONE) install RHEL and XP reference platform to internal test machines
• (DONE) assist Alice in setup of test harness, tests and graph server

Q1 2007

• create Vista, Windows server, Linux and Mac test-specific reference platforms
• identify and order batches of test hardware - at least two of each ref platform on each applicable hardware configuration
  • document hardware platform
• document procedure for installing new test machines e.g.
  • for IT:
    • installing ref image
    • installing client software
  • for build:
    • basic configuration
  • for QA:
    • decide on right location for test harness, tests and test data (CVS?), changes picked up automatically on test machines
• decide on driver for performance tests (e.g. Buildbot). Needs:
  • trigger test based on build availability
  • report test run status to tinderbox
  • farm out jobs to pools of machines as-needed
• port performance test harness to Mac/Linux
• buildbot testing infrastructure
  • (DONE) setup linux master
  • (DONE) setup linux & win32 slave
  • setup mac slave
  • (DONE) setup make check
  • (DONE) setup reftest on linux, win32 & mac
  • (DONE) document installation
  • (DONE) setup tinderbox & email notification (reporting to MozillaTest
  • (DONE) add mochitest
  • prettify tinderbox reports
  • add js tests
  • add/port leak tests