QA/Test Automation/2010-05-19

From MozillaWiki
Jump to: navigation, search

previous meeting | Meetings | next meeting »


  • Matt, Tony, Ayan, Anthony, Al, Mark Locklear, Henrik


Test Creation
  • Implementation of one Add-ons Manager test (Tobbi)
  • Landed helper function for DTD entities => tests need to be updated (Adrian)
Test Fixing
  • 2 tests have been fixed
  • 3 tests under review (Anthony, Mark)
Update Tests
  • Have been rewritten to use the new TestRun class
  • Can now automatically distinct between minor and major updates
Cloud Testing
  • Talked with Clint and we have to re-think the way how to setup the environment for the user (need to talk to AMO security guys)
Add-ons Testing
  • First tests for Google Toolbar under review
  • Needs test-run script by end of this week (Testday May 28th)
(Release) Testing
  • BFT test-run script has been rewritten to use the new TestRun class (can install builds now)
  • Crontab helper script has been created, which runs daily Mozmill tests against Shiretoko, Namoroka, and Minefield
  • First run of daily Minefield builds showed a lot of failures, have to update the API's as soon as possible
Result Dashboard
  • No progress


  • Last two weeks
    • See project updates
  • Next two weeks
    • Work with Google team and create Add-ons test-run script
    • Update existing shared API's to fix failures in Minefield
    • Documentation for QMO2
    • Requirements for cloud testing extension
  • Last two weeks
    • 3 patch reviews
    • Added getDefaultHomepage() to UtilsAPI
    • Fixed 2 tests using getDefaultHomepage()
      • testUntrustedConnectionErrorPage
      • testRestoreHomepageToDefault
    • Took on role of Release Update Tester
      • Ran update checks via Mozmill across several releases:
        • 3.6.4b1->3.6.4b3, betatest & beta
        • 3.5.x->3.5.10, betatest & beta
        • 3.6.x->3.6.4b3, betatest & beta
        • 3.5.10->3.6.4b3, betatest & beta
        • 3.6.x -> 3.6.4build4, betatest & beta
        • 3.6.4build3 -> 3.6.4build4 betatest & beta & release
        • Platforms: Win7, WinXP, Ubuntu, MacOSX
  • Next two weeks
    • fix remaining Security failures (priority: testBlueLarry)
    • continue to run update checks for any/all releases
    • review patches as necessary
    • test Henrik's new update scripts
Al Billings
  • Last two weeks: Nothing but Firefox 3.6.4 sustainability work and minor tasks
  • Next two weeks:
    • Shipping 3.6.4 after smoketesting build 5 on 6/1. (This will free up my time.)
    • Working on Bug 553616 and Bug 553617 Awesomebar tests. Hope to have first test ready for review within a week.

Contributor Status

  • Name


  • New machines for Mozmill tests (daily test-runs vs. release testing)
    • See bug 566795
      • Suggesting 2 Mac Pros
      • Specs: Dual 2.26GHz quad-core, 8GB RAM, 640GB HDD
      • Cost: $3400 USD each
      • Value:
        • Allows us to run 5 VMs + local OS in tandem (no more shutting down, starting up VMs)
        • No more waiting for daily test runs to finish for release/update testing and test development
    • Challenges:
      • Get the new machines set up with VMs, Mozmill environments
      • Get the new machines on the VPN
      • Other challenges?
  • QMO2 Content Migration
    • Who and what?
  • Mozmill 1.4.1 not in review queue on AMO
Team Video
  • Introduction (what is Mozmill, why do we use it, what is special...)
  • Overview about existing projects (from test creation, over automation and reporting to analyzing reports)

Action Items

  • Henrik will check this weeks testday and talk with jgriffin about the used test framework and how it can help us for the cloud testing extension
  • Henrik will work with Anthony to create a good set of QMO2 content
  • Anthony will help to drive the Mozmill Addons Scripting testday next week
  • Anthony should check all broken tests for possible use of the default homepage
  • Matt will comment on the bug for the proposal of machines (3 choices) and send an email to our internal list for discussion.
  • Matt: We should also do test-runs against 64bit builds (if environments are available)
  • Matt: A completely automated release testing cycle is wanted for the future
  • Matt and Henrik will work on getting an template up for individual status reporting on the team page