QA/Test Automation/2011-02-23

From MozillaWiki
Jump to: navigation, search

previous meeting | Meetings | next meeting »

Dial in

 # 650-903-0800 or 650-215-1282 x92 Conf# 315 (US/INTL)
 # 1-800-707-2533 (pin 369) Conf# 315 (US)
 # #mozmill for backchannel


  • Anthony, Henrik, Aaron, Matt, Al, Geo, Adrian, Dave

Last weeks action Items

  • Finish on-demand updates project page
    • Page is here. It's in pretty good shape, but needs to be updated by Henrik for download script design
  • Document failed test process from henrik's email at appropriate place
    • In progress, have info from Henrik, not finished.
  • [DONE] Create project page for Cloud Testing project
  • Find out if we can get metrics for the upcoming qatest vm
  • [DONE] Find out if it is possible for us to upload files to QMO (test environment)
  • [DONE] Add links to project pages for project headings
  • Create PHP script for downloading test environments

Goals Overview

Risky Goals

Project Status

On-demand update tests (Geo/Henrik)

  • Finished to the demo stage
  • Now does the staged download from config file and Pulse multiplexed kickoff
  • Demoed for Matt on 2/22
  • Need to plan for deployment on qa-set

Endurance Tests (Dave/Anthony)

  • Working on several new tests including Panorama and Flash tests
  • Hopefully submitting a patch this week for the --addons command line parameter
  • Attempting to replicate performance related bugs

Panorama Tests (Anthony/Al)

  • 2 tests landed but had to be backed out due to unseen regressions during local testing
  • Clay is helping us investigate
  • remaining tests still in development

Redesign Shared Modules (Geo/Henrik)

  • Sprint 1 finished, branch diff under review by Henrik
  • Sprint 2 underway
  • Henrik has done initial implementation of assert/expect modules, needs revision after feedback from A-team
  • Geo has picked Back/Forward/Home button tests to begin porting in order to flesh controls, is in process now.
  • Some open questions remain about how to align this with A-team's plans for Mozmill 2.0 and possibly Add-ons SDK testing in the future.

Shadow test server (Al/Henrik)

  • Still waiting for IT to create the VM.
    • Apparently we now need a security review in order for this to happen. No updates from security team yet when asked how to do this.
  • Anyone with work blocked by this should file a bug and make it dependent on bug 621794.


Mozmill Crowd

  • Investigating a way how to distribute Mercurial pre-installed in test environments without requiring the user to install Xtools on OS X (bug 545432)


  • Targeting a couple of minor fixes to enhance the handling of queries

Cloud Automation (Al/Henrik)

  • Al is going to examine use of Amazon for Linux VMs.

Tests / Shared Modules

General (Henrik)
  • No updates
Broken Tests
  • Firefox 4.0
    • We are clean except some latest regressions introduced by website and client changes (3 failures)
  • Firefox 3.5/3.6 (Anthony/Aaron)
    • no change
    • website regressions (above) affect branches as well

Automation Scripts / Infrastructure (Henrik)

  • no updates

Personal Status

For the personal status please check the weekly status updates:


  • [CARRY OVER] WebQA Test Automation status inclusion?
  • Refactoring the mozmill-tests repository
    • As more and more test-runs get added as more confusing the current structure is
    • Proposal:
      • addons
      • endurance
      • lib (formerly shared-modules/)
      • general (formerly firefox/)
      • l10n
      • restart (probably not necessary with mozmill 2.0)
      • test-files
      • update
  • Work week to follow all-hands?
  • Module refactor, Add-Ons SDK and other external concerns?

Meeting Notes

  • Q1 goals
    • update tests
      • pretty good shape
      • staging script and pulse execution script already can be executed
    • endurance tests
      • already full-filled the goal
      • even more tests will be implemented
    • panorama
      • targeting to get 50% of the tests done
    • shared modules
      • way behind the schedule - we have to keep up
      • concerns with working with a-team
    • shadow server
      • still blocked on IT for setting up the VM
      • branch out ourselves if no response
  • Projects
    • Cloud Testing
      • Amazon optimized for server installations - different environment
  • Round Table
    • WebQA
      • Structure QA Automation into different groups Desktop, WebQA, Mobile
      • Label meeting as Desktop QA Automation
      • Definition during all-hands week
    • Work week for qa automation
      • Sounds fine for after the all hands week
    • External concerns
      • Be careful about interfaces because that's what test writers will have to use
      • Still keep an eye on other projects and investigate
      • Investigate tests for jetpacks into the mozmill crowd extension

Action items

  • Talk with IT to get the shadow server setup
  • [DONE] Send mail to qa-auto for proposal of mozmill-tests refactor (bug 637306)
  • [DONE] Send mail for possible qa automation work week