QA/Test Automation/2010-08-25

From MozillaWiki
Jump to: navigation, search

previous meeting | Meetings | next meeting »


  • Al, Geo, Matt, Anthony, Aaron, Henrik, Adrian

Last weeks action Items

  • [DONE] Henrik: Send testday invitation to mozmill-dev and planet
  • [DONE] Henrik: We need a bug for Mozmill 2.0 to allow usage of different web servers (bug 590432)
  • [DONE] Henrik: Create outline proposal for team recruiting video
  • [MISSED] Matt: Check what limits test harnesses on buildbot to use other local webservers
  • [DONE] Matt: Discuss with Marcia the length of each video for teams
  • [DONE] Geo: We need feedback from Clint about the scope and feature set for Mozmill 2.0
  • [MISSED] All: Check the proposed docs for the shared module refactoring from Geo


Mozmill (Henrik)
  • Mozmill 1.5 has been released on Monday
    • Thanks everyone for your help in filing bugs and verifying fixes!
  • Mozmill 1.5.1: Not targeted yet but will contain bug fixes and the events patch
  • Mozmill 2.0: Add your proposals
  • We should wait at least a week until we upgrade Mozmill on our qa-set and qa-horus boxes
  • Post-Mortem for Mozmill 1.5
    • Scheduled maintenance releases with clear set of fixes and features (1 per month?)
    • Setup a list of prioritized feature requests for 2.0
Test creation / shared modules (Henrik)
  • All l10n backports have been landed for older branches (1.9.2 and 1.9.1)
Local Test Data (Anthony)
  • 20% complete
  • No progress due to heavy release work (4.0b4, 3.6.9, 3.5.12)
  • Targeting Sept 1st completion
Broken Tests
  • Firefox 4.0 (Geo)
    • Broken tests are still broken, re: missed goal in personal section below.
    • Have started test runs with Mozmill 1.5 final, will be generating new bugs/fix list from those runs.
  • Firefox 3.5/3.6 (Anthony)
  • No update
Update Tests (Henrik)
  • SoftwareUpdateAPI has been successfully backported to 1.9.2 bug 567258
  • Releng is interested to run Mozmill nightly updates bug 588398

MozMill Crowd extension (Henrik)
  • no update
Add-ons Testing (Henrik)
  • no update
Automation (Henrik)
  • Automation scripts allow a custom repository url now bug 571978
  • Support software updates for Thunderbird bug 580611
  • Add support to automation scripts to modify the report data bug 586336
  • Renamed BFT testrun to General testrun bug 586513
  • Mozmill calls sys.exit() - restart-test run doesn't get executed bug 589003
  • Other fixes:
    • Global timeout of Mozmill killed software update tests bug 586652
    • Use canonical path to test modules bug 586792 (needs review)
Dashboard (Henrik)
  • No update - I can hopefully start this week

Risky Goals


  • Last two weeks
    • 1 week vacation
    • [ON TRACK] Create shared module for add-ons manager and finalize list of tests
    • [DONE] Update all automation scripts to be compatible with Mozmill 1.5
    • [DROPPED] Update the software update script to send full reports to brasstacks (next quarter)
    • [MISSED] Finalize specs for Web dashboard
    • [MISSED] Continue work on the crowd extension
  • Next two weeks
    • Update AddonsAPI and tests to work with Firefox 4.0
    • Create list of Mozmill tests to implement
    • Create spec for the dashboard (General test-run pages)
  • Last two weeks
    • [MISSED] Complete conversion of 31 tests to use local-data
  • Next two weeks
    • Complete conversion of 31 tests to use local-data
  • Last two weeks
    • Nothing, focused on release testing work.
  • Next two weeks
    • Get script for Bug 553618 re-fixed and submitted.
    • Figure out what we can do with microsummary restrictions.
  • Last two weeks:
    • Continuing with Shared API Refactor project
    • [MISSED] Fixes for bug 573582, bug 573584, bug 575250
    • Reviewing test results for any additional breakage
    • Help with 1.4.2 beta release as needed
  • Next two weeks:
    • Generating functioning prototypes for refactored UI modules
    • Collecting team feedback re: prototypes
    • [CARRY OVER] Fixes for bug 573582, bug 573584, bug 575250
  • Last two weeks:
    • Help with the 1.4.2 release
    • Continue conversion of tests to use local-data
    • Continue investigating Linux VM failures (qa-mozmill/horus)
  • Next two weeks:
    • [CARRY OVER] of conversion of 31 tests to use local-data
    • Refactoring tasks in tests for 1.5
  • Last two weeks:
    • [DONE] L10n support in existing tests backport to 1.9.1
    • [ON TRACK] prototyped idea of CommonA11yL10nAPI and L10nAPI
  • Next two weeks:
    • Documentation + UML Diagrams for CommonA11yL10nAPI and L10nAPI
    • implementation of the CommonA11yL10nAPI and L10nAPI
    • Targeting the end of September for all the L10n work completion

Contributor Status

  • Name


  • QA Automation video
    • 1st level: Global overview to recruite new community members (Work Week)
    • How can we knock someone up to help out in our team?
      • What is Mozmill
      • Why is UI automation important (i.e. time reduction, different locales)
      • In which areas you can help
        • Test execution and result reporting (Crowd extension)
        • Test and shared module creation
        • Test failure analysis (dashboard) and fixing broken tests
        • Improving automation scripts and the Crowd extension
    • Suggestions
      • High level what we do and how you get involved
      • Matrix of people in the room without automation vs. with automation
  • Sience Fair like introduction video of QA members (eventually Work Week)
  • Detailed QA Automation videos with members
    • 2nd level: Howto for major automation goals by team members (All Hands?)
      • In depth presentation of main projects with screencasts
  • QA Automation presentation (Henrik)
    • Who has time to help and put the presentation together?
    • Henrik has existing presentations from a couple of talks as template
  • Release testing automation tasks (Matt)
    • yesterday it became evident that we need to redefine automation tasks and responsibilities for release testing
  • Improve communication with Auto-Tools team (Matt)

Action Items

  • Geo: Schedule meeting for debugging/execution labs for automation
  • Geo: Send out list of broken tests w/ Fx4 and MM1.5
  • [DONE] Henrik: Update QA lab machines and databases to use Mozmill 1.5 (Sep. 1st)
  • [ON TRACK] Henrik: Send out email for more details to introduction (recruiting) video
  • [DONE] Henrik: Send out email to start the work for the automation presentation
  • [DONE] Henrik: Schedule meeting for release tasks and machine reservation
  • [DROPPED] Henrik: Setup a list of prioritized feature requests for 2.0
    • Will work together with the A-team during our work week
  • Matt: Check what limits test harnesses on buildbot to use other local webservers (connect with John O'Duin)
  • [DONE] Matt: Request a post mortem for Mozmill 1.5
  • All: Check the proposed docs for the shared module refactoring from Geo