QA/Test Automation/2010-12-01

previous meeting | Meetings | next meeting »

Dial in

 # 650-903-0800 or 650-215-1282 x92 Conf# 315 (US/INTL)
 # 1-800-707-2533 (pin 369) Conf# 315 (US)
 # irc.mozilla.org #mozmill for backchannel

Last weeks action Items

• ? Al (Anthony)?: Check existing tests if they are already covered by mochitests and worth to transform local
• Aaron (Geo): Tagging of tests and how to get them into buildbot
• [DONE] Geo (Henrik): Schedule meeting about trigger notifications for test-runs
• [DONE] Geo: Start thread about which parts the Etherpad document should cover
• [DONE] Matt (Henrik): Finding a lead and 2nd lead for the Panorama goal
• [DONE] Matt (Henrik): Goals review and addition of new goals to team goals
• [ON TRACK] Henrik: Work with Waverly on updating the spreadsheet for Add-ons Manager tests - add labels for automated tests

Goals Overview

• QA Q4 goals

Project Status

Endurance Tests

• Hoping to kick this off today - Dave/Aaron

Panorama Tests

• 15 / 21 identified tests do not rely on Drag & Drop
• ashughes:
  • Spoke with Matt about taking lead on this and I am interested
  • I know Al and Cameron were interested as well so can we get a final word on this?

Mozmill Crowd Extension (Henrik)

• Currently: Collecting dependencies and building test-environments for all platforms
• Next: move to nsIProcess API

Mozmill Result Dashboard (Henrik)

• Reworking project pages
• Next: Proposal for visualization and queries of update results

Build bot - local data (Anthony)

• No change - high resource cost, low value at this point
• Doable: 4 test pages + 1 test
• Much depends on proxy and certificate server handling which is not currently doable
• Not likely to see any change this quarter

Others

DOM Walker + L10n API (Adrian)

Tests / Shared Modules

• General (Henrik)
  • Refactored modal dialog has been landed on all branches
    • Some hidden older failures have been revealed
    • Some regressions introduced
    • Interests from Simon Steward to use the module for Selenium

• Test Refactoring (Anthony)
  • Styleguide drafted - final call for review/feedback
    • We need a template to serve as example of this styleguide
    • What is the criteria of review comment vs review rejection? We should be very explicit on this so that there are no surprises with review results. NOTE: This might be more appropriate for this document and linking the two.
  • Tasks for Phase I have been outline and will go into a project page shortly
  • Expect work to begin next week

• Broken Tests
  • Firefox 4.0 (Geo)
  • Firefox 3.5/3.6 (Anthony)
    • 3.6: 18 Failures - up from 13 since modalDialogAPI
    • 3.5: 18 Failures - up from 9 since modalDialogAPI
    • Aaron focusing on modalDialogAPI refactoring in tests
    • Anthony focusing on pre-existing failures
    • Fix Rate: 3/week

Automation / Infrastructure

• Update Testing (Henrik)
  • Reports now support multiple updates in a row
• Add-ons Testing (Henrik)
  • no update

Personal Status

For the personal status please check the weekly status updates:

• 101129 - 101203

Roundtable

General

• Project management software (Henrik)
  • See weekly etherpad

Issues

Meeting Notes