QA/Test Automation/2010-12-01

previous meeting | Meetings | next meeting »

Dial in

 # 650-903-0800 or 650-215-1282 x92 Conf# 315 (US/INTL)
 # 1-800-707-2533 (pin 369) Conf# 315 (US)
 # irc.mozilla.org #mozmill for backchannel

Last weeks action Items

• ? Al (Anthony)?: Check existing tests if they are already covered by mochitests and worth to transform local
• Aaron (Geo): Tagging of tests and how to get them into buildbot
• [DONE] Geo (Henrik): Schedule meeting about trigger notifications for test-runs
• [DONE] Geo: Start thread about which parts the Etherpad document should cover
• [DONE] Matt (Henrik): Finding a lead and 2nd lead for the Panorama goal
• [DONE] Matt (Henrik): Goals review and addition of new goals to team goals
• [ON TRACK] Henrik: Work with Waverly on updating the spreadsheet for Add-ons Manager tests - add labels for automated tests

Goals Overview

• QA Q4 goals

Project Status

Endurance Tests

• Hoping to kick this off today - Dave/Aaron
  • Kickoff ideas/brainstorming - http://etherpad.mozilla.com:9000/KabTTn3qnA

Panorama Tests

• 15 / 21 identified tests do not rely on Drag & Drop
• ashughes:
  • Spoke with Matt about taking lead on this and I am interested
  • I know Al and Cameron were interested as well so can we get a final word on this?

Mozmill Crowd Extension (Henrik)

• Currently: Collecting dependencies and building test-environments for all platforms
• Next: move to nsIProcess API

Mozmill Result Dashboard (Henrik)

• Reworking project pages
• Next: Proposal for visualization and queries of update results

Build bot - local data (Anthony)

• No change - high resource cost, low value at this point
• Doable: 4 test pages + 1 test
• Much depends on proxy and certificate server handling which is not currently doable
• Not likely to see any change this quarter

Others

DOM Walker + L10n API (Adrian)

Tests / Shared Modules

• General (Henrik)
  • Refactored modal dialog has been landed on all branches
    • Some hidden older failures have been revealed
    • Some regressions introduced
    • Interests from Simon Steward to use the module for Selenium

• Test Refactoring (Anthony)
  • Styleguide drafted - final call for review/feedback
    • NOTE: Please be aware that there are some things left out of this version of the styleguide. The initial version serves to communicate those things we have all already agreed on. Discussion and revision will improve on version 1.0.
    • We need a template to serve as example of this styleguide
    • What is the criteria of review comment vs review rejection? We should be very explicit on this so that there are no surprises with review results. NOTE: This might be more appropriate for this document and linking the two.
    • JSDoc commenting style - not final so I'd like to leave out things like tags from this version of the styleguide. A simplified comment-only version should be good enough for this version of the styleguide.
  • Tasks for Phase I have been outline and will go into a project page shortly
  • Expect work to begin next week

• Broken Tests
  • Firefox 4.0 (Geo)
  • Firefox 3.5/3.6 (Anthony)
    • 3.6: 18 Failures - up from 13 since modalDialogAPI
    • 3.5: 18 Failures - up from 9 since modalDialogAPI
    • Aaron focusing on modalDialogAPI refactoring in tests
    • Anthony focusing on pre-existing failures
    • Fix Rate: 3/week

Automation / Infrastructure

• Update Testing (Henrik)
  • Reports now support multiple updates in a row
• Add-ons Testing (Henrik)
  • no update

Personal Status

For the personal status please check the weekly status updates:

• 101129 - 101203

Roundtable

General

• Project management software (Henrik)
  • See weekly etherpad
  • Proposal for wiki pages

Issues

Meeting Notes