previous meeting | Meetings | next meeting »

Dial in

 # 650-903-0800 or 650-215-1282 x92 Conf# 315 (US/INTL)
 # 1-800-707-2533 (pin 369) Conf# 315 (US)
 # irc.mozilla.org #mozmill for backchannel

Attendees

• Anthony, Dave, Aaron, Al, Cameron, Geo, Matt, Henrik

Last weeks action Items

• [MISSED] Geo: Update documentation for handling of failing Mozmill tests
  • At this point, have lost the thread on this. Need to touch base with Henrik to find out what final plan was and where docs should be updated.
• [DONE] Anthony: Update coding styles and keep discussion on-going to get remaining important rules defined

Goals Overview

• Q1 goals

Risky Goals

Project Status

On-demand update (bft) tests (Geo/Henrik)

• Project page started here, still filling in info.
• Henrik has made great progress on staging downloads
  • bug 528064
  • bug 628659
• Other part is simultaneous test launch
  • Currently planning on using ssh/dsh, or similar
  • Will be staging a test version on qa-mozmill

Endurance Tests (Dave/Anthony)

• Tracking bug created. bug 629065
• Patch uploaded for shared-module and initial test snippet. bug 629069
• Patch uploaded for endurance testrunner. bug 629077
• Created branch for endurance tests reports in Mozmill dashboard. bug 629083 repository
• Reports have been generated using the current patches and published to a couchone account:
  • Open/Close Add-ons Manager
    • Iterations: 5, 10, 20, 50, 100, 200

Panorama Tests (Anthony/Al)

• Community contributor (uyclay) has 4 tests which are nearly ready to checkin
• Drag&Drop issue bug 515776 "resolved"
• All tests should now be non-blocked, in theory
• If you want to help out, see the list and assign something to yourself

Redesign Shared Modules (Geo/Henrik)

• Tentative project plan has been added here. Still undergoing review b/w Henrik and Geo, but will be finalized this week.
• Will be sending out periodic information for review by team as we hit milestones
• Plan on team-wide pilot at the end of Q1, where everyone can try implementing tests against the library and give feedback.
• Waverley has also expressed interest in ramping up into automation development; they can help us out a lot by being a "test community."

Shadow test server (Al/?)

• Initial Apache server being created in an IT-run ESX server. See bug 621794.
• Once server is up, will need to create some sort of rsync script to mirror content from QA repo initially.
• Will also need to determine what certificates we want for https testing.

Others

Mozmill Crowd

• Mozmill Crowd 0.1 and an improved version 0.1.1 has been released
• We already have some daily users (81 downloads, 8 active users)
• Working on project page updates and milestones for this quarter
• Next planned version of Mozmill Crowd by next week
• Mozmill Crowd talk during FOSDEM 2011 on Sunday, Feb 6th

Dashboard

• Working on project updates and milestones for this quarter
• Implementation of necessary views for l10n tests earlier this week

Cloud Automation (Al/?)

• Creating initial project page
  • Determining initial milestones, requirements, and dependencies.
  • This is investigation in the immediate, near-term

Tests / Shared Modules

General (Henrik)

• No progress - first Panorama tests still under review

Broken Tests

• Firefox 4.0
  • Landed a lot of fixes; down to one single test failure on Linux
• Firefox 3.5/3.6 (Anthony/Aaron)
  • Bug with window.close() not being properly handled (bug 626674) - fix landed, will check testrun tomorrow to check for residual failures resolved
  • Bug with testPasteLocationBar (bug 624682) - reverted from new controller MenuAPI to using the old controller click access of popup menu items, revealed another issue on 3.5 with keypress copying

Automation Scripts / Infrastructure (Henrik)

• Download script for builds has been finished and can be used
• Work happens now on the wrapper script to auto-start download and the test-run.

Personal Status

For the personal status please check the weekly status updates:

• 110117 - 110121
• 110124 - 110128

Roundtable

General

• Testing of 32bit builds on Ubuntu 64?
• Value of the execution of Mozmill tests for all locales (Henrik)
  • l10: A huge amount of issues has been covered by the testrun
    • Results
    • Screenshots
    • Tests take about 80 minutes
  • Update: 1 crasher bug has been identified for 3 locales (bug 628829)
    • Results
    • Tests take about 2h for partial updates
  • Pros vs. Cons
    • Check stability and quality of all localized builds
    • No need for manual update spot checks
    • Limited number of machines (what about the new order?)
    • We should test from Tier1 -> Tier3 (limit to Tier1 for smoketests and test channels?)
    • What about BFT tests? A single test-run for a locale will take about 12 minutes. We could run in parallel while Waverly tests manually

Issues

Meeting Notes

• Endurance
  • A valuable resource of information would be graphs over graphs
  • Compare test-runs to see improvement/degradation of memory usage or performance
• Shadow Test Server
  • We need a 2nd lead on the project.
  • Send an email to the list if you have interest
  • Otherwise Henrik will take the position
• Mozmill Dashboard
  • Overlay results with appropriate bug ids
  • Overlay results with additionally meta data added by the logged in user
• General
  • Everyone should check the own handling of projects and collect ideas for improvements in next quarter
  • For test failures we have to focus on 3.6 test failures before 3.5 failures

Action items

Anthony

• Update refactoring discussion items and communicate to team
• Make sure spreadsheet is updated with Panorama status

Al

• Talk with Anthony to find the requirements for SSL certificats on the shadow server

Geo

• Finish on-demand updates project page
• Figure out what jetpack is doing for docgen (ask myk/analyze sdk)
• Document failed test process from henrik's email at appropriate place

Henrik

• Send Geo desired place for failed test docs
• Add 32-bit version of Firefox to our daily testruns on Ubuntu 64-bit

Matt/Al

• Matt/Al: work to find the goal for cloud automation goal