QA/Test Automation/2010-06-16

From MozillaWiki
Jump to: navigation, search

previous meeting | Meetings | next meeting »


  • adrian, aaron, al, anthony, geo, henrik, matt
Who's new
  • Introducing Aaron Train and Adrian Kalla


Test Creation
  • 2 new BFT tests (SessionStore and Add-ons Manager)
Broken Tests
  • Firefox 3.7
    • Fixed 28 broken tests (API updates, Litmus data)
    • ~8 broken normal tests left (5 of them happen for all platforms)
    • All broken restart tests fixed
  • Firefox 3.5/3.6
    • Mozmill Failures
    • Fixed 5 failed tests (Litmus Data, SSL Disabled)
    • ~22 broken tests which fail reliably:
      • Failed 5/5 local test runs on at least one platform and one branch
      • Reported in Brasstacks within the last 24 hours
    • Top-10 Fails based on local reproducibility and brasstacks frecency:
      • testRestoreHomepageToDefault
      • testClearFormHistory [Mark Locklear]
      • testDownloadManagerClosed
      • testPasswordNotSaved [ashughes]
      • testDisabledElements
      • testAccessLocationBar
      • testIdentityPopupOpenClose [ashughes]
      • testOpenInBackground
      • testSafeBrowsingNotification [aaronmt]
      • testSuggestHistoryBookmarks

Update Tests
  • Redesigned SoftwareUpdateAPI to work with Firefox 3.7
  • Needs back porting to branches (after testing)
  • Fixes a problem when preBuildID = postBuildID
Cloud Testing
  • Talked with Clint for remaining questions
  • Started to create the design specs
Add-ons Testing
  • Working on Fidesfit extension integration (bug 570493)
(Release) Testing
  • Both MacPro machines have been setup completely
  • Daily tests on 10.6, Ubuntu, Win2000, WinXP, Win Vista, Win7, Win7 64-bit
Result / Dashboard

Risky Goals

  • Result dashboard


  • Last two weeks
    • Supplying and verifying patches for Mozmill 1.4.2
    • Updated shared API's (SoftwareUpdate, Search)
    • Finally fixed long outstanding handleWindow helper function bug 518386
    • Fixed 5 broken Mozmill tests
    • Created set of result data proposal
  • Next two weeks
    • Finish Result Data project and get it applied for Mozmill 1.4.2
    • Finishing spec on Cloud Testing extension and have a prototype working
    • Write initial specs for Web Dashboard
    • Collecting ideas for Q3 goals
  • Last two weeks
    • Slow progress made on Session Store BFTs thanks to Aaron
      • 1 test complete, 1 test in-progress
    • Reviewed 14 patches
    • 7 cycles of release update testing
    • Resolved Litmus Data and SSLDisabled failures
  • Next two weeks
    • Fix top 10 failures (Aaron et al to assist)

  • Last two weeks
  • Next two weeks:
  • Last two weeks:
  • Next two weeks:
    • Continuing ramp up
    • Update tests to use nsIBrowserHistory.removeAllPages() Bug 572403
    • More test failure fixing
  • Last two weeks:
  • Next two weeks:
    • Taking a look at test failures
    • Continue session tests if possible
  • Last two weeks:
  • Next two weeks:
    • Prepare a time line for Mozmill L10n Tests creation
    • Start working on making the current tests work on localized builds

Contributor Status

  • Name


  • Break-out session and science fair attendance submitted
  • Strange hangs of Mozmill during test-runs on several machines (bug 571630)
  • Auto-complete tests failures due to a problem with Shiretoko (bug 572143)

Action Items

  • Run Mozmill tests against 3.7 and update failure spreadsheet (Anthony)
  • Investigation and fixing of 3.7 only failures (Geo and Henrik)
  • Feedback for result data proposal (Everyone)
  • Setting team goals for Q3 (Henrik)