Releases/Firefox 12/Test Plan

From MozillaWiki
< Releases
Revision as of 10:17, 19 December 2011 by Otilia.anica (talk | contribs) (Created page with "= Firefox 12 Test Plan = == Summary == {| class="fullwidth-table" |- | style="width:15%" | '''Milestone''' | [https://wiki.mozilla.org/Firefox/Flight_Tracking#Firefox_12 Firef...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Firefox 12 Test Plan

Summary

Milestone Firefox 12
Status Currently in Nightly -- Moves to Aurora on January 31, 2012
Lead Juan Becerra (irc: juanb)
2nd Anthony Hughes (irc: ashughes)

The following is the test plan for Firefox 12 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 12. After the release this document will be used as an archive of what was done to validate this release.

If you have some free time, please pick a task below and get in touch with one of the leads.

Details

The following is a description of the tasks which need to be completed throughout this release.

Task Owner Status
Automation Triage Softvision ON TRACK - 2011-11-14
Bug Triage Softvision ON TRACK - 2011-11-14
Feature Sign-offs Softvision ON TRACK - 2011-11-14
Full Functional Tests (Aurora) iBeta MISSED? - 2011-11-14
Website Compatibility (Aurora) uTest MISSED? - 2011-11-14
Milestones
Beta 2 ashughes Builds expected 2011-11-16
Beta 1 ashughes DONE - 2011-11-11
Aurora Merge ashughes DONE - 2011-09-27
Nightly Merge ashughes DONE - 2011-08-16

Automation

BFTs

Task Triage the Mozmill dashboard for test failures and regressions
Lead Vlad Maniac (Softvision)
Peer Alex Lakatos (Softvision)
Status NEW (12), ASSIGNED (9), FIXED (3), REGRESSIONS (0)
Instructions
  • Check the Mozmill dashboard for failing tests for the day
  • Cross reference any failure to the list of NEW and ASSIGNED bugs
    • If the bug is NEW, assign it to yourself and try to fix it
    • If the bug is ASSIGNED and there has been no activity on the bug in over a week, offer your assistance to fix the bug
    • If there is NO bug on file, file it using this template
  • If you need assistance, ask ashughes in the #mozmill channel
Bugs Found

Endurance

Task Triage the Mozmill Endurance dashboard for test failures and performance regressions
Lead Mihaela Velimiroviciu (Softvision)
Peer
Status Spreadsheet (Nightly)
Instructions
  • Check the dashboard for the performance numbers from the most recent nightly
  • Add the data to the spreadsheet
  • Report any performance spikes or test failures to Dave Hunt and Anthony Hughes
Bugs Found

Bug Triage

Lead Vlad Maniac
Peer Alex Lakatos
Queries QA+, FIXED, UNCONFIRMED
Bugs Found

Features

Track features which have landed in Firefox 9 and are ready to be tested (owners)

The following is a statement of feature readiness for release.

Feature Owner Status
JS Type Inference N/A No QA Needed - 2011-11-14 (ashughes)
Do Not Track Signal Ioana Budnar SIGNED OFF on beta 6 - 2011-12-14
Multi-process IndexDB N/A Unscoped as per e10s status - 2011-11-11
PFS2 Datasource Virgil Dicu No release target specified. Nothing implemented yet. - 2011-12-14
Links
Scoped Features, Unscoped Features

Full Functional Tests

The following tracks testing done by iBeta for full functional test coverage. This task is performed once during Aurora only.

  • Lead: Anthony Hughes
  • [ link to test suite]
  • [ link to results of testing]
  • [ link to bugs filed by iBeta]

Website Compatibility

The following tracks testing done by uTest for website compatibility. This task is performed after every merge.

  • Lead: Anthony Hughes
  • Link to test suite
  • Link to report for Nightly Web Compatibility
  • Link to report for Aurora Web Compatibility
  • Link to report for Beta Web Compatibility
  • Link to bugs filed

Nightly Merge

Verify that the builds generated from the merge to Nightly have updated properly.

Nightly
  • [DONE] [ashughes] Windows XP: PASS
    • Source: Mozilla/5.0 (Windows NT 5.1; rv:8.0a1) Gecko/20110813 Firefox/8.0a1
    • Target: Mozilla/5.0 (Windows NT 5.1; rv:9.0a1) Gecko/20110817 Firefox/9.0a1
  • [DONE] [ashughes] Windows 7: PASS
    • Source: Mozilla/5.0 (Windows NT 6.1; rv:8.0a1) Gecko/20110814 Firefox/8.0a1
    • Target: Mozilla/5.0 (Windows NT 6.1; rv:9.0a1) Gecko/20110817 Firefox/9.0a1
  • [DONE] [ashughes] Mac OS X 10.6: PASS
    • Source: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:8.0a1) Gecko/20110812 Firefox/8.0a1
    • Target: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0a1) Gecko/20110817 Firefox/9.0a1
  • [DONE] [ashughes] Ubuntu 11.04: PASS
    • Source: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:8.0a1) Gecko/20110815 Firefox/8.0a1
    • Target: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:9.0a1) Gecko/20110817 Firefox/9.0a1
Issues Found

Aurora Merge

The following are the detailed results of the automated BFT test runs.

  • [DONE] [ashughes] Windows 2000: PASS
  • [DONE] [ashughes] Windows XP: PASS
  • [DONE] [ashughes] Windows Vista: PASS
  • [DONE] [ashughes] Windows 7: PASS
  • [DONE] [ashughes] Windows 7 64-bit: PASS
  • [DONE] [ashughes] Mac OS X 10.6: PASS
  • [DONE] [ashughes] Mac OS X 10.7: PASS
  • [DONE] [ashughes] Ubuntu 11.04: PASS
  • [DONE] [ashughes] Ubuntu 11.04 64-bit: PASS
  • Reports: Mozmill (8.0a2)
Issues Found
  • Two tests are constantly failing and which need to be fixed as soon as possible.

Updates

The following are details of update verifications for Firefox 9 merge to Aurora

Auroratest
  • [DONE] [ashughes] Windows XP: PASS
    • Source: Mozilla/5.0 (Windows NT 5.1; rv:8.0a2) Gecko/20110920 Firefox/8.0a2
    • Target: Mozilla/5.0 (Windows NT 5.1; rv:9.0a2) Gecko/20110929 Firefox/9.0a2
  • [DONE] [ashughes] Windows 7: PASS
    • Source: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0a2) Gecko/20110926 Firefox/8.0a2
    • Target: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0a2) Gecko/20110929 Firefox/9.0a2
  • [DONE] [ashughes] Mac OS X 10.6: PASS
    • Source: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:8.0a2) Gecko/20110926 Firefox/8.0a2
    • Target: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0a2) Gecko/20110929 Firefox/9.0a2
  • [DONE] [ashughes] Ubuntu 11.04: PASS
    • Source: Mozilla/5.0 (X11; Linux i686; rv:8.0a2) Gecko/20110923 Firefox/8.0a2
    • Target: Mozilla/5.0 (X11; Linux i686; rv:9.0a2) Gecko/20110929 Firefox/9.0a2
Aurora
  • [DONE] [ashughes] Windows XP: PASS
    • Source: Mozilla/5.0 (Windows NT 5.1; rv:8.0a2) Gecko/20110920 Firefox/8.0a2
    • Target: Mozilla/5.0 (Windows NT 5.1; rv:9.0a2) Gecko/20110930 Firefox/9.0a2
  • [DONE] [ashughes] Windows 7: PASS
    • Source: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0a2) Gecko/20110926 Firefox/8.0a2
    • Target: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0a2) Gecko/20110930 Firefox/9.0a2
  • [DONE] [ashughes] Mac OS X 10.6: PASS
    • Source: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:8.0a2) Gecko/20110926 Firefox/8.0a2
    • Target: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0a2) Gecko/20110930 Firefox/9.0a2
  • [DONE] [ashughes] Ubuntu 11.04: PASS
    • Source: Mozilla/5.0 (X11; Linux i686; rv:8.0a2) Gecko/20110923 Firefox/8.0a2
    • Target: Mozilla/5.0 (X11; Linux i686; rv:9.0a2) Gecko/20110930 Firefox/9.0a2
Issues Found
Retrieved from "https://wiki.mozilla.org/index.php?title=Releases/Firefox_12/Test_Plan&oldid=379825"