Releases/Firefox 25/Test Plan: Difference between revisions
< Releases
Jump to navigation
Jump to search
Simona Badau (talk | contribs) |
|||
| Line 54: | Line 54: | ||
| class="header" | Owner | | class="header" | Owner | ||
| class="header" | Documentation | | class="header" | Documentation | ||
| class="header" | Status | | class="header" | Status | ||
|- | |- | ||
| Web Audio | | Web Audio | ||
| Manuela Muntean | | Manuela Muntean | ||
| [https://wiki.mozilla.org/WebAudio/Test_Plan Test Plan] | | [https://wiki.mozilla.org/WebAudio/Test_Plan Test Plan], [https://bugzilla.mozilla.org/show_bug.cgi?id=779297 Tracking bug] | ||
| {{done|}} Ready for Beta | |||
| {{done|}} | |||
|- | |- | ||
| Prototype DOM Gamepad API | | Prototype DOM Gamepad API | ||
| Alexandra Lucinet | | Alexandra Lucinet | ||
| [https://bugzilla.mozilla.org/show_bug.cgi?id=604039 Tracking bug] | | [https://bugzilla.mozilla.org/show_bug.cgi?id=604039 Tracking bug] | ||
| | | {{drop|}} Delayed to an unspecified milestone | ||
|- | |- | ||
| Find Bar Redesign | | Find Bar Redesign | ||
| Manuela Muntean | | Manuela Muntean | ||
| [https://wiki.mozilla.org/FindBarRedesign/Test_Plan Test Plan] | | [https://wiki.mozilla.org/FindBarRedesign/Test_Plan Test Plan], [https://bugzilla.mozilla.org/show_bug.cgi?id=565552 Tracking Bug] | ||
| {{done|}} Ready for Beta | |||
| {{done|}} | |||
|- | |- | ||
| Background Thumbnail Service | | Background Thumbnail Service | ||
| Paul Silaghi | | Paul Silaghi | ||
| [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=870100 Tracking Bug] | |||
| [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=870100 | | {{drop|}} Delayed to Firefox 26 via {{bug|912763}} | ||
| | |||
|- | |- | ||
| Reset Firefox Saves Windows and Tabs | | Reset Firefox Saves Windows and Tabs | ||
| Ioana Budnar | | Ioana Budnar | ||
| [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=833943 Tracking Bug] | |||
| [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=833943 | | {{done|}} Ready for Beta | ||
| {{done|}} | |||
|- | |- | ||
| Reset Profile When Unused for Months | | Reset Profile When Unused for Months | ||
| Simona Marcu | | Simona Marcu | ||
| [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=498181 Tracking Bug] | |||
| [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=498181 | | {{done|}} Ready for Beta | ||
| {{done|}} | |||
|- | |- | ||
| CSS3 background-attachment:local Support | | CSS3 background-attachment:local Support | ||
| Petruta Rasa | | Petruta Rasa | ||
| [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=483446 Tracking Bug] | |||
| [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=483446 | | {{done|}} Ready for Beta | ||
| {{done|}} | |||
|- | |- | ||
| Support in-line HTML5 iframes | | Support in-line HTML5 iframes | ||
| Alexandra Lucinet | | Alexandra Lucinet | ||
| [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=802895 Tracking Bug] | |||
| [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=802895 | | {{ok|}} Sign-off in Progress | ||
| {{ok|}} | |||
|- | |- | ||
| Session Restore (backend work + tab close & restore) | | Session Restore (backend work + tab close & restore) | ||
| Cornel Ionce | | Cornel Ionce | ||
| [https://bugzilla.mozilla.org/show_bug.cgi?id=910167 Tracking Bug] | |||
| [https://bugzilla.mozilla.org/ | | {{ok|}} Sign-off in Progress | ||
| {{ok|}} | |||
|- | |- | ||
| Larry Panel | | Larry Panel | ||
| Bogdan Maris | | Bogdan Maris | ||
| [https://bugzilla.mozilla.org/show_bug.cgi?id=885366 Tracking Bug] | |||
| [https://bugzilla.mozilla.org/ | | {{done|}} Ready for Beta | ||
| {{done|}} | |||
|- | |- | ||
| Emacs Map#forEach and Set#forEach | | Emacs Map#forEach and Set#forEach | ||
| Paul Silaghi | | Paul Silaghi | ||
| [https://bugzilla.mozilla.org/show_bug.cgi?id=866847 Tracking Bug] | |||
| [https://bugzilla.mozilla.org/show_bug.cgi?id=866847 | | {{done|}} QA not needed | ||
| | |||
|- | |- | ||
| ES6 math functions | | ES6 math functions | ||
| Mihai Morar | | Mihai Morar | ||
| [https://bugzilla.mozilla.org/show_bug.cgi?id=717379 Tracking Bug] | |||
| [https://bugzilla.mozilla.org/show_bug.cgi?id=717379 | | {{done|}} QA not needed | ||
| | |||
|} | |} | ||
Revision as of 18:22, 5 September 2013
« Firefox 24 | Firefox 25 | Firefox 26 »
Firefox 25 Test Plan
Summary
| Status | Currently in Aurora -- merging to Beta on September 19, 2013 |
| Lead | Anthony Hughes (irc: ashughes) |
| 2nd | Juan Becerra (irc: juanb) |
| Builds | aurora.mozilla.org |
The following is the test plan for Firefox 25 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 25. After the release this document will be used as an archive of what was done to validate this release.
If you have some free time, please pick a task below and get in touch with one of the leads.
Schedule
| June 24, 2013 | Firefox 25 lands on Nightly |
| August 5, 2013 | Firefox 25 moves up to Aurora |
| September 19, 2013 | Firefox 25 moves up to Beta |
| October 22, 2013 | Firefox 25 Release Candidate available for testing |
| October 29, 2013 | Firefox 25 is released |
Features
- Sign-off Criteria
- All user stories covered by manual or automated tests
- Manual testing complete with no blocking regressions
- Automated testing complete with no blocking regressions
- 0 untriaged unconfirmed bugs
- 0 untriaged unverified fixes
- Status
| Feature | Owner | Documentation | Status |
| Web Audio | Manuela Muntean | Test Plan, Tracking bug | [DONE] Ready for Beta |
| Prototype DOM Gamepad API | Alexandra Lucinet | Tracking bug | [DROPPED] Delayed to an unspecified milestone |
| Find Bar Redesign | Manuela Muntean | Test Plan, Tracking Bug | [DONE] Ready for Beta |
| Background Thumbnail Service | Paul Silaghi | Tracking Bug | [DROPPED] Delayed to Firefox 26 via bug 912763 |
| Reset Firefox Saves Windows and Tabs | Ioana Budnar | Tracking Bug | [DONE] Ready for Beta |
| Reset Profile When Unused for Months | Simona Marcu | Tracking Bug | [DONE] Ready for Beta |
| CSS3 background-attachment:local Support | Petruta Rasa | Tracking Bug | [DONE] Ready for Beta |
| Support in-line HTML5 iframes | Alexandra Lucinet | Tracking Bug | [ON TRACK] Sign-off in Progress |
| Session Restore (backend work + tab close & restore) | Cornel Ionce | Tracking Bug | [ON TRACK] Sign-off in Progress |
| Larry Panel | Bogdan Maris | Tracking Bug | [DONE] Ready for Beta |
| Emacs Map#forEach and Set#forEach | Paul Silaghi | Tracking Bug | [DONE] QA not needed |
| ES6 math functions | Mihai Morar | Tracking Bug | [DONE] QA not needed |
- Important URLs
Automation Triage
- Important URLs
- What to Do
- Check the daily Mozmill reports (see above) for any failing tests
- If you notice a failure, open the report to identify the test and failure message
- Correlate this failure to a known bug by checking the known failures (see above) in Bugzilla
- If the bug is NEW or ASSIGNED add a comment indicating the platform(s) and locale(s) affected, including a URL to any reports
- If the bug is RESOLVED or VERIFIED, add a comment indicating the platform(s) and locale(s) affected, including a URL to any reports, and set the status to REOPENED
- If there is no bug, report it and list it below
- Bugs Reported
- no bugs reported
Daily BFT Testing
- Important URLs
- Moztrap tests
- Moztrap test results
- Draft ideas for tests
- What to Do
- Run the Moztrap tests for your operating system
- For any test that fails, check the known bugs
- If the bug is UNCONFIRMED provide details about your environment and your steps to reproduce the issue, set the status to NEW
- If the bug is FIXED provide details about your environment and your steps to reproduce the issue, ask if the bug should be REOPENED
- If there is no bug report, report it and list it below
- Bugs Reported
- no bugs reported
Crashkill
- Conduct daily stability checks
- Review the explosiveness reports
- If a crash meets topcrash criteria
- Report a bug via one of the crash reports if there isn't already a bug on file
- If there's already a bug on file make sure it has the topcrash keyword, tracked and is escalated accordingly
- Use bughunter to find URLs and reproduce a pageload crash for a given signature, correlate to crashstats
Bug Triage
| Query | Description | Remaining |
| QAWANTED | Bugs requiring immediate investigative assistance | 0 |
| VERIFYME | Bugs with fixes requiring immediate verification | 38 |
| FIXED | Bugs with fixes which should be verified as soon as possible | 160 |
| REGRESSION | Bugs requiring identification of the first affected build | 226 |
| UNCONFIRMED | Bugs requiring confirmation if they are reproducible in a current build | 70 |
Known Issues
356 Total; 0 Open (0%); 165 Resolved (46.35%); 191 Verified (53.65%);
Aurora
Branch Migration
Pre Sign-off Tasks
- [DONE] Firefox branch migrated (Release Management)
- [DONE] Mozmill Tests branch migrated (Automation Development)
- [DONE] Updates throttled on aurora channel (Release Engineering)
- [DONE] Updates verified throttled on aurora channel (QA)
Sign-off Tasks
- [DONE] Mozmill Daily functional testruns pass with no unexpected failures/regressions (report)
- [DONE] Mozmill Daily l10n testruns pass with no unexpected failures/regressions (report)
- [DONE] Mozmill Daily endurance testruns pass with no unexpected failures/regressions (report)
- [DONE] Mozmill Daily remote testruns pass with no unexpected failures/regressions (report)
- [DONE] Mozmill Daily auroratest update testruns pass with no unexpected failures/regressions (report)
Post Sign-off Tasks
- [DONE] Aurora sign-off email sent (QA)
- [DONE] Updates unthrottled on aurora channel (Release Engineering)
- [DONE] Updates verified unthrottled on aurora channel (QA)
- [MISSED] Verify Stub Installers stage the new Aurora build (bug 910442)
Weekly Testing Status
Week 1
- Automation: 0 unknown failures (report)
Week 2
- Automation: 0 unknown failures (report)
Week 3
- Automation: 0 unknown failures (report)
Week 4
- Automation: 0 unknown failures (report)
Week 5
Week 6
Pre-Beta Sign-off
- Features
- Functional testing
- Bug status