Releases/Firefox 24/Test Plan
From MozillaWiki
< Releases
« Firefox 23 | Firefox 24 | Firefox 25 »
Contents
- 1 Firefox 24 Test Plan
- 1.1 Summary
- 1.2 Details
- 1.3 Features
- 1.4 Known Issues
- 1.5 Release Candidate
- 1.6 Beta
- 1.7 Pre-beta Feature testing and Sign-off
- 1.8 Aurora Merge
- 1.9 Bug Triage
- 1.10 Nightly
Firefox 24 Test Plan
Summary
| Milestone | Firefox 24 |
| Status | Currently in Beta - to Release on Week of September 17, 2013 |
| Lead | Tracy Walker (irc: tracy) |
| 2nd | Anthony Hughes (irc: ashughes) |
| Builds | [ftp] [http] [web] |
The following is the test plan for Firefox 24 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 24. After the release this document will be used as an archive of what was done to validate this release.
If you have some free time, please pick a task below and get in touch with one of the leads.
Details
The following is a description of the tasks which need to be completed throughout this release.
- 2013.05.14: [DONE] Nightly Merge
- 2013.06.24: [DONE] Aurora Merge
- 2013.08.02: [DONE] Pre-Beta feature sign-offs
- 2013.08.08: [DONE] Beta 1
- 2013.08.13: [DONE] Beta 2
- 2013.08.15: [DONE] Beta 3
- 2013.08.20: [DONE] Beta 4
- 2013.08.22: [DONE] Beta 5
- 2013.08.27: [DONE] Beta 6
- 2013.08.29: [DONE] Beta 7
- 2013.09.03: [DONE] Beta 8
- 2013.09.05: [DONE] Beta 9
- 2013.09.10: [DONE] Beta 10
- 2013.09.13: [DONE] Release Candidate
- 2013.09.17: [DONE] Final
Features
See the Feature Release Tracking wiki for a general overview
| Feature | Owner | Status | Tests |
Endurance Automation
The purpose of this section is to track any potential memory regressions via spikes reported in the dashboard.
- What to Do
- Check the Endurance Charts
- If you see an anomalous event (ie. spike or dip), get the report for the specific event
- If the event happens twice in a row, file a bug under Mozmill Tests
- Notify Anthony Hughes of the issue
- Record the event in this wiki below
- Current Status (Mihai Morar) - Last Updated - 2013.07.26
| ID | Summary | Priority | Status |
|---|---|---|---|
| 887791 | Higher memory consumption on Mac 10.7.5 and 10.8.4 since June the 22nd | -- | VERIFIED |
1 Total; 0 Open (0%); 0 Resolved (0%); 1 Verified (100%);
Functional Automation
The purpose of this section is to track any potential functional regressions via results reported in the dashboard.
- What to Do
- Check the testrun reports
- If you see any failing tests, check that there are not any bugs on file for it already
- If not, file a bug under Mozmill Tests
- Notify Tracy Walker of the issue
- Record the event in this wiki below
- Current Status (SoftVision) - Last Updated - yyyy-mm-dd
Bug Triage
The purpose of this section is to triage UNCONFIRMED incoming bugs.
- What to Do
- Check the UNCONFIRMED bugs
- Test the bugs in the latest Firefox Nightly
- If you can't reproduce the bug, ask for the reporter to provide more information
- If you reproduce the bug, add your testing information and mark the bug NEW
- Notify Tracy Walker of the NEW issue
- Current Status (SoftVision) - Last Updated
Bug Fix Verifications
note: this doesn't show the security fixes that have been verified.
250 Total; 0 Open (0%); 41 Resolved (16.4%); 209 Verified (83.6%);
Known Issues
15 Total; 1 Open (6.67%); 14 Resolved (93.33%); 0 Verified (0%);
Release Candidate
Build Info
| Builds | FTP, HTTP |
| Build ID | 20130910201120 |
| Changes | pushlog |
| Configs | mozmill |
Checklist
| Task | Status |
| Regression Tests | [DONE] |
| Automated Tests | [DONE] |
| Bug Fix Verifications | [DONE] |
| Update Tests (betatest) | [DONE] |
| Update Tests (releasetest) | [DONE] |
| Update Tests (beta) | [DONE] |
| Stability & Support sign-off | [DONE] |
Regression Tests
Exploratory testing on the below areas:
- Breakpad End-to-End functionality (guidelines)
- Partner repack update (guidelines)
- Windows share compatibility smoketests (guidelines)
| Platform | Status (results) |
| Windows 7 64-bit | [DONE] |
| Windows XP | [DONE] |
| Mac OSX 10.8 | [DONE] |
| Ubuntu 12.04 32-bit | [DONE] |
Automated Tests
| Test | Result | Criteria |
| Automated Tests | PASS (details) | All Mozmill ondemand_functional testruns PASS with no new failures
|
Failures (known)
Update Tests
| Test | Result | Criteria |
| Update Tests (betatest) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (releasetest) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (beta) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
New Issues
Bugzilla query error
error, http-bad-status, Array
Beta
Beta 10
Build Info
| Builds | FTP, HTTP |
| Build ID | 20130909203154 |
| Changes | pushlog |
| Configs | mozmill |
Checklist
| Task | Status |
| Regression Tests | [DONE] |
| Automated Tests | [DONE] |
| Bug Fix Verifications | [ON TRACK] on going in beta cycles |
| Update Tests (betatest) | [DONE] |
| Update Tests (releasetest) | [DONE] |
| Update Tests (beta) | [DONE] |
| Stability & Support sign-off | [DONE] |
Regression Tests
Exploratory testing on the below areas:
- Breakpad End-to-End functionality (use this MozTrap test for guidelines).
- Web compatibility exploratory smoketests
| Platform | Status (results) |
| Windows 7 64-bit | [DONE] |
| Windows 8 32-bit | [DONE] |
| Mac OSX 10.6 | [DONE] |
| Ubuntu 12.04 32-bit | [DONE] |
Automated Tests
| Test | Result | Criteria |
| Automated Tests | PASS (details) | All Mozmill ondemand_functional testruns PASS with no new failures
|
Failures (known)
Update Tests
| Test | Result | Criteria |
| Update Tests (betatest) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (releasetest) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (beta) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
New Issues
No results.
0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);
Beta 9
Build Info
| Builds | FTP, HTTP |
| Build ID | 20130905180733 |
| Changes | pushlog |
| Configs | mozmill |
Checklist
| Task | Status |
| Regression Tests | [DONE] |
| Automated Tests | [DONE] |
| Bug Fix Verifications | [ON TRACK] on going in beta cycles |
| Update Tests (betatest) | [DONE] |
| Update Tests (releasetest) | [DONE] |
| Update Tests (beta) | [DONE] |
| Stability & Support sign-off | [DONE] |
Regression Tests
Exploratory testing on the below areas:
- WebRTC (one-on-one calls from each OS to each OS, multiple attendants calls, Firefox to Firefox, Firefox to other browers etc).
- The multiple attendants calls have delays, burbles, and background noise - according to Bug 902428 these issues are improved in the latest Nightly 26.0a1 and are not fixed in Firefox 24 beta.
- Breakpad End-to-End functionality (use this MozTrap test for guidelines).
| Platform | Status (results) |
| Windows 7 32-bit | [DONE] |
| Windows 8 64-bit | [DONE] |
| Mac OSX 10.7 32-bit mode | [DONE] |
| Ubuntu 12.04 64-bit | [DONE] |
Automated Tests
| Test | Result | Criteria |
| Automated Tests | PASS (details) | All Mozmill ondemand_functional testruns PASS with no new failures
|
Failures (known)
Update Tests
| Test | Result | Criteria |
| Update Tests (betatest) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (releasetest) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (beta) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
New Issues
| ID | Summary | Priority | Status |
|---|---|---|---|
| 913043 | [adbe 3627468] Flash 11.9 crash in F1987069282 | -- | RESOLVED |
| 913449 | Contextual menu not visible around an iframe containing Flash content | P5 | RESOLVED |
| 913474 | Scrollbar overlaps the chat window | -- | RESOLVED |
3 Total; 0 Open (0%); 3 Resolved (100%); 0 Verified (0%);
Beta 8
Build Info
| Builds | FTP, HTTP |
| Build ID | 20130902131354 |
| Changes | pushlog |
| Configs | mozmill |
Checklist
| Task | Status |
| Regression Tests | [DONE] |
| Automated Tests | [DONE] |
| Bug Fix Verifications | [ON TRACK] on going in beta cycles |
| Update Tests (betatest) | [DONE] |
| Update Tests (releasetest) | [DONE] |
| Update Tests (beta) | [DONE] |
| Stability & Support sign-off | [DONE] |
Regression Tests
Exploratory testing on the below areas:
- SocialAPI demo (http://mixedpuppy.github.io/socialapi-demo/)
- Print to file
- PDF Viewer
| Platform | Status (results) |
| Windows 7 32-bit | [DONE] |
| Windows 8 64-bit | [DONE] |
| Mac OSX 10.7 | [DONE] |
| Ubuntu 64-bit | [DONE] |
Automated Tests
| Test | Result | Criteria |
| Automated Tests | PASS (details) | All Mozmill ondemand_functional testruns PASS with no new failures
|
Failures (known)
- test Failure bug 912023 - ' A new window has been opened' in /testPrivateBrowsing/testOpenClosePBKeyboardShortcut.js
Update Tests
| Test | Result | Criteria |
| Update Tests (betatest) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (releasetest) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (beta) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
New Issues
| ID | Summary | Priority | Status |
|---|---|---|---|
| 911981 | Chat panels are lost while detaching/attaching | -- | VERIFIED |
| 912110 | crash in nsXPCWrappedJS::Release() with DatamngrHlpFF24.dll (Bandoo) | -- | RESOLVED |
2 Total; 0 Open (0%); 1 Resolved (50%); 1 Verified (50%);
Beta 7
Build Info
| Builds | FTP, HTTP |
| Build ID | 20130829135643 |
| Changes | pushlog |
| Configs | mozmill |
Checklist
| Task | Status |
| Regression Tests | [DONE] |
| Automated Tests | [DONE] |
| Bug Fix Verifications | [ON TRACK] on going in beta cycles |
| Update Tests (betatest) | [DONE] |
| Update Tests (releasetest) | [DONE] |
| Update Tests (beta) | [DONE] |
| Stability & Support sign-off | [DONE] |
Regression Tests
Exploratory testing around the following areas:
- Audio/Video (H.264 and MP3 decoding).
- Content Security Policy (guidelines in bug csp and wiki - most of this feature is already tested automatically, so only bug work and exploratory testing is needed, http://csptesting.herokuapp.com/).
- DevTools: Browser Console (errors, warnings, info, logs of all types).
| Platform | Status (results) |
| Windows XP | [DONE] |
| Windows 7 64-bit | [DONE] |
| Mac OSX 10.8 | [DONE] |
| Ubuntu 32-bit | [DONE] |
Automated Tests
| Test | Result | Criteria |
| Automated Tests | PASS (details) | All Mozmill ondemand_functional testruns PASS with no new failures
|
Failures (known)
Update Tests
| Test | Result | Criteria |
| Update Tests (betatest) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (releasetest) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (beta) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
New Issues
| ID | Summary | Priority | Status |
|---|---|---|---|
| 911164 | Current window loses focus temporarily when hovering the pointer over the QuickTime volume button | P4 | RESOLVED |
1 Total; 0 Open (0%); 1 Resolved (100%); 0 Verified (0%);
Beta 6
Build Info
| Builds | FTP, HTTP |
| Build ID | 20130826142034 |
| Changes | pushlog |
| Configs | mozmill |
Checklist
| Task | Status |
| Regression Tests | [DONE] |
| Automated Tests | [DONE] |
| Bug Fix Verifications | [ON TRACK] on going in beta cycles |
| Update Tests (betatest) | [DONE] |
| Update Tests (releasetest) | [DONE] |
| Update Tests (beta) | [DONE] |
| Stability & Support sign-off | [DONE] |
Regression Tests
Exploratory testing around the following areas:
- New Tab page (all options/functionality)
- Unified search preferences (Search bar, about:home, contextual menu option, URL bar)
| Platform | Status (results) |
| Windows XP | [DONE] |
| Windows 7 64-bit | [DONE] |
| Mac OSX 10.8 | [DONE] |
| Ubuntu 32-bit | [DONE] |
Automated Tests
| Test | Result | Criteria |
| Automated Tests | PASS (details) | All Mozmill ondemand_functional testruns PASS with no new failures
|
Failures (known)
Update Tests
| Test | Result | Criteria |
| Update Tests (betatest) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (releasetest) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (beta) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
New Issues
| ID | Summary | Priority | Status |
|---|---|---|---|
| 909700 | DuckDuckGo Plus gives strange error message after installation on a clean profile | -- | RESOLVED |
| 909715 | DuckDuckGo Plus remains the selected engine in the search bar after changing it with Google and restarting Firefox | -- | RESOLVED |
2 Total; 0 Open (0%); 2 Resolved (100%); 0 Verified (0%);
Beta 5
Build Info
| Builds | FTP, HTTP |
| Build ID | 20130822154523 |
| Changes | pushlog |
| Configs | mozmill |
Checklist
| Task | Status |
| Regression Tests | [DONE] |
| Automated Tests | [DONE] only de, en-US, fr, it run by automation |
| Bug Fix Verifications | [ON TRACK] on going in beta cycles |
| Update Tests (betatest) | [DONE] |
| Update Tests (releasetest) | [DONE] |
| Update Tests (beta) | [DONE] |
| Stability & Support sign-off | [DONE] |
Regression Tests
Exploratory testing around the following areas:
- SocialAPI (chat - chat, open/close/detach/rearrange multiple chat windows, comment, like, share etc)
- Session Restore (manual and automated restore, restore at startup, restore after crash etc)
- Printing (use as many printing options as possible)
| Platform | Status (results) |
| Windows Vista | [DONE] |
| Windows 7 32-bit | [DONE] |
| Mac OSX 10.6 | [DONE] |
| Ubuntu 32-bit | [DONE] |
Automated Tests
| Test | Result | Criteria |
| Automated Tests | PASS (details) | All Mozmill ondemand_functional testruns PASS with no new failures
|
Failures (known)
Update Tests
| Test | Result | Criteria |
| Update Tests (betatest) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (releasetest) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (beta) |
result (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
New Issues
| ID | Summary | Priority | Status |
|---|---|---|---|
| 908639 | Scrollbar overlapping the "Choose an emoticon" button in a chat window | -- | RESOLVED |
| 908641 | Chat area misplaced when the chat is detached | -- | RESOLVED |
| 908642 | Chat window is duplicated, after it was previously detached | -- | RESOLVED |
| 908643 | Last message and writing area are not visible in the chat window after re-attaching it | -- | RESOLVED |
| 908646 | Messages are hidden when detaching and attaching the window chat | -- | RESOLVED |
| 908648 | "Turn On" button remains disabled after pressing Oops, undo this! | -- | RESOLVED |
| 908650 | Chatting is still possible after turning the chat off | -- | RESOLVED |
| 908651 | The sidebar overlaps the chat window | -- | VERIFIED |
| 908654 | Chat window settings button misplaced upwards | -- | RESOLVED |
9 Total; 0 Open (0%); 8 Resolved (88.89%); 1 Verified (11.11%);
Beta 4
Build Info
| Builds | FTP, HTTP |
| Build ID | 20130819170952 |
| Changes | pushlog |
| Configs | mozmill |
Checklist
| Task | Status |
| Regression Tests | [DONE] |
| Automated Tests | [DONE] |
| Bug Fix Verifications | [DONE] on going in beta cycles |
| Update Tests (betatest) | [DONE] |
| Update Tests (releasetest) | [DONE] |
| Update Tests (beta) | [DONE] |
| Stability & Support sign-off | [DONE] |
Regression Tests
Exploratory testing around the following areas:
- Crash Reporter spot checks (use this MozTrap test as a guideline)
- WebRTC
- Using the Moztrap test as a guideline, verify that Firefox 24.0b4 is no worse than Firefox 23.0.1 when making AppRTC calls of >5 minutes in length.
| Caller Platform | Callee Platform | Caller Browser | Callee Browser | Result | Notes/Issues |
| Windows XP | Windows 7 | Firefox 23.0.1 | Firefox 24.0b4 | [DONE] | Caller: ~2s delay immediately Callee: no issues |
| Windows XP | Windows 8 | Firefox 23.0b4 | Firefox 23.0.1 | [DONE] | Caller: ~2s delay immediately Callee: ~2s delay immediately |
| Windows 7 | Windows XP | Firefox 23.0.1 | Firefox 24.0b4 | [DONE] | Caller: poor sound quality (unclear noise), delay (2s) after 5 minutes, increasing as time goes by Callee: ~2s delay immediately |
| Windows 7 | Mac OSX | Chrome Stable | Firefox 24.0b4 | [DONE] | Caller: no delay, poor sound quality Callee: poor sound quality (unclear noise), no delay |
| Windows 7 | Linux | Firefox 24.0b4 | Firefox 23.0.1 | [DONE] | Caller: no issues Callee: no issues |
| Windows 8 | Windows XP | Firefox 24.0b4 | Firefox 23.0.1 | [DONE] | Caller: ~1.5s delay immediately |
| Windows 8 | Mac OSX | Firefox 23.0.1 | Firefox 24.0b4 | [DONE] | Caller: ~2s delay immediately Callee: ~1.5s delay immediately |
| Windows 8 | Linux | Chrome Stable | Firefox 24.0b4 | [DONE] | Caller: no issues Callee: no issues |
| Mac OSX | Windows 7 | Firefox 24.0b4 | Firefox 24.0b4 | [DONE] | Caller: no issues Callee: no issues |
| Mac OSX | Linux | Firefox 23.0.1 | Firefox 24.0b4 | [DONE] | Caller: no issues Callee: no issues |
| Linux | Windows 7 | Firefox 24.0b4 | Firefox 23.0.1 | [DONE] | Caller: increasing delay (2+ seconds) after 10 minutes Callee: no delay, poor sound quality (unclear noise) |
| Linux | Windows 8 | Chrome Stable | Firefox 24.0b4 | [DONE] | Caller: no delay, small hangs, good audio/video quality Callee: small hangs if browsing other websites during chat, no delay, good audio/video quality |
| Linux | Mac OSX | Firefox 23.0.1 | Firefox 24.0b4 | [DONE] | Caller: small hangs, poor sound quality (unclear noise), no delay Callee: small hangs, no delay *Note: at min 30, the call interrupted, the video blocked and audio stopped. |
When testing, here are the guidelines to follow:
- Have only one browser open on each machine at any given time
- Have only one call running at any given time on a particular machine
- Make sure the caller and callee are always on *different* machines.
- Please don't test any 3+-way calls for this sanity check -- We just want to see the results for 1:1 (basic) calling
- If testers find regressions during their testing, Randell and Maire can help track down if the regressions are real or not
| Platform | Status (results) |
| Windows Vista | [DONE] |
| Windows 7 32-bit | [DONE] |
| Mac OSX 10.6 | [DONE] |
| Ubuntu 32-bit | [DONE] |
Automated Tests
| Test | Result | Criteria |
| Automated Tests | PASS (details) | All Mozmill ondemand_functional testruns PASS with no new failures
|
Failures (known)
- bug 907101 Test failure "Certificate's country code is displayed - '(US)' should equal '(US,postalCode=94041,STREET=650 Castro St Ste 300)'" in testGreenLarry.js was seen in every testrun.
Update Tests
| Test | Result | Criteria |
| Update Tests (betatest) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (releasetest) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (beta) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
New Issues
| ID | Summary | Priority | Status |
|---|---|---|---|
| 907213 | Small hangs on webRTC calls initiated from Linux | P4 | RESOLVED |
| 907214 | webRTC call freezes after 20 to 30 minutes on https://apprtc.webrtc.org/ page | -- | RESOLVED |
2 Total; 0 Open (0%); 2 Resolved (100%); 0 Verified (0%);
Beta 3
Build Info
| Builds | FTP, HTTP |
| Build ID | 20130815180504 |
| Changes | pushlog |
| Configs | mozmill |
Checklist
| Task | Status |
| Regression Tests | [DONE] |
| Automated Tests | [DONE] |
| Bug Fix Verifications | [ON TRACK] on going in beta cycles |
| Update Tests (betatest) | [DONE] |
| Update Tests (releasetest) | [DONE] |
| Update Tests (beta) | [DONE] |
| Stability & Support sign-off | [DONE] |
Regression Tests
Exploratory testing around the following areas:
- Crash Reporter spot checks (use this MozTrap test as a guideline)
| Platform | Status (results) |
| Windows 7 64-bit | [DONE] |
| Windows 8 32-bit | [DONE] |
| Mac OSX 10.7 | [DONE] |
| Ubuntu 64-bit | [DONE] |
- Notes & Issues Found
Automated Tests
| Test | Result | Criteria |
| Automated Tests | PASS (details) | All Mozmill ondemand_functional testruns PASS with no new failures
|
Failures (known)
Update Tests
| Test | Result | Criteria |
| Update Tests (betatest) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (releasetest) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (beta) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
New Issues
| ID | Summary | Priority | Status |
|---|---|---|---|
| 880140 | crash in nsDeviceContextSpecX::EndPage @ decode_data when printing | -- | VERIFIED |
1 Total; 0 Open (0%); 0 Resolved (0%); 1 Verified (100%);
Beta 2
Build Info
| Builds | FTP, HTTP |
| Build ID | 20130812173056 |
| Changes | pushlog |
| Configs | mozmill |
Checklist
| Task | Status |
| Regression Tests | [DONE] |
| Automated Tests | [DONE] |
| Bug Fix Verifications | [ON TRACK] on going in beta cycles} |
| Update Tests (betatest) | [DONE] |
| Update Tests (releasetest) | [DONE] |
| Update Tests (beta) | [DONE] |
| Stability & Support sign-off | [DONE] |
Regression Tests
Exploratory testing around the following areas:
- Mac OSX scrollbars (simple pages, rich content pages, long pages etc)
- WebRTC (one-on-one calls, multiple attendants calls, Firefox to Firefox, Firefox to other browers etc)
- PDF Viewer (open simple and complex PDFs, print PDF, complete PDF form etc)
- Network Monitor (different protocols, status codes, cookies etc)
| Platform | Status |
| Windows 7 64-bit | [DONE] PASS |
| Windows 8 32-bit | [DONE] PASS |
| Mac OSX 10.7 | [DONE] PASS |
| Ubuntu 12.10 64-bit | [DONE] PASS |
Automated Tests
| Test | Result | Criteria |
| Automated Tests | PASS (details) | All Mozmill ondemand_functional testruns PASS with no new failures
|
Failures (known)
Update Tests
| Test | Result | Criteria |
| Update Tests (betatest) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (releasetest) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (beta) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
New Issues
| ID | Summary | Priority | Status |
|---|---|---|---|
| 904561 | [10.7] Scrollbars shouldn't grow on hover on OSX 10.7 (only on 10.8 and up) | -- | VERIFIED |
| 904575 | crash [@ libobjc.A.dylib@0xb150][@ libobjc.A.dylib@0xb158] | -- | RESOLVED |
| 904598 | WebRTC not working on Ubuntu and Mac OS X machines | -- | VERIFIED |
3 Total; 0 Open (0%); 1 Resolved (33.33%); 2 Verified (66.67%);
Beta 1
Because of l10n repack issues on Windows, Beta 1 will ship with all locales on Mac and Linux but only en-US on Windows. Release August 9th of 24.0beta1build#2
Build Info
| Builds | FTP, HTTP |
| Build ID | 20130806170643 |
| Changes | pushlog |
| Configs | mozmill |
Checklist
| Task | Status |
| Feature Sign-off | [DONE] |
| Regression Tests | [DONE] |
| Automated Tests | [DONE] |
| Bug Fix Verifications | [ON TRACK] on going in beta cycles |
| Update Tests (betatest) | [DONE] |
| Update Tests (releasetest) | [DONE] |
| Update Tests (beta) | [DONE] |
| Stability & Support sign-off | [DONE] |
Feature Sign Off
| Feature | QA Contact | Status |
| link to bug or feature page | contact(s) | [DONE] status |
Notes
- Web Audio has also been disabled on Firefox 24, after the pre-beta sign off.
Regression Tests
| Platform | Status (results) |
| Windows 7 32-bit | [DONE] PASS |
| Windows 8 64-bit | [DONE] PASS |
| Mac OSX 10.8 | [DONE] PASS |
| Ubuntu Linux 32-bit | [DONE] PASS |
Automated Tests
| Test | Result | Criteria |
| Automated Tests | PASS (details) | All Mozmill ondemand_functional testruns PASS with no new failures
|
Failures (known)
Update Tests
| Test | Result | Criteria |
| Update Tests (betatest) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (releasetest) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
| Update Tests (beta) |
PASS (details) | All Mozmill ondemand_update testruns PASS with no new failures
|
New Issues
| ID | Summary | Priority | Status |
|---|---|---|---|
| 902463 | This Connection is Untrusted message on facebook apps using clean profile | -- | RESOLVED |
| 902468 | Firefox gets stuck in Print Preview while trying to print a certain pdf file | P3 | RESOLVED |
| 902486 | Firefox doesn't start on Mac OS X using a remote profile from Windows | -- | RESOLVED |
3 Total; 0 Open (0%); 3 Resolved (100%); 0 Verified (0%);
Pre-beta Feature testing and Sign-off
- Platforms: Windows XP/7/8, Ubuntu 13.04, Mac 10.7/10.8
| Feature | Owner | Win32 | Mac | Linux |
| Web Audio | Manuela Muntean | PASS | PASS | PASS |
Notes:
- The following features have been disabled on Firefox 24:
- Prototype DOM Gamepad API (Alexandra Lucinet)
- Block cookies from sites I haven't visited (Bogdan Maris)
Aurora Merge
| Builds | nightlies |
| Changeset | tip (pushlog) |
- Checklist
- [DONE] Automated Testing
- [DONE] Update Testing on auroratest
- {{|}} Update Testing on aurora
Automated Testing
The following are the detailed results of the automated Mozmill test runs.
| Platform | |
| Windows XP | [DONE] - 2013-06-27 |
| Windows Vista | [DONE] - 2013-06-27 |
| Windows 7 32-bit | [DONE] - 2013-06-27 |
| Windows 7 64-bit | [DONE] - 2013-06-27 |
| Windows 8 32-bit | [DONE] - 2013-06-27 |
| Windows 8 64-bit | [DONE] - 2013-06-27 |
| Mac OS X 10.6 | [DONE] - 2013-06-27 |
| Mac OS X 10.7 | [DONE] - 2013-06-27 |
| Mac OS X 10.8 | [DONE] - 2013-06-27 |
| Ubuntu 12.04 32-bit | [DONE] - 2013-06-27 |
| Ubuntu 12.04 64-bit | [DONE] - 2013-06-27 |
- Test Failures (known bugs)
Update Testing
Spotcheck updates from a previous Aurora to Aurora 24.
| Platform | auroratest channel | aurora channel |
| Windows 7 64-bit | [DONE]
en-US 23.a2 20130527004019 - 20130627004018 |
|
| Mac OS X 10.7 | [DONE]
22.a2 en-US 20130420004013 - 20130627004018 |
|
| Ubuntu 12.04 32-bit | [DONE]
en-US 21.a2 20130317042012 - 20130627004018 |
- Issues
Important Features/Bugs
Bug Triage
- VERIFYME: verify the fix and do some exploratory testing around the issue
- FIXED: triage these bugs and test to ensure the fix is effective
- QAWANTED: triage these bugs and test to fulfill the QA request
- UNCONFIRMED: triage these bugs for potential new regressions
QAWANTED
Bugs requiring QA's assistance in debugging (steps, regression range, exploratory testing, verification, etc)
Nightly
Merge
- Functional Automation - result
- Update Spotchecks - result
