Releases/Firefox 38/Test Plan/38.0.5/Beta/1

From MozillaWiki
Jump to: navigation, search

«  Firefox 38 RC | Firefox 38.0.5 Beta 1 | Firefox 38.0.5 Beta 2  »

Firefox 38.0.5 Beta 1 (build 1) Test Plan

QE recommendation regarding the quality of the build

Recommendation based on results from manual testing CONDITIONAL SIGN OFF (many issues)

Risks with this Beta and ways to mitigate them through testing

Please list any risks related to the quality of this release.

  • 1. Reading List disabling
    • The Reading List feature is no longer a part of the 38.0.5 release
    • Testing to make sure that the feature is not enabled
    • Will be covered by Sanity testing
  • 2. Suggested Tiles disabling
    • The Suggested Tiles feature was backed out into Firefox 39
    • Testing to make sure that the feature is not enabled
    • Will be covered by Sanity testing
  • 3. Reader View
    • Important feature for the 38.0.5 Release
    • Testing to ensure that the Reader View feature is stable enough to be tested
    • Will be covered by Exploratory testing
  • 4. Pocket
    • Important feature for the 38.0.5 Release
    • Testing to ensure that the Pocket integration feature is stable enough to be tested
    • Will be covered by Exploratory testing
  • 5. Screensharing
    • Important feature for the 38.0.5 Release
    • Testing to ensure that the Hello Screensharing feature has no major issues
    • Will be covered by Exploratory testing

Features

See the Feature Release Tracking gdoc for a general overview


Feature Owner Status Details Test Plan
Reader View Andrei Vaida [AT RISK] New feature in Firefox 38.0.5 Test Plan
Pocket Integration Andrei Vaida [AT RISK] New feature in Firefox 38.0.5 Test Plan
Hello screensharing Bogdan Maris [AT RISK] New feature in Firefox 38.0.5 Test Plan

Results

New bugs

Full Query
ID Summary Status Resolution
1163519 ReaderParent.jsm:95:12 throws an error in the Browser Console when clicking the "Enter Reader View" button from the Location Bar VERIFIED FIXED
1163523 Clicking the body of a page displayed in Reader View places a focus ring on it RESOLVED WORKSFORME
1163526 Reader View promo panel is automatically dismissed RESOLVED WORKSFORME
1163534 AboutReader.jsm:248:0 throws an error in the console when closing an about:reader tab RESOLVED WORKSFORME
1163576 Unable to remove pages from the Pocket list VERIFIED FIXED
1163577 Reader View button shouldn't be shown on Pocket website RESOLVED WORKSFORME
1163581 Unexpected error message displayed when navigating from Pocket list RESOLVED INVALID
1163589 Some pages don`t render correctly after exiting Reader View RESOLVED DUPLICATE
1163613 The panel shown when attempting to Save to Pocket in offline mode is dismissed too soon RESOLVED WORKSFORME
1163651 [Windows]View Pocket List icon from Bookmarks menu is missing. VERIFIED FIXED
1163655 View Pocket List menuitem is not at top if Bookmarks button is moved to menu panel. VERIFIED FIXED
1163917 The built-in Pocket is NOT completely disabled if Pocket is already installed and used as an add-on VERIFIED FIXED
1163999 Screensharing also stops when user selects 'Stop Sharing' from the camera/microphone permissions drop-down RESOLVED WORKSFORME
1164000 Webcam windows are misplaced when joining a screenshared conversation with No Video selected RESOLVED WORKSFORME
1164001 Webcam windows are misplaced when joining a screenshared conversation with muted video RESOLVED WORKSFORME

15 Total; 0 Open (0%); 10 Resolved (66.67%); 5 Verified (33.33%);

New bugs to this Beta

Full Query
ID Summary Status Resolution
1163519 ReaderParent.jsm:95:12 throws an error in the Browser Console when clicking the "Enter Reader View" button from the Location Bar VERIFIED FIXED
1163523 Clicking the body of a page displayed in Reader View places a focus ring on it RESOLVED WORKSFORME
1163526 Reader View promo panel is automatically dismissed RESOLVED WORKSFORME
1163534 AboutReader.jsm:248:0 throws an error in the console when closing an about:reader tab RESOLVED WORKSFORME
1163576 Unable to remove pages from the Pocket list VERIFIED FIXED
1163577 Reader View button shouldn't be shown on Pocket website RESOLVED WORKSFORME
1163581 Unexpected error message displayed when navigating from Pocket list RESOLVED INVALID
1163589 Some pages don`t render correctly after exiting Reader View RESOLVED DUPLICATE
1163613 The panel shown when attempting to Save to Pocket in offline mode is dismissed too soon RESOLVED WORKSFORME
1163651 [Windows]View Pocket List icon from Bookmarks menu is missing. VERIFIED FIXED
1163655 View Pocket List menuitem is not at top if Bookmarks button is moved to menu panel. VERIFIED FIXED
1163917 The built-in Pocket is NOT completely disabled if Pocket is already installed and used as an add-on VERIFIED FIXED
1163999 Screensharing also stops when user selects 'Stop Sharing' from the camera/microphone permissions drop-down RESOLVED WORKSFORME
1164000 Webcam windows are misplaced when joining a screenshared conversation with No Video selected RESOLVED WORKSFORME
1164001 Webcam windows are misplaced when joining a screenshared conversation with muted video RESOLVED WORKSFORME

15 Total; 0 Open (0%); 10 Resolved (66.67%); 5 Verified (33.33%);

Known Issues

  • Major issues
Full Query
ID Summary Status Resolution
1133623 crash in mozilla::layers::CompositorD3D11::BeginFrame RESOLVED FIXED
1137602 Windows only - Global indicator does not call any pop-ups from Firefox RESOLVED INCOMPLETE
1145106 crash in atidxx32.dll@0x196f7 RESOLVED WORKSFORME
1160577 Error message is no longer shown when a page isn't readable VERIFIED FIXED
1163931 "View Pocket List" prevents going back to previous page due to server redirects NEW

5 Total; 1 Open (20%); 3 Resolved (60%); 1 Verified (20%);

  • Low impact issues
Full Query
ID Summary Status Resolution
1057757 Large OOM in mozilla::layers::BufferRecycleBin::GetBuffer (during video decoding) RESOLVED WONTFIX
1136265 Reading mode desktop images shouldn't be upscaled beyond their _originalWidth, even if they take more than half of the screen. RESOLVED WORKSFORME
1137598 Reversed controls for videos that have a css transform flipping the video (no attribute/option to flip video output) RESOLVED INACTIVE
1137606 Background image from screen sharing while the webcam window of the person who is receiving the call is in fullscreen RESOLVED WORKSFORME
1145106 crash in atidxx32.dll@0x196f7 RESOLVED WORKSFORME
1146882 Starting tab sharing before the call will not show the tab when someone joins conversation RESOLVED WONTFIX
1151415 [Linux] Focus rings are displayed in Reader View for the buttons available in the font panel RESOLVED INVALID
1154271 crash in nvwgf2um.dll@0x7fec13 RESOLVED WORKSFORME
1160401 Tag text is selectable in Pocket door hanger NEW
1160407 Pocket's "Open Pocket" and "Sign In"/"Create Account" link won't work well with Private Browsing RESOLVED FIXED
1161018 The Reader View promo panel can no longer be dismissed after opening a new window RESOLVED WORKSFORME
1161022 The UI elements from the Reader View promo panel show alignment and position related discrepancies RESOLVED WORKSFORME
1161023 Unusable and very small conversation window notification after hello window is detached RESOLVED INCOMPLETE
1161057 Type controls panel vanishes when zooming out at the bottom of an article RESOLVED FIXED
1161518 The user is able to drag and drop the links from the Pocket panel RESOLVED WORKSFORME
1162014 Long tags cut off after trying to save them, overlayed/hidden by white area NEW
1162750 Reader View tour tool tip cannot be dismissed by clicking outside the door hanger RESOLVED WONTFIX
1163498 CMD+0 keyboard shortcut does not reset Reader View's text size RESOLVED INCOMPLETE

18 Total; 2 Open (11.11%); 16 Resolved (88.89%); 0 Verified (0%);


Regression Tests

  • There will be NO Regression Set for 38.0.5 BETA
  • Regression testing will be performed as needed for the new features in Firefox 38.0.5

Exploratory Testing

  • Testing is performed around areas most affected by fixes since the previous Beta (in cases when there are no Regression tests to cover these areas, or those tests have already been run as part of the Full Beta Regression set)
Platform Assignee Result: report
Windows 7 64-bit (GPU: AMD Radeon HD 6450) Bogdan Maris [DONE]
Windows 8.1 32-bit (GPU: nVIDIA GeForce 620) Cornel Ionce [DONE]
Mac OS X 10.9.5 (GPU: Intel Iris Pro 1024 MB) Alexandra Lucinet [DONE]
Ubuntu 14.04 64-bit (GPU: AMD Radeon 3000) Andrei Vaida [DONE]
Tested Area Testing Type Results (report)
Reading List disabling Sanity testing PASS
Suggested Tiles disabling Sanity testing PASS
Reader View Exploratory testing CONDITIONAL PASS
Pocket Exploratory testing CONDITIONAL PASS
Screensharing Exploratory testing CONDITIONAL PASS

Branch Bug Fix Verifications

  • Verified bugs
Full Query
ID Summary Status Resolution
1157197 Please enable reader view for 38.0.5 VERIFIED FIXED
1157682 Should ignore query string / hashes for checking for "home page" VERIFIED FIXED
1158281 Match Pocket's Reader View Sepia Theme VERIFIED FIXED
1158322 Sidebar affects a page while printing VERIFIED FIXED
1160076 Broken in-content preferences Search tab with browser.search.showOneOffButtons = false VERIFIED FIXED
1160577 Error message is no longer shown when a page isn't readable VERIFIED FIXED
1160578 Odd empty door hanger panel when you're on an internal Firefox page VERIFIED FIXED
1161504 The "Save" button from the Pocket panel appears to have a visited styling applied VERIFIED FIXED
1161881 Enable Pocket by default (in supported locales) VERIFIED FIXED
1162147 "View Pocket List" menuitem should be at top of bookmarks menu VERIFIED FIXED
1163319 Pocket button in hamburger menu breaks layout VERIFIED FIXED
1163349 "View Pocket List" menuitem not working VERIFIED FIXED

12 Total; 0 Open (0%); 0 Resolved (0%); 12 Verified (100%);


Status tags

  • [NOT STARTED] - {{mnotstarted}} - little or no work has started on this task
  • [ON TRACK] - {{mok}} - task is actively being worked on and those assigned feel comfortable and confident in the quality of the build.
  • [DONE] - {{mdone}} - task is completed and those assigned feel comfortable and confident in the quality of the build.
  • [AT RISK] - {{mrisk}}- blocking bug / high number of bugs / other confidence-shakers makes us feel not comfortable with the quality of the build.
  • [IN PROGRESS] - {{mprog}} - actively being tested on
  • [INVESTIGATION NEEDED] - {{minvest}} - more investigation needed