Releases/Firefox 37/Test Plan/Beta/6

From MozillaWiki
Jump to: navigation, search

«  Firefox 37 Beta 5 | Firefox 37 Beta 6 | Firefox 37 Beta 7  »

Firefox 37 Beta 6 (build 1) Test Plan

QE recommendation regarding the quality of the build

Recommendation based on results from manual testing GOOD FOR RELEASE

Risks with this Beta and ways to mitigate them through testing

Please list any risks related to the quality of this release.

  • 1. Video/Audio
    • A few Video/Audio fixes landed in current version and expecting MSE to be disabled for OS X in 37 (Verified as Disabled)
    • Testing to ensure that YouTube playback works without major regressions, and MSE is disabled for OS X
    • Will be covered by Sanity testing on the YouTube default player
  • 2. Flash
    • Fix landed for bug: https://bugzil.la/1132432
    • Testing to ensure that flash content still plays without issues
    • Will be covered by Sanity testing
  • 3. Secure Browsing
    • Part of the Beta Regression run
    • Testing to ensure that the most common secure sites work without any major issues
    • Will be covered by Regression testing
  • 4. PDF
    • Part of the Beta Regression run
    • Testing to ensure that PDFs work without any major issues
    • Will be covered by Regression testing

Features

See the Feature Release Tracking wiki for a general overview


Feature Owner Status Details Test Plan
New Search Petruta Rasa [ON TRACK] Follow up on additional fixes for the New Search entered in Firefox 34 - NO final QA sign-off -
In-content preferences Camelia Badau DROPPED FROM 37 No more work required Test Plan
Flash Protected Mode disabling Andrei Vaida DROPPED FROM 37 No more work required -
EME Bogdan Maris DROPPED FROM 37 No more work required -
MSE for YoutTube Bogdan Maris [ON TRACK] Constantly monitoring for new issues and Verification of important fixes - NO final QA sign-off -

Results

Bug Summary

  • New bugs
Full Query
ID Summary Status Resolution
1144124 Select all in PDFs also selects text in the pdf.js UI NEW
1144148 [Find bar] Searched word is replaced with previous one after changing focus RESOLVED WORKSFORME
1144172 [Youtube][HTML5] Specific video stops at 4 seconds on some hardware RESOLVED DUPLICATE

3 Total; 1 Open (33.33%); 2 Resolved (66.67%); 0 Verified (0%);

  • New bugs to this Beta

NONE

  • Known Issues
Full Query
ID Summary Status Resolution
719114 Crash in [@ js::GCMarker::processMarkStackTop ] from heap corruption NEW
947079 Passive mixed content indicator (grey triangle) shows up on full HTTPS sites that have no mixed content RESOLVED FIXED
1020341 Switching several times between print preview orientations has a significant impact performance-wise NEW
1030622 Second search searches initial text with new hightlighting VERIFIED FIXED
1096357 "Unable to run script because scripts are blocked internally" JS error appear in browser console while browsing facebook and youtube on MacOS RESOLVED INCOMPLETE
1109085 Total number for find results constantly changes when going through matching strings in PDF file RESOLVED DUPLICATE
1127858 Bad framerate and high CPU when context menu is open, window out of focus or hovering some overlays (non-MSE HTML5 WebM) RESOLVED INCOMPLETE
1131638 Multiple AMD video cards have issues displaying 1080@60fps videos VERIFIED FIXED
1132433 YouTube settings menu is misplaced after playing several high definition videos from the recommended videos section RESOLVED INCOMPLETE
1132860 Error printing when trying to print a pdf file after exiting 'Presentation Mode' RESOLVED WORKSFORME
1143988 Make YouTube 360° videos work well RESOLVED WORKSFORME

11 Total; 2 Open (18.18%); 7 Resolved (63.64%); 2 Verified (18.18%);


Regression Tests

  • Tests are run from a Full Regression set, created before Beta 2, to cover areas most affected by fixes in current Beta (Firefox 37)
  • Regression tests are run over the interval Beta 2 -> Release
  • Tests to run for each Beta are picked to cover areas most affected by fixes since previous Beta (if there are no such areas, tests are picked randomly from the Full Beta Regression set).
  • Tests are available in Mozilla Moztrap.
  • In case of failure in any regression tests, bugs need to be raised ASAP before proceeding to the exploratory testing.

Regression tests:

  • Executed (in previous beta builds)
    • [PASS] Open and Print Common File Formats
    • [FAIL] Customize toolbars
    • [FAIL] Play WebGL games
    • [FAIL] Play WebGL animations
    • [PASS] Breakpad End-to-End Functionality
    • [FAIL] Desktop Firefox Web Compatibility
    • [FAIL] Play different video formats
    • [FAIL] Play various audio formats
    • [FAIL] Desktop Firefox Add-on Compatibility
    • [FAIL] Browsing secure sites
    • [FAIL] Show PDF
  • Remaining (for next beta builds)
    • [PASS/FAIL] Desktop Firefox Plugin Compatibility
    • [PASS/FAIL] Mac OSX Signing Verification
    • [PASS/FAIL] Verify update applies with root owned files
    • [PASS/FAIL] Mac OSX 32-bit Mode
    • [PASS/FAIL] Firefox scrolling & zooming
    • [PASS/FAIL] Session Restore smoketest

Exploratory Testing

  • Testing is performed around areas most affected by fixes since the previous Beta (in cases when there are no Regression tests to cover these areas, or those tests have already been run as part of the Full Beta Regression set)
Platform Assignee Result: report
Windows 7 32-bit (GPU: nVIDIA GeForce 620) Cornel Ionce [DONE]
Windows 8 64-bit (GPU: AMD Radeon HD 3000) Vasilica Mihasca [DONE]
Mac OS X 10.9.5 (GPU: Intel Iris Pro 1024 MB) Alexandra Lucinet [DONE]
Ubuntu 14.04 64-bit (GPU: AMD Radeon HD 6450) Bogdan Maris [DONE]
Tested Area Testing Type Results (report)
Video/Audio Sanity testing CONDITIONAL PASS
Flash Sanity testing PASS
Secure Browsing Regression testing PASS
PDF Regression testing PASS

Branch Bug Fix Verifications

  • Verified bugs
Full Query
ID Summary Status Resolution
1102039 panel left behind and delayed when minimizing VERIFIED FIXED
1104213 capture stack traces on timeline markers VERIFIED FIXED
1110336 Update sync old-dialog-based preferences UI to match FxA migration flows VERIFIED FIXED
1111967 Add an option to disable SSDP in Firefox VERIFIED FIXED
1131808 Firefox crash in mozilla::layers::YCbCrImageDataSerializer::InitializeBufferInfo VERIFIED FIXED
1132432 Flash not showing on some car site VERIFIED FIXED
1133356 animated gif stalls when page opened locally VERIFIED FIXED
1137459 FxA logs contain sensitive information VERIFIED FIXED
1140440 Mouse chooses options when search menu pops out under it VERIFIED FIXED
1141241 crash in mozilla::MFTDecoder::SendMFTMessage(_MFT_MESSAGE_TYPE, unsigned long) VERIFIED FIXED

10 Total; 0 Open (0%); 0 Resolved (0%); 10 Verified (100%);


Status tags

  • [NOT STARTED] - {{mnotstarted}} - little or no work has started on this task
  • [ON TRACK] - {{mok}} - task is actively being worked on and those assigned feel comfortable and confident in the quality of the build.
  • [DONE] - {{mdone}} - task is completed and those assigned feel comfortable and confident in the quality of the build.
  • [AT RISK] - {{mrisk}}- blocking bug / high number of bugs / other confidence-shakers makes us feel not comfortable with the quality of the build.
  • [IN PROGRESS] - {{mprog}} - actively being tested on
  • [INVESTIGATION NEEDED] - {{minvest}} - more investigation needed