Releases/Firefox 37/Test Plan/Beta/1

«  Firefox 36 | Fennec 37 Beta 1 | Firefox 37 Beta 2  »

Firefox 37 Beta 1 (build 1) Test Plan

• Release Week: February 23th, 2015
• QA Contact: Robert Kaiser; Florin Mezei
• IRC: irc://irc.mozilla.org#qa ping kairo; FlorinMezei
• Build 1: Firefox 37 Beta 1 | changeset | pushlog

Features

See the Feature Release Tracking wiki for a general overview

Feature	Owner	Status	Test Plan
In-content preferences	Camelia Badau	Feature was disabled in 36 Beta, not sure if it will get in 37 beta	Test Plan
Flash Protected Mode	Andrei Vaida	Feature was disabled in 36 Beta, not sure if it will get in 37 beta

Risks with this Beta and ways to mitigate them through testing

Please list any risks related to the quality of this release.

• 1. shipping MSE for youtube - risky area (tried that in 36 beta as well), needs testing to ensure this is ready for shipping and users won't be affected by critical bugs
  • MSE will be enabled on Mac OSX on Beta 37 - if testing results with Beta 1 are showing serious issues, MSE should be disabled for Mac OSX
  • Testing MSE in this beta will be covered with exploratory testing with youtube
• 2. Disable Dev edition features in 37 beta 1 build
  • Testing will verify and validate that No DevEdition features are carried from Aurora
• 3. Possible disabling of In-content preferences feature in 37 beta (was also disabled in 36 beta)
  • Testing the feature to verify and validate that disabling the feature does not cause regressions.
• 4. Possible disabling of Flash protected mode feature in 37 beta (was also disabled in 36 beta)
  • Testing the feature to verify and validate that disabling the feature does not cause regressions.
• 5. Possible regressions introduced by bug fixes in Firefox 37
  • Testing will include bug verification for most important fixes as well as regression and exploratory testing on several platforms around thise fixes.

Testing Areas

• Manual
  • Regression Tests
  • Exploratory testing around recent changes/fixes
  • Bug verification

• Automated (configs for ondemand)
  • {{|}} Automation Functional tests
  • {{|}} Updates on beta-cdntest channel
  • {{|}} Updates on beta channel

Results

Bug Summary

• New bugs
• New bugs to this Beta
• Known Issues

5. Possible regressions introduced by bug fixes

Regression Tests

QA Task	Person/Group	How Often?
Regression Tests	Softvision	Every Beta

Regression tests will be run around areas changed from last beta build when the builds are ready. Tests are available on Mozilla Moztrap under the regression suite. In any case of failure in any regression tests, bugs need to be raised ASAP before proceeding to the exploratory testing.

Regression testing areas:

• Desktop Firefox Add-on Compatibility
• Breakpad End-to-End Functionality
• Desktop Firefox Plugin Compatibility
• Desktop Firefox Web Compatibility
• Stop accepting MD5 as a hash algorithm in signatures
• Mac OSX Signing Verification
• Verify update applies with root owned files
• Mac OSX 32-bit Mode
• NVidia 3D Vision Regression Test

Test Area	Operating Systen	Completed (when, tester name)	Results

Exploratory Testing

QA Task	Person/Group	How Often?
Exploratory testing	Softvision	Every Beta

Test Area	Operating System	Completed (when, tester name)	Results

Branch Bug Fix Verifications

QA Task	Person/Group	How Often?
Verification	Softvision	Every Beta

• Firefox (37) Bugs

• Verified bugs

Status tags

   [NOT STARTED] - [NOT STARTED] - little or no work has started on this task
   [ON TRACK] - [ON TRACK] - task is actively being worked on and those assigned feel comfortable and confident in the quality of the build.
   [DONE] - [DONE]  - task is completed and those assigned feel comfortable and confident in the quality of the build.
   [AT RISK] - [AT RISK] - blocking bug / high number of bugs / other confidence-shakers makes us feel not comfortable with the quality of the build.
   [IN PROGRESS] - [IN PROGRESS] - actively being worked on
   [INVESTIGATION NEEDED] - [INVESTIGATION NEEDED] - more investigation needed