Releases/Firefox 37/Test Plan/Beta/2: Difference between revisions

Jump to navigation Jump to search

Releases/Firefox 37/Test Plan/Beta/2 (view source)

Revision as of 13:56, 3 March 2015

331 bytes removed ,  3 March 2015
no edit summary
No edit summary
No edit summary
Line 5: Line 5:
* QA Contact: [mailto:kairo@mozilla.com Robert Kaiser]; [mailto:florin.mezei@softvisioninc.eu Florin Mezei]
* QA Contact: [mailto:kairo@mozilla.com Robert Kaiser]; [mailto:florin.mezei@softvisioninc.eu Florin Mezei]
* IRC: irc://irc.mozilla.org#qa ping kairo; FlorinMezei
* IRC: irc://irc.mozilla.org#qa ping kairo; FlorinMezei
* Build 1: [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/37.0b2-candidates/build1/ Firefox 37 Beta 2] | [https://hg.mozilla.org/releases/mozilla-beta/rev/192f6746dc45 changeset] |  
* Build 1: [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/37.0b2-candidates/build1/ Firefox 37 Beta 2] | [https://hg.mozilla.org/releases/mozilla-beta/rev/67a12d77e65d changeset] | (ID: 20150302192546)


== QE recommendation regarding the quality of the build ==
== QE recommendation regarding the quality of the build ==
Line 16: Line 16:
== Risks with this Beta and ways to mitigate them through testing ==
== Risks with this Beta and ways to mitigate them through testing ==
''Please list any risks related to the quality of this release.''
''Please list any risks related to the quality of this release.''
*'''1. [https://bugzilla.mozilla.org/show_bug.cgi?id=778617 MSE for YouTube] - risky area (significant number of bug fixes)'''
* '''1. YouTube (HTML5 player) - major focus in Beta 37'''
** Feature was also in Firefox 36 for part of the Beta cycle
** Several fixes since previous Beta (https://bugzil.la/1096089; https://bugzil.la/1132796; https://bugzil.la/1133633; https://bugzil.la/1134064; https://bugzil.la/1136576)
** Testing to ensure this is ready for shipping and users won't be affected by critical bugs
** Testing to ensure the new fixes did not cause any new regressions
** MSE will be [https://bugzilla.mozilla.org/show_bug.cgi?id=1134398 enabled] on Mac OS X on Beta 37 - if testing results with Beta 1 are showing serious issues, MSE should be disabled for Mac OSX
** Will be covered by exploratory testing on YouTube with the HTML5 player
** Will be covered by exploratory testing on YouTube
* '''2. Open H264'''
* '''2. Flash protected mode disabling feature in 37 beta (was also disabled in 36 beta)'''
** https://bugzil.la/1129369 has the potential of breaking OpenH264 (see comment #8)
** Testing the feature to verify and validate that disabling the feature does not cause regressions
** Testing the feature to verify that the plugin is still properly installed and works fine with WebRTC
** Will be covered by exploratory testing on websites with Flash content
** Will be covered by sanity testing
* '''3. [https://bugzilla.mozilla.org/show_bug.cgi?id=958874 Bing search with HTTPS]'''
* '''3. Loop'''
** Testing the feature to verify that Bing search now works through HTTPS, and search suggestions show up correctly
** https://bugzil.la/1137469 and https://bugzil.la/1137141 are important fixes that entered this Beta
** Will be covered by exploratory testing
** Testing the feature to verify that Loop works correctly and the two fixes above work as expected
* '''4. In-content Preferences feature (may be disabled mid-beta)'''
** Will be covered by sanity testing
** Feature was also in Firefox 36 for the first half of the Beta cycle
* '''4. Printing'''
** Testing the feature to verify that In-content Preferences show up without major issues
** Some issues with printing seen for Firefox 36 (e.g. https://bugzil.la/1136855)
** Will be covered by basic sanity testing (general UI and basic functionality)
** Testing the feature to verify that Printing works without major issues
* '''5. Disable Dev edition features in 37 beta 1 build'''  
** Will be covered by regression testing
** Testing will verify and validate that NO DevEdition features are carried from Aurora
** Will be covered by basic sanity testing
* '''6. EME (disabled and hidden in Firefox 37)'''
** Testing will check that the download of the [https://bugzilla.mozilla.org/show_bug.cgi?id=1089867 plugin] hasn't regressed the OpenH264 download for use on WebRTC/Hello
** Will be covered by exploratory testing


== Features ==
== Features ==
Line 97: Line 92:
</bugzilla>
</bugzilla>
*'''Known Issues'''
*'''Known Issues'''
none
<bugzilla>
{
"id":"905902,1092166,1124287",
"include_fields":"id,summary,status,resolution"
}
</bugzilla>


=== Regression Tests ===
=== Regression Tests ===


* Tests are run from a Full Regression set, created before Beta 2, to cover areas most affected by fixes in current Beta (Firefox 37)
* Tests are run from a Full Regression set, created before Beta 2, to cover areas most affected by fixes in current Beta (Firefox 37)
* Regression tests will be run over the interval Beta 2 -> Release  
* Regression tests are run over the interval Beta 2 -> Release  
* Tests to run for each Beta are picked to cover areas most affected by fixes since previous Beta (if there are no such areas, tests are picked randomly from the Full Beta Regression set).  
* Tests to run for each Beta are picked to cover areas most affected by fixes since previous Beta (if there are no such areas, tests are picked randomly from the Full Beta Regression set).  
* Tests will be available in [https://moztrap.mozilla.org/manage/cases/ Mozilla Moztrap].  
* Tests are available in [https://moztrap.mozilla.org/results/cases/?filter-run=6554 Mozilla Moztrap].  
* In any case of failure in any regression tests, bugs need to be raised ASAP before proceeding to the exploratory testing.
* In any case of failure in any regression tests, bugs need to be raised ASAP before proceeding to the exploratory testing.


Regression testing areas ''(final list to be finalized before Beta 2)'':
Regression tests:
* Desktop Firefox Add-on Compatibility
* Executed
* Breakpad End-to-End Functionality
** none
* Desktop Firefox Plugin Compatibility
* Remaining
* Desktop Firefox Web Compatibility
** Desktop Firefox Add-on Compatibility
* Mac OSX Signing Verification
** Breakpad End-to-End Functionality
* Verify update applies with root owned files
** Desktop Firefox Plugin Compatibility
* Mac OSX 32-bit Mode
** Desktop Firefox Web Compatibility
* NVidia 3D Vision Regression Test
** Mac OSX Signing Verification
** Verify update applies with root owned files
** Mac OSX 32-bit Mode
** Play WebGL games
** Play WebGL animations
** Firefox scrolling & zooming
** Show PDF
** Open and Print Common File Formats
** Browsing secure sites
** Session Restore smoketest
** Customize toolbars
** Play different video formats
** Play various audio formats


=== Exploratory Testing ===
=== Exploratory Testing ===
Line 125: Line 137:
  | class="header" style="text-align:center;" | Platform
  | class="header" style="text-align:center;" | Platform
  | class="header" style="text-align:center;" | Assignee
  | class="header" style="text-align:center;" | Assignee
  | class="header" style="text-align:center;" | Result: [https://etherpad.mozilla.org/Fx37b1 report]
  | class="header" style="text-align:center;" | Result: [https://etherpad.mozilla.org/Fx37b2 report]
  |-
  |-
  |  
  | Windows 7 32-bit
  |  
  | Camelia Badau
  | style="text-align:center;" | {{|}}
  | style="text-align:center;" | {{ok|}}
  |-
  |-
  |  
  | Windows 8 64-bit
  |  
  | Catalin Varga
  | style="text-align:center;" | {{|}}
  | style="text-align:center;" | {{ok|}}
  |-
  |-
  |  
  | Mac OS X 10.8.5
  |
  | Cornel Ionce
  | style="text-align:center;" | {{|}}
  | style="text-align:center;" | {{ok|}}
  |-
  |-
  |  
  | Ubuntu 14.04 32-bit
  |  
  | Petruta Rasa
  | style="text-align:center;" | {{|}}
  | style="text-align:center;" | {{ok|}}
  |}
  |}


Line 148: Line 160:
  | class="header" style="width:20%; text-align:center;" | Tested Area
  | class="header" style="width:20%; text-align:center;" | Tested Area
  | class="header" style="width:20%; text-align:center;" | Testing Type
  | class="header" style="width:20%; text-align:center;" | Testing Type
  | class="header" style="width:20%; text-align:center;" | Results ([https://etherpad.mozilla.org/Fx37b1 report])
  | class="header" style="width:20%; text-align:center;" | Results ([https://etherpad.mozilla.org/Fx37b2 report])
|-
|
| Exploratory testing
| style="background:#9D9; text-align:center;" | RESULT
|-
|
| Exploratory testing
| style="background:#9D9; text-align:center;" | RESULT
  |-
  |-
  |  
  | YouTube (HTML5 player)
  | Exploratory testing
  | Exploratory testing
  | style="background:#9D9; text-align:center;" | RESULT
  | style="background:##FFFF00; text-align:center;" | IN PROGRESS
  |-
  |-
  |
  | Open H264
  | Basic Sanity testing
  | Sanity testing
  | style="background:#9D9; text-align:center;" | RESULT
  | style="background:##FFFF00; text-align:center;" | IN PROGRESS
  |-
  |-
  |
  | Loop
  | Basic Sanity testing
  | Sanity testing
  | style="background:#9D9; text-align:center;" | RESULT
  | style="background:##FFFF00; text-align:center;" | IN PROGRESS
  |-
  |-
  |  
  | Printing
  |  
  | Regression testing
  | style="background:#9D9; text-align:center;" | RESULT
  | style="background:##FFFF00; text-align:center;" | IN PROGRESS
  |}
  |}


Florin.Mezei
Confirmed users
2,482

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1059380"

Navigation menu