MozillaWiki

Releases/Firefox 39/Test Plan: Difference between revisions

Page Discussion

Releases/Firefox 39/Test Plan (view source)

Revision as of 15:26, 7 August 2015

13,213 bytes added ,  7 August 2015
→‎Testing
No edit summary
 
(44 intermediate revisions by 5 users not shown)
Line 217: Line 217:


==== Testing ====
==== Testing ====
* Manual - '''[[Releases/Firefox 39/Test Plan/Beta/2 | Firefox 39 Beta 2 build 1]]'''
* Manual - '''[[Releases/Firefox 39/Test Plan/Beta/2 | Firefox 39 Beta 2 build 1 Test Plan]]'''
** {{|}} Areas tested - TBA
** {{mdone|}} Areas tested - Smoke and Exploratory testing were performed on Video Compatibility, Graphics, EME, Scroll & Zoom, followed by a spot-check covering websites with Flash content and PDF navigation. [https://goo.gl/p2S3Qm See detailed test results]
** {{|}} [https://goo.gl/hAoDke Bug verification]
** {{mdone|}} Bug verification


* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx39b2|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand], build2)  
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx39b2|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand])  
** {{|}} Updates on ''beta-cdntest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=beta-cdntest&from=2015-05-24&to=2015-05-25&target=39.0 report])
** {{done|}} Updates on ''beta-cdntest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=beta-cdntest&from=2015-05-24&to=2015-05-25&target=39.0 report])
** {{|}} Updates on ''beta'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=beta&from=2015-05-24&to=2015-05-25&target=39.0 report])
** {{done|}} Updates on ''beta'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=beta&from=2015-05-24&to=2015-05-25&target=39.0 report])


=== Beta 3 ===
==== Build Information ====
* [https://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/39.0b3-candidates/build1/ build1]: [https://hg.mozilla.org/releases/mozilla-beta/rev/d51242ae365d changeset] (build ID: 20150604162752)
==== Testing ====
* Manual - '''[[Releases/Firefox 39/Test Plan/Beta/3 | Firefox 39 Beta 3 build 1 Test Plan]]'''
** {{done|}} [https://goo.gl/w2HcxB Regression testing] with focus on: Plug-ins, Search
** {{done|}} [https://goo.gl/KGvYHp Bug verification]
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx39b3|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand])
** {{done|}} Updates on ''beta-cdntest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=beta-cdntest&from=2015-06-05&to=2015-06-05&target=39.0 report])
** {{done|}} Updates on ''beta'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=beta&from=2015-06-05&to=2015-06-05&target=39.0 report])
=== Beta 4 ===
==== Build Information ====
* [https://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/39.0b4-candidates/build2/ build2]: [https://hg.mozilla.org/releases/mozilla-beta/rev/780e1c2b8013 changeset] (build ID: 20150609130336)
==== Testing ====
* Manual - '''[[Releases/Firefox 39/Test Plan/Beta/4 | Firefox 39 Beta 4 build 2 Test Plan]]'''
** {{done|}} [https://goo.gl/OmLtmw Regression testing] with focus on: Add-ons, Sync
** {{done|}} [https://goo.gl/KGvYHp Bug verification]
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx39b4|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand])
** {{done|}} Updates on ''beta-cdntest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=beta-cdntest&from=2015-06-10&to=2015-06-10&target=39.0 report])
** {{done|}} Updates on ''beta'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=beta&from=2015-06-10&to=2015-06-10&target=39.0 report])
=== Beta 5 ===
==== Build Information ====
* [https://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/39.0b5-candidates/build1/ build1]: [https://hg.mozilla.org/releases/mozilla-beta/rev/b70403ba2e8f changeset] (build ID: 20150611135623)
==== Testing ====
* Manual - '''[[Releases/Firefox 39/Test Plan/Beta/5 | Firefox 39 Beta 5 build 1 Test Plan]]'''
** {{done|}} [https://goo.gl/SWozYK Regression testing] with focus on: Breakpad, Update with root owned files, Video compatibility
** {{done|}} [https://goo.gl/KGvYHp Bug verification]
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx39b5|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand])
** {{done|}} Updates on ''beta-cdntest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=beta-cdntest&from=2015-06-12&to=2015-06-12&target=39.0 report])
** {{done|}} Updates on ''beta'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=beta&from=2015-06-12&to=2015-06-12&target=39.0 report])
=== Beta 6 ===
==== Build Information ====
* [https://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/39.0b6-candidates/build1/ build1]: [https://hg.mozilla.org/releases/mozilla-beta/rev/05b522f50491 changeset] (build ID: 20150615125213)
==== Testing ====
* Manual - '''[[Releases/Firefox 39/Test Plan/Beta/6 | Firefox 39 Beta 6 build 1 Test Plan]]'''
** {{done|}} [https://goo.gl/pLHhq7 Regression testing] with focus on: Secure Browsing, Hello, Mac OS X Signing, Audio Compatibility, Mac OS X 32-bit Mode
** {{done|}} [https://goo.gl/KGvYHp Bug verification]
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx39b6|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand])
** {{done|}} Updates on ''beta-cdntest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=beta-cdntest&from=2015-06-16&to=2015-06-16&target=39.0 report])
** {{done|}}} Updates on ''beta'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=beta&from=2015-06-16&to=2015-06-16&target=39.0 report])
=== Beta 7 ===
==== Build Information ====
* [https://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/39.0b7-candidates/build1/ build1]: [https://hg.mozilla.org/releases/mozilla-beta/rev/16b72b74efde changeset] (build ID: 20150618135210)
==== Testing ====
* Manual - '''[[Releases/Firefox 39/Test Plan/Beta/7 | Firefox 39 Beta 7 build 1 Test Plan]]'''
** {{done|}} [https://goo.gl/wjc5xQ Regression testing] with focus on: Plug-in Compatibility, Video Compatibility
** {{done|}} [https://goo.gl/KGvYHp Bug verification]
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx39b7|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand])
** {{|}} Updates on ''beta-cdntest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=beta-cdntest&from=2015-06-19&to=2015-06-19&target=39.0 report])
** {{|}}} Updates on ''beta'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=beta&from=2015-06-19&to=2015-06-19&target=39.0 report])
== Release Candidates ==
=== 39.0 build3 ===
==== Build Information ====
* [http://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/39.0-candidates/build3/ build3]: [https://hg.mozilla.org/releases/mozilla-release/rev/3614e50ca05c changeset] (build ID: 20150622181234)
==== Testing ====
* Manual - '''[[Releases/Firefox 39/Test Plan/RC/1 | Firefox 39.0 RC build 3 Test Plan]]'''
** {{done|}} [https://goo.gl/nxLTfG Regression Testing] with focus on: Web compatibility, Secure browsing
** {{done|}} [https://goo.gl/KGvYHp Bug verification]
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx39RC|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand])
** This is the 39.0 release build. We should test updates on beta-cdntest for a preliminary sign-off (Release Management must give the "go" for pushing to beta), then on beta after the push to beta, and then on the day before release (Monday), after a full sign-off and push to that channel, run tests on release-cdntest.
** {{done|}} Updates on ''beta-cdntest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=beta-cdntest&from=2015-06-24&to=2015-06-24&target=38.0 report])
=== 39.0 build4 ===
==== Build Information ====
* [https://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/39.0-candidates/build4/ build4]: [https://hg.mozilla.org/releases/mozilla-release/rev/f075b15b50d5 changeset] (build ID: 20150624153222)
==== Testing ====
* Manual - '''[[Releases/Firefox 39/Test Plan/RC/2 | Firefox 39.0 RC build 4 Test Plan]]'''
** {{done|}} [https://goo.gl/Hq8ai6 Smoke Testing] focused on Web compatibility
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx39RCbuild4|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand])
** This is the 39.0 release build. We should test updates on beta-cdntest for a preliminary sign-off (Release Management must give the "go" for pushing to beta), then on beta after the push to beta, and then on the day before release (Monday), after a full sign-off and push to that channel, run tests on release-cdntest.
** {{done|}} Updates on ''beta-cdntest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=beta-cdntest&from=2015-06-25&to=2015-06-25&target=38.0 report])
** {{done|}} Updates on ''beta'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=beta&from=2015-06-25&to=2015-06-25&target=38.0 report])
=== 39.0 build5 ===
==== Build Information ====
* [https://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/39.0-candidates/build5/ build5]: [https://hg.mozilla.org/releases/mozilla-release/rev/78c25ce5f523 changeset] (build ID: 20150626112833)
==== Testing ====
* Manual
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx39RCbuild5|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand])
** This is the 39.0 release build. We should test updates on beta-cdntest for a preliminary sign-off (Release Management must give the "go" for pushing to beta), then on beta after the push to beta, and then on the day before release (Monday), after a full sign-off and push to that channel, run tests on release-cdntest.
** {{done|}} Updates on ''beta-cdntest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=beta-cdntest&from=2015-06-25&to=2015-06-25&target=38.0 report])
** {{done|}} Updates on ''beta'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=beta&from=2015-06-25&to=2015-06-25&target=38.0 report])
=== 39.0 build6 ===
==== Build Information ====
* [https://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/39.0-candidates/build6/ build6]: [https://hg.mozilla.org/releases/mozilla-release/rev/d3b3e57e8088 changeset] (build ID: 20150630154324)
==== Testing ====
* Manual - '''[[Releases/Firefox 39/Test Plan/RC/6 | Firefox 39.0 RC build 6 Test Plan]]'''
** {{done|}} [https://etherpad.mozilla.org/Fx39RC6 Smoke Testing] focused on: Web compatibility (with AntiVirus add-ons), Add-on compatibility
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx39RCbuild6|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand])
** This is the 39.0 release build. We should test updates on beta-cdntest for a preliminary sign-off (Release Management must give the "go" for pushing to beta), then on beta after the push to beta, and then on the day before release (Monday), after a full sign-off and push to that channel, run tests on release-cdntest.
** {{done|}} Updates on ''beta-cdntest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=beta-cdntest&from=2015-07-01&to=2015-07-01&target=38.0 report])
** {{done|}} Updates on ''beta'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=beta&from=2015-07-01&to=2015-07-01&target=38.0 report])
** {{done|}} Updates on ''release-cdntest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=release-cdntest&from=2015-07-02&to=2015-07-02&target=38.0 report])
** {{done|}} Updates on ''release'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=release&from=2015-07-02&to=2015-07-02&target=38.0 report])
=== Release 39.0 ===
==== Build Information ====
* See Release Candidate Build 6
==== Testing ====
* Update Testing - see Release Candidate Build 6
* Throttle Testing (see [[QA/Desktop_Firefox/Walkthroughs/Release_Coordination#Update_Throttling|Walkthrough]] for details)
** Used commands:
./throttle.sh https://aus3.mozilla.org/update/3/Firefox/38.0.5/20150525141253/WINNT_x86-msvc/en-US/release/Windows_NT%206.3/default/default/update.xml
./throttle.sh https://aus3.mozilla.org/update/3/Firefox/38.0.5/20150525141253/Darwin_x86_64-gcc3-u-i386-x86_64/en-US/release/Darwin%2013.4.0/default/default/update.xml
./throttle.sh https://aus3.mozilla.org/update/3/Firefox/38.0.5/20150525141253/Linux_x86-gcc3/en-US/release/Linux%203.13.0-45-generic%20%28GTK%202.24.23%29/default/default/update.xml
./throttle.sh https://aus3.mozilla.org/update/3/Firefox/38.0.5/20150525141253/Linux_x86_64-gcc3/en-US/release/Linux%203.16.0-30-generic%20%28GTK%202.24.25%29/default/default/update.xml
** Results
*** After release push (expected to be @ 25%), 2015-07-02: Win32: 25% - Mac: 25% - Linux32: 23% - Linux64: 24%
*** After disabling updates (expected to be @ 0%), omitted
*** After fully enabling updates (expected to be @ 100%), 2015-07-08: Win32: 100% - Mac: 99% - Linux32: 100% - Linux64: 100%
=== 39.0.3 build2 ===
==== Build Information ====
* [https://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/39.0.3-candidates/build2/ build2]: [https://hg.mozilla.org/releases/mozilla-release/rev/GECKO390_2015063018_RELBRANCH changeset] (build ID: 20150806001005)
==== Testing ====
* Manual - '''[[Releases/Firefox 39/Test Plan/39.0.3 | Firefox 39.0.3 build 2 Test Plan]]'''
** {{done|}} [https://docs.google.com/spreadsheets/d/1TjZTco65z79UmaQfARPYlrmlCb_QykEaMy2F3-M_gkA/edit#gid=90573885 Smoke Testing] focused on: PDF Compatibility, Add-on compatibility, Browser Customization (Themes)
** {{done|}} [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=1178058%2C1179262&list_id=12450895 Bug fixes verification]
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx39.0.3|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand])
** {{done|}} Updates on ''release-cdntest'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=release-cdntest&from=2015-08-06&to=2015-08-06&target=38.0 report])
** {{done|}} Updates on ''release'' channel ([http://mozmill-release.blargon7.com/#/update/detail?&channel=release&from=2015-08-06&to=2015-08-06&target=38.0 report])


[[category:Test_Plans|Firefox]][[category:QA|Firefox Test Plans]]
[[category:Test_Plans|Firefox]][[category:QA|Firefox Test Plans]]
Otilia.anica
Confirmed users
2,280

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1077928...1088883"