MozillaWiki

Releases/Firefox 54/Test Plan: Difference between revisions

Page Discussion

Releases/Firefox 54/Test Plan (view source)

Revision as of 07:35, 30 June 2017

5,437 bytes added ,  30 June 2017
m
→‎54.0.1
 
(13 intermediate revisions by 2 users not shown)
Line 412: Line 412:
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx54RC|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand])  
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx54RC|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand])  
** {{mdone|}}  Updates on ''beta-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=e832ed037a3c&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta-cdntest( report])
** {{mdone|}}  Updates on ''beta-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=e832ed037a3c&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta-cdntest( report])
** {{mnotstarted|}} Updates on ''beta'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=e832ed037a3c&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta( report])
** {{mdone|}} Updates on ''beta'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=e832ed037a3c&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta( report])
** {{mnotstarted|}} Updates on ''release-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=e832ed037a3c&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release-cdntest( report])
** {{mdone|}} Updates on ''release-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=e832ed037a3c&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release-cdntest( report])
** {{mnotstarted|}} Updates on ''release'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=e832ed037a3c&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release( report])
** {{mdone|}} Updates on ''release'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=e832ed037a3c&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release( report])
** Throttle Testing (see [[QA/Desktop_Firefox/Walkthroughs/Release_Coordination#Update_Throttling|Walkthrough]] for details)
*** Used commands:
*** ./throttle.sh "https://aus3.mozilla.org/update/3/Firefox/53.0.3/20170518000419/WINNT_x86-msvc/en-US/release/Windows_NT%206.3(nowebsense)/default/default/update.xml"
*** ./throttle.sh "https://aus3.mozilla.org/update/3/Firefox/53.0.3/20170518000419/WINNT_x86_64-msvc/en-US/release/Windows_NT%2010.0(nowebsense)/default/default/update.xml"
*** ./throttle.sh https://aus3.mozilla.org/update/3/Firefox/53.0.3/20170518000419/Darwin_x86_64-gcc3-u-i386-x86_64/en-US/release/Darwin%2013.4.0/default/default/update.xml
*** ./throttle.sh https://aus3.mozilla.org/update/3/Firefox/53.0.3/20170518000419/Linux_x86-gcc3/en-US/release/Linux%203.13.0-45-generic%20%28GTK%202.24.23%29/default/default/update.xml
*** ./throttle.sh https://aus3.mozilla.org/update/3/Firefox/53.0.3/20170518000419/Linux_x86_64-gcc3/en-US/release/Linux%203.16.0-30-generic%20%28GTK%202.24.25%29/default/default/update.xml
*** ./throttle.sh "https://aus3.mozilla.org/update/3/Firefox/53.0.3/20170518000419/WINNT_x86-msvc/en-US/release/Windows_NT%206.3/default/default/update.xml"
*** ./throttle.sh "https://aus3.mozilla.org/update/3/Firefox/53.0.3/20170518000419/WINNT_x86_64-msvc/en-US/release/Windows_NT%2010.0/default/default/update.xml"
*** Results:
**** After release push (expected to be @ 25%): 2017-06-13:  '''Win32: 24% - Win64: 25% - Mac: 28% - Linux32: 100% - Linux64: 100%'''
***** Linux shows at 100% because we serve 53.0.3 whenever we don't offer 54.0
**** After disabling updates (expected to be @ 0%): 2017-06-14: '''Win32: 0% - Win64: 0% - Mac: 0% - Linux32: 0% - Linux64: 0%'''
**** After fully enabling updates (expected to be @ 100%): 2017-06-21: '''Win32: 100% - Win64: 100% - Mac: 100% - Linux32: 100% - Linux64: 100%'''
 
=== 54.0.1 ===
==== Build Information ====
* Build 1: [https://archive.mozilla.org/pub/firefox/candidates/54.0.1-candidates/build1/ 54.0.1-build1] | [https://hg.mozilla.org/releases/mozilla-release/rev/90f18f9c15f7 90f18f9c15f7] | (ID: 20170628075643)
 
==== Testing ====
* Manual - [https://wiki.mozilla.org/Releases/Firefox_54/Test_Plan/Beta/54.0.1 Firefox 54.0.1]
** {{mdone|}} Exploratory Testing with focus on areas affected by fixes: Printing, Downloads, Video Compatibility, and New Tab Page.
*** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/reports/view/4852 in TestRail] (requires access rights) or [https://drive.google.com/open?id=0B5sd-Y_uHNdrXzhQNnd2NmJMSkk as a pdf].
** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit?pli=1#gid=960315414 Bug verification] with focus on driver fixes: [https://bugzil.la/1366744 1366744], [https://bugzil.la/1373109 1373109], [https://bugzil.la/1375708 1375708], [https://bugzil.la/1371995 1371995], and [https://bugzil.la/1371962 1371962]
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx54.0.1|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand])
** {{mdone|}} Updates on ''release-localtest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=90f18f9c15f7&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release-localtest( report])
** {{mdone|}} Updates on ''release-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=90f18f9c15f7&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release-cdntest( report])
** {{mdone|}} Updates on ''release'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=90f18f9c15f7&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release( report])
** Throttle Testing (see [[QA/Desktop_Firefox/Walkthroughs/Release_Coordination#Update_Throttling|Walkthrough]] for details)
*** Used commands:
*** ./throttle.sh "https://aus3.mozilla.org/update/3/Firefox/54.0/20170608105825/WINNT_x86-msvc/en-US/release/Windows_NT%206.3(nowebsense)/default/default/update.xml"
*** ./throttle.sh "https://aus3.mozilla.org/update/3/Firefox/54.0/20170608105825/WINNT_x86_64-msvc/en-US/release/Windows_NT%2010.0(nowebsense)/default/default/update.xml"
*** ./throttle.sh https://aus3.mozilla.org/update/3/Firefox/54.0/20170608105825/Darwin_x86_64-gcc3-u-i386-x86_64/en-US/release/Darwin%2013.4.0/default/default/update.xml
*** ./throttle.sh https://aus3.mozilla.org/update/3/Firefox/54.0/20170608105825/Linux_x86-gcc3/en-US/release/Linux%203.13.0-45-generic%20%28GTK%202.24.23%29/default/default/update.xml
*** ./throttle.sh https://aus3.mozilla.org/update/3/Firefox/54.0/20170608105825/Linux_x86_64-gcc3/en-US/release/Linux%203.16.0-30-generic%20%28GTK%202.24.25%29/default/default/update.xml
*** ./throttle.sh "https://aus3.mozilla.org/update/3/Firefox/54.0/20170608105825/WINNT_x86-msvc/en-US/release/Windows_NT%206.3/default/default/update.xml"
*** ./throttle.sh "https://aus3.mozilla.org/update/3/Firefox/54.0/20170608105825/WINNT_x86_64-msvc/en-US/release/Windows_NT%2010.0/default/default/update.xml"
*** Results:
**** After release push (expected to be @ 25%): 2017-06-29:  '''N/A'''
**** After disabling updates (expected to be @ 0%): 2017-06-29: '''N/A'''
**** After fully enabling updates (expected to be @ 100%): 2017-06-29: '''Win32: 97% - Win64: 97% - Mac: 97% - Linux32: 97% - Linux64: 97%'''
***** Most likely at 100% - showing 97% probably due to a hiccup in the server
Florin.Mezei
Confirmed users
2,482

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1173090...1174798"