canmove, Confirmed users, Bureaucrats and Sysops emeriti
4,747
edits
QA/Mozmill Test Automation/On Demand Update Testing/Documentation: Difference between revisions
QA/Mozmill Test Automation/On Demand Update Testing/Documentation (view source)
Revision as of 15:44, 16 August 2011
, 16 August 2011→Sample File
| (8 intermediate revisions by one other user not shown) | |||
| Line 7: | Line 7: | ||
== Heartbeat Emitter == | == Heartbeat Emitter == | ||
* Generates traffic to keep Pulse connections from being automatically closed. | * Generates traffic to keep [[#Release_Listener|release listener]] Pulse connections from being automatically closed. | ||
* Fine to run more than one. If you're not sure, start one. | * Fine to run more than one. If you're not sure, start one. | ||
| Line 52: | Line 52: | ||
* --functional will redefine which test script is called for functional requests. Again, the default accomodates our internal systems. | * --functional will redefine which test script is called for functional requests. Again, the default accomodates our internal systems. | ||
* --debug lets you know when things are ignored due to platform/cluster mismatches. This can be very noisy, but it's useful when you have no idea why something isn't responding. | * --debug lets you know when things are ignored due to platform/cluster mismatches. This can be very noisy, but it's useful when you have no idea why something isn't responding. | ||
* The listeners always print output to the console regarding what they're testing. This can be useful to make sure they're working. | |||
* If listeners freeze up, chances are they're not receiving a [[#Heartbeat_Emitter|heartbeat]]. | |||
= Release Prep = | = Release Prep = | ||
| Line 64: | Line 66: | ||
* The configuration file is not used during the actual release testing process. | * The configuration file is not used during the actual release testing process. | ||
Our testing scripts know how to navigate a pre-created directory hierarchy to perform tests; the #Staging_Script staging script] creates that hierarchy by reading the configuration file and putting the binaries to be tested against in the appropriate places. | Our testing scripts know how to navigate a pre-created directory hierarchy to perform tests; the [[#Staging_Script|staging script]] creates that hierarchy by reading the configuration file and putting the binaries to be tested against in the appropriate places. | ||
=== Sample File === | === Sample File === | ||
| Line 90: | Line 92: | ||
; Rest of keys are test matrix for that OS, as: | ; Rest of keys are test matrix for that OS, as: | ||
; version=locale1 locale2 locale3 | ; version=locale1 locale2 locale3 | ||
; 3.6.18=en-US (release build of Firefox 3.6.18) | |||
; 3.6.18#2=de (candidate builds #2 of the upcoming Firefox 3.6.18) | |||
[win7] | [win7] | ||
| Line 125: | Line 129: | ||
* For functional tests, that will be once the binary is available for testing. | * For functional tests, that will be once the binary is available for testing. | ||
* For update tests the binaries to be staged are previous versions and can be staged at any time. | * For update tests the binaries to be staged are previous versions and can be staged at any time. | ||
* While we typically do so internally, you do not need to provide a section for every OS/listener. Unspecified OSes will not be tested against. | * While we typically do so internally, you do not need to provide a section for every OS/[[#Release_Listener|listener]]. Unspecified OSes will not be tested against. | ||
=== Usage === | === Usage === | ||