MozillaWiki

QA/Ondemand Update Tests

< QA
Revision as of 14:56, 5 May 2016 by Florin.Mezei (talk | contribs) (Created page with "= Summary = This page describes in detail the steps you need to take to run Ondemand Update tests for various Firefox builds. The setup below assumes you have access to [http:...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Summary

This page describes in detail the steps you need to take to run Ondemand Update tests for various Firefox builds. The setup below assumes you have access to Jenkins, either from within the internal network or via VPN.

Setting up the config file

  1. Look in Mozilla wiki for the previous build, for example if you were to test 44.0.2
    1. Go to https://wiki.mozilla.org/index.php?title=Releases/Firefox_44/Test_Plan
    2. Go to the info for the previous build: https://wiki.mozilla.org/index.php?title=Releases/Firefox_44/Test_Plan#44.0.1
    3. Under the "Testing"->"Automated" section there should be a link that says “configs” - click on it
    4. Copy the pre-formatted text (starting with [testrun]) in a “config.ini” file on your computer - you can name the file any way you want but it's recommended you give it a name that will help you keep track of it (e.g. "46b7_build1_beta-cdntest_win64.ini" for a config file used to test 46 Beta 7 build 1, with a 64 bit Windows build, on the beta-cdntest channel)
    5. Open the config file for editing
    6. Modify the “target-version” value to the version corresponding to the build you are testing, making sure the correct format is used (see examples below):
      1. Beta - 46.0b4#2 - means 46 Beta 4, build 2 (found here: http://archive.mozilla.org/pub/firefox/candidates/46.0b4-candidates/build2/)
      2. Release/Dot release - 44.0.2#3 - means 44.0.2, build 3 (found here: http://archive.mozilla.org/pub/firefox/candidates/44.0.2-candidates/build3/)
      3. ESR - 45.0.1esr#2 - means ESR 45.0.1, build 2 (found here: http://archive.mozilla.org/pub/firefox/candidates/45.0.1esr-candidates/build2/)
    7. Modify the “channel” value to the channel you want to test (notice how the order to test the channels is specified in each corresponding test plan:
      1. Beta - https://wiki.mozilla.org/Releases/Firefox_46/Test_Plan#Beta_4 -> Testing -> Automated = order is: “beta-cdntest”, then “beta”
      2. Release/Dot Release - https://wiki.mozilla.org/Releases/Firefox_44/Test_Plan#44.0.2 -> Testing -> Automated = order is: “release-localtest”, then “release-cdntest”, then “release”
      3. ESR - https://wiki.mozilla.org/Releases/Firefox_45_ESR/Test_Plan#Firefox_45.0.1_ESR -> Checklist = order is: “esr-localtest”, then “esr-cdntest”, then “esr”
    8. Pick one version in the config file (e.g. 44.0.1) and change it to the previous version (in this case 44.0.2) everywhere in the file (this way you'll also test the update from the immediately previous version)
    9. Save the file locally

Notes

  1. Example of test plans for Firefox and ESR (just replace the version in the URL to get to the wiki you need):
    1. https://wiki.mozilla.org/Releases/Firefox_44/Test_Plan
    2. https://wiki.mozilla.org/Releases/Firefox_45_ESR/Test_Plan
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Ondemand_Update_Tests&oldid=1131230"