Confirmed users
3,990
edits
ReleaseEngineering/Mozharness/How to run tests as a developer: Difference between revisions
Jump to navigation
Jump to search
ReleaseEngineering/Mozharness/How to run tests as a developer (view source)
Revision as of 13:52, 15 September 2014
, 15 September 2014no edit summary
No edit summary |
No edit summary |
||
| Line 1: | Line 1: | ||
'''NOTE: You can now use your LDAP credentials to authenticate and download the files that are behind Http authentication (e.g. pvtbuilds).''' | '''NOTE: You can now use your LDAP credentials to authenticate and download the files that are behind Http authentication (e.g. pvtbuilds).''' | ||
== | = Brief intro = | ||
Remember that there are jobs that download builds and run tests or performance tests, hence, requiring --installer-url and/or --test-url. | |||
Other jobs, simply create a build or run code analysis, hence, do not need --installer-url and/or --test-url. | |||
To read more about mozharness you can reach these: | To read more about mozharness you can reach these: | ||
| Line 10: | Line 9: | ||
* https://wiki.mozilla.org/ReleaseEngineering/Mozharness | * https://wiki.mozilla.org/ReleaseEngineering/Mozharness | ||
== | = Steps = | ||
== Common steps == | |||
=== Step 1 - Clone Mozharness === | === Step 1 - Clone Mozharness === | ||
Clone Mozharness: | Clone Mozharness: | ||
| Line 22: | Line 22: | ||
This activates the LDAP authentication for private files. | This activates the LDAP authentication for private files. | ||
It also removes --read-buildbot-configs from the list of actions. | It also removes --read-buildbot-configs from the list of actions. | ||
== Unit/talos tests steps == | |||
If you're trying to reproduce a unit test suite or a talos suite follow this last step. Otherwise, go to the "Build and other jobs" section. | |||
=== Step 4 - Append --installer-url and/or --test-url === | === Step 4 - Append --installer-url and/or --test-url === | ||
| Line 32: | Line 35: | ||
NOTICE: Talos jobs do not require --test-url. | NOTICE: Talos jobs do not require --test-url. | ||
=== | == Build and other jobs == | ||
You don't have to use --installer-url | |||
= Filing bugs = | |||
If you have tried mozharness and it is not doing what you need then [https://bugzilla.mozilla.org/enter_bug.cgi?product=Release%20Engineering&component=Mozharness file a bug] and CC :armenzg. | |||
= Example = | |||
We start with this production command: | We start with this production command: | ||
/tools/buildbot/bin/python scripts/scripts/b2g_emulator_unittest.py --cfg b2g/emulator_automation_config.py --test-suite mochitest --this-chunk 1 --total-chunks 9 --blob-upload-branch mozilla-central --download-symbols ondemand | /tools/buildbot/bin/python scripts/scripts/b2g_emulator_unittest.py --cfg b2g/emulator_automation_config.py --test-suite mochitest --this-chunk 1 --total-chunks 9 --blob-upload-branch mozilla-central --download-symbols ondemand | ||
| Line 42: | Line 51: | ||
--test-url http://pvtbuilds.pvt.build.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds/mozilla-central-emulator/20140910173159/b2g-35.0a1.en-US.android-arm.tests.zip | --test-url http://pvtbuilds.pvt.build.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds/mozilla-central-emulator/20140910173159/b2g-35.0a1.en-US.android-arm.tests.zip | ||
== | = Run mozharness ... = | ||
== ...on your local machine == | |||
Follow all the steps mentioned above. | Follow all the steps mentioned above. | ||
== | == ...with your local build == | ||
TODO: I believe --installer-path instead of --installer-url can be used with --cfg dev_config.py, however, we have to verify this and document it properly. | TODO: I believe --installer-path instead of --installer-url can be used with --cfg dev_config.py, however, we have to verify this and document it properly. | ||
== | If you figure it out please add it here and I will review it. | ||
== ...on a loaner machine == | |||
Follow all the steps mentioned above plus add these variables: | Follow all the steps mentioned above plus add these variables: | ||
* export '''DISPLAY=:0''' | * export '''DISPLAY=:0''' | ||
| Line 54: | Line 66: | ||
** This will allow your job to upload files to blobber | ** This will allow your job to upload files to blobber | ||
= Other examples = | |||
== Running Mozharness with a local version of Talos and Firefox on Linux == | |||
These instructions can be more general, but this should be a useful starting point for any similar configuration. | These instructions can be more general, but this should be a useful starting point for any similar configuration. | ||
| Line 120: | Line 132: | ||
</pre> | </pre> | ||
= Deprecated = | |||
== Step 3 - B) Create a buildprops.json == | |||
In the previous section we mention to use --installer-url as well as --test-url, this is not necessary if you create a file called buildprops.json since the URLs are already specified inside of it. | In the previous section we mention to use --installer-url as well as --test-url, this is not necessary if you create a file called buildprops.json since the URLs are already specified inside of it. | ||
You can find the information you need by loading a tbpl log and looking for ''"08:14:31 INFO - Using buildbot properties"''. | You can find the information you need by loading a tbpl log and looking for ''"08:14:31 INFO - Using buildbot properties"''. | ||
Don't try to make sense of it. Just copy it without the datestamps (I know, it sucks). | Don't try to make sense of it. Just copy it without the datestamps (I know, it sucks). | ||
== Step 4 - B) --no-read-buildbot-config == | |||
Append --no-read-buildbot-config to the command. | Append --no-read-buildbot-config to the command. | ||
A developer config removes the action "read-buildbot-config" from the list of actions.<br /> | A developer config removes the action "read-buildbot-config" from the list of actions.<br /> | ||
Some jobs could work with just this and not need a developer config, however, this is unlikely. | Some jobs could work with just this and not need a developer config, however, this is unlikely. | ||
== Step 4 - Use a developer config == | |||
You should be able to simply add --cfg foo_dev.py to the line that is run in production and make it easier for a developer to run the tests (e.g. --cfg android/androidarm.py --cfg android/androidarm'''_dev.py'''). All the files are under the ''configs'' directory. | You should be able to simply add --cfg foo_dev.py to the line that is run in production and make it easier for a developer to run the tests (e.g. --cfg android/androidarm.py --cfg android/androidarm'''_dev.py'''). All the files are under the ''configs'' directory. | ||