TestEngineering/Performance/Talos/Running: Difference between revisions

Jump to navigation Jump to search

TestEngineering/Performance/Talos/Running (view source)

Revision as of 16:20, 12 August 2019

2,437 bytes removed ,  12 August 2019
no edit summary
No edit summary
No edit summary
Line 52: Line 52:
= Are my Talos results OK? =
= Are my Talos results OK? =
The best way to answer this question is to push to try server and compare the performance of your push against a baseline, using Perfherder (it is recommended hitting retrigger a few times to get at least 5 data points for each test). After pushing to try with Talos tests enabled, you will receive an email with a link to Perfherder where you can compare your results. Make sure you look at the different platforms.
The best way to answer this question is to push to try server and compare the performance of your push against a baseline, using Perfherder (it is recommended hitting retrigger a few times to get at least 5 data points for each test). After pushing to try with Talos tests enabled, you will receive an email with a link to Perfherder where you can compare your results. Make sure you look at the different platforms.
= Hacking on Talos Itself =
== Dev Environment ==
Prerequisites:
Local clone of mozilla-central and a successful [https://developer.mozilla.org/en-US/docs/Simple_Firefox_build mozilla-central Firefox build] completed.
On Linux, setuptools installed:
    apt-get install python-setuptools
Not required on Windows. For other operating systems see [https://pypi.python.org/pypi/setuptools Setuptools].
   
On Linux, virtualenv installed:
    pip install virtualenv
Not required on Windows. For other operating systems see [http://www.virtualenv.org/en/latest/virtualenv.html#installation Virtualenv].
Setup talos (linux):
    cd mozilla-central/testing/talos
    virtualenv .
    . bin/activate
    python setup.py develop
Setup talos (Windows with mozilla-build):
    cd mozilla-central/testing/talos
    python INSTALL.py  # only required once or after updating mozilla-build
    source Scripts/activate  # you can also use '.' (dot) instead of 'source'
== Testing your Talos Patch Locally ==
Testing locally involves running some subset of the Talos tests on
desktop and possibly mobile.  Obviously not all permutations of tests
and the ways of running them can be tried, so common sense should be
used as to what is run.  You may also want to run Talos' internal
unittests: http://hg.mozilla.org/build/talos/file/tip/tests
You should tailor your choice of tests to pick those that cover what
you've changed programmatically, but in general you should probably
run at least one startup test and one pageloader test.  A good
baseline might be the 'tresize' and 'tsvgx' test suites.
Run the tests from mozilla-central/testing/talos (with the venv activated, as noted above):
talos --develop --executablePath pathtofirefox --activeTests tart
The '--develop' option indicates to run in develop mode and store the results in local.json + local.out.
The '--executablePath' option tells Talos where the firefox installation we want to run is located (i.e. '~/mozilla/objdir/dist/bin/firefox' or whatever the full path is to your firefox executable that will be running the tests).
The '--activeTests' argument provides is a list of tests we want to run (if running multiple tests, separate each test name with ':').
For a list of complete options:
    talos --help
== Testing your Talos Patch on Try Server ==
Just push your talos patch to the try server. See [[#Try_Server|the Try Server section]] above.
Davehunt
Confirmed users
2,207

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1216362"

Navigation menu