Confirmed users
2,197
edits
TestEngineering/Performance/Talos: Difference between revisions
Jump to navigation
Jump to search
no edit summary
m (Davehunt moved page TestEngineering/Performance/Sheriffing/Talos to TestEngineering/Performance/Talos) |
No edit summary |
||
| Line 22: | Line 22: | ||
* Taskcluster runs a talos job on a hardware machine when one is available - this is bootstrapped by [https://searchfox.org/mozilla-central/source/testing/mozharness/mozharness/mozilla/testing/talos.py Mozharness] | * Taskcluster runs a talos job on a hardware machine when one is available - this is bootstrapped by [https://searchfox.org/mozilla-central/source/testing/mozharness/mozharness/mozilla/testing/talos.py Mozharness] | ||
** [https://searchfox.org/mozilla-central/source/testing/mozharness/mozharness/mozilla/testing/talos.py Mozharness\ downloads the build, talos.zip (found in [http://hg.mozilla.org/mozilla-central/tip/testing/talos/talos.json talos.json]), and creates a virtualenv for running the test. | ** [https://searchfox.org/mozilla-central/source/testing/mozharness/mozharness/mozilla/testing/talos.py Mozharness\ downloads the build, talos.zip (found in [http://hg.mozilla.org/mozilla-central/tip/testing/talos/talos.json talos.json]), and creates a virtualenv for running the test. | ||
** Mozharness[ | ** Mozharness [[Buildbot/Talos/Running#How_Talos_is_Run_in_Production|configures the test and runs it]] | ||
** After the test is completed the data is uploaded to [https://treeherder.mozilla.org/perf.html#/graphs PerfHerder] | ** After the test is completed the data is uploaded to [https://treeherder.mozilla.org/perf.html#/graphs PerfHerder] | ||
* Treeherder displays a green (all OK) status and has a link to the [https://treeherder.mozilla.org/perf.html#/graphs PerfHerder] | * Treeherder displays a green (all OK) status and has a link to the [https://treeherder.mozilla.org/perf.html#/graphs PerfHerder] | ||
* 13 pushes later, [http://hg.mozilla.org/graphs/file/tip/server/analysis/analyze_talos.py analyze_talos.py] is ran which compares your push to the previous 12 pushes and next 12 pushes to look for a [ | * 13 pushes later, [http://hg.mozilla.org/graphs/file/tip/server/analysis/analyze_talos.py analyze_talos.py] is ran which compares your push to the previous 12 pushes and next 12 pushes to look for a [[Buildbot/Talos/Data#Regressions|regression]] | ||
** if a regression is found, it will be posted on [https://treeherder.mozilla.org/perf.html#/alerts Perfherder Alerts] | ** if a regression is found, it will be posted on [https://treeherder.mozilla.org/perf.html#/alerts Perfherder Alerts] | ||
| Line 33: | Line 33: | ||
To create a new test you ideally need a webpage(s) that performs the test(s) desired. The results can be a simple alert. | To create a new test you ideally need a webpage(s) that performs the test(s) desired. The results can be a simple alert. | ||
Once you've got that, just talk to folks in # | Once you've got that, just talk to folks in #perftest (:davehunt, :rwood, :igoldan) and they will handle the majority of the [[/Misc#Adding_a_new_test|steps required]]. | ||
=== Bugs === | === Bugs === | ||
| Line 41: | Line 41: | ||
=== Subpages === | === Subpages === | ||
* Definition of [ | * Definition of [[/Tests|tests]] | ||
* How to [ | * How to [[/Profiling|use Talos profiling]]. | ||
* How to [ | * How to [/Running|Run Talos]], and how it is run in production. | ||
** How to run Talos on [[Mobile/Fennec/Android#talos mobile]] | ** How to run Talos on [[Mobile/Fennec/Android#talos mobile]] | ||
* What our raw | * What our raw [[/Data|data]] looks like and what our formulas for filtering and summarization. | ||
* History and other uses of [ | * History and other uses of [[/Misc|Talos]] | ||
=== Blog posts about Talos === | === Blog posts about Talos === | ||
| Line 54: | Line 54: | ||
* http://ask.mozilla.org/question/45/how-can-i-know-if-my-push-to-try-regresses-talos-results/ | * http://ask.mozilla.org/question/45/how-can-i-know-if-my-push-to-try-regresses-talos-results/ | ||
* http://limpet.net/mbrubeck/2013/11/10/improving-regression-detection.html | * http://limpet.net/mbrubeck/2013/11/10/improving-regression-detection.html | ||
* [ | * [[Buildbot/Talos|duplicated & updated from old page]] | ||