Confirmed users
3,376
edits
Buildbot/Talos/Data: Difference between revisions
m
s/old repo links/m-c links/
m (s/replicates/data points/) |
m (s/old repo links/m-c links/) |
||
| Line 27: | Line 27: | ||
== Subtest Filters == | == Subtest Filters == | ||
We have a variety of [ | We have a variety of [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/filter.py filters] defined for Talos. I will explain what each filter is, and you can see the exact settings used for each filter by looking at the individual [https://wiki.mozilla.org/index.php?title=Buildbot/Talos/Tests tests]. | ||
=== ignore_first === | === ignore_first === | ||
| Line 33: | Line 33: | ||
* input: an array of subtest data points | * input: an array of subtest data points | ||
* returns: an array of data points | * returns: an array of data points | ||
* source: [ | * source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/filter.py#l127 filter.py] | ||
* used in most tests with X=1, X=2, and X=5 (5 is the normal case) | * used in most tests with X=1, X=2, and X=5 (5 is the normal case) | ||
| Line 40: | Line 40: | ||
* input: an array of subtest data points | * input: an array of subtest data points | ||
* returns: a single value | * returns: a single value | ||
* source: [ | * source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/filter.py#l58 filter.py] | ||
* used in most tests | * used in most tests | ||
| Line 47: | Line 47: | ||
* input: an array of subtest data points | * input: an array of subtest data points | ||
* returns: a single value | * returns: a single value | ||
* source: [ | * source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/filter.py#l50 filter.py] | ||
* used in kraken for subtests | * used in kraken for subtests | ||
| Line 54: | Line 54: | ||
* input: an array of dromaeo (DOM|CSS) subtest data points | * input: an array of dromaeo (DOM|CSS) subtest data points | ||
* returns: a single number (geometric_mean of the metric summarization) | * returns: a single number (geometric_mean of the metric summarization) | ||
* source: [ | * source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/filter.py#l92 filter.py] | ||
* used in dromaeo_dom and dromaeo_css to build a single value for the subtests | * used in dromaeo_dom and dromaeo_css to build a single value for the subtests | ||
| Line 60: | Line 60: | ||
* input: an array of v8_7 subtest data points | * input: an array of v8_7 subtest data points | ||
* returns: a single value representing the benchmark weighted score for the subtest | * returns: a single value representing the benchmark weighted score for the subtest | ||
* source: [ | * source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/filter.py#l168 filter.py] | ||
* used in v8_7 for the subtests | * used in v8_7 for the subtests | ||
| Line 72: | Line 72: | ||
* inputs: array of subtest summarized data points (one point per subtest) | * inputs: array of subtest summarized data points (one point per subtest) | ||
* returns: a single value representing the geometric mean of all the subtests | * returns: a single value representing the geometric mean of all the subtests | ||
* source: [ | * source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/filter.py#l114 filter.py] | ||
* used for most tests | * used for most tests | ||
| Line 79: | Line 79: | ||
* inputs: array of v8 subtest summaries | * inputs: array of v8 subtest summaries | ||
* returns: a single v8 score | * returns: a single v8 score | ||
* source: [ | * source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/output.py#l102 output.py] | ||
* used for v8 version 7 only | * used for v8 version 7 only | ||
| Line 86: | Line 86: | ||
* inputs: array of Canvasmark subtest results | * inputs: array of Canvasmark subtest results | ||
* returns: a single Canvasmark score | * returns: a single Canvasmark score | ||
* source: [ | * source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/output.py#l115 output.py] | ||
* used for Canvasmark only | * used for Canvasmark only | ||
| Line 93: | Line 93: | ||
* inputs: array of Kraken subtest results | * inputs: array of Kraken subtest results | ||
* returns: a single Kraken score | * returns: a single Kraken score | ||
* source: [ | * source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/output.py#l108 output.py] | ||
* used for Kraken only | * used for Kraken only | ||