160
edits
Buildbot/Talos/Tests: Difference between revisions
Jump to navigation
Jump to search
Sync with :rwood's changes from January 18, 2018
(Sync with :rwood's changes from January 18, 2018) |
|||
| Line 19: | Line 19: | ||
* [https://wiki.mozilla.org/Buildbot/Talos/Tests#tresize tresize] | * [https://wiki.mozilla.org/Buildbot/Talos/Tests#tresize tresize] | ||
* [https://wiki.mozilla.org/Buildbot/Talos/Tests#sessionrestore.2Fsessionrestore_no_auto_restore.2Fsessionrestore_many_windows sessionrestore[_no_auto_restore/_many_windows]] | * [https://wiki.mozilla.org/Buildbot/Talos/Tests#sessionrestore.2Fsessionrestore_no_auto_restore.2Fsessionrestore_many_windows sessionrestore[_no_auto_restore/_many_windows]] | ||
* [https://wiki.mozilla.org/Buildbot/Talos/Tests#CanvasMark tcanvasmark] (this is the one test that is a startup test but unique as we run 5 cycles and report a series of numbers) | |||
=== Page Load Tests === | === Page Load Tests === | ||
| Line 60: | Line 61: | ||
Tests which are imported benchmarks are: | Tests which are imported benchmarks are: | ||
* [https://wiki.mozilla.org/Buildbot/Talos/Tests#CanvasMark Canvasmark] | |||
* [https://wiki.mozilla.org/Buildbot/Talos/Tests#V8.2C_version_7 V8 Version 7] | * [https://wiki.mozilla.org/Buildbot/Talos/Tests#V8.2C_version_7 V8 Version 7] | ||
* [https://wiki.mozilla.org/Buildbot/Talos/Tests#kraken Kraken] | * [https://wiki.mozilla.org/Buildbot/Talos/Tests#kraken Kraken] | ||
| Line 241: | Line 243: | ||
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them. | * None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them. | ||
=== CanvasMark === | |||
* contact: :snorp, :milan, :jmaher | |||
* source: https://github.com/kevinroast/CanvasMark | |||
* type: [https://wiki.mozilla.org/Buildbot/Talos#Page_Load_Tests PageLoader] | |||
* data: 5 cycles of the entire benchmark, each subtest will have 5 data points (see below) | |||
* summarization: | |||
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] data point, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 4; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l522 source: test.py] | |||
** score: [https://wiki.mozilla.org/Buildbot/Talos/Data#Canvasmark_metric canvasmark metric] on all summarized subtests | |||
* '''Higher is better''' | |||
{| cellspacing="1" cellpadding="1" border="1" | |||
|- | |||
| Talos test name | |||
| Description | |||
|- | |||
|tcanvasmark | |||
| | |||
|} | |||
These tests run the third-party [http://www.kevs3d.co.uk/dev/canvasmark/ CanvasMark] benchmark suite, which measures the browser's ability to render a variety of canvas animations at a smooth framerate as the scenes grow more complex. | |||
Results are a score "based on the length of time the browser was able to maintain the test scene at greater than 30 FPS, multiplied by a weighting for the complexity of each test type". | |||
{| class="mw-collapsible mw-collapsed wikitable" | |||
! Example data | |||
|- | |||
| <pre> | |||
1;Asteroids - Vectors;704;672;713;738;796 | |||
2;Asteroids - Bitmaps- shapes- text;1105;1139;1002;1122;1050 | |||
3;Asteroids - Shapes- shadows- blending;740;731;775;725;732 | |||
4;Arena5 - Vectors- shadows- bitmaps- text;841;679;865;942;873 | |||
5;Plasma - Maths- canvas shapes;583;583;557;604;632 | |||
6;3D Rendering - Maths- polygons- image transforms;340;445;374;425;425 | |||
7;Pixel blur - Math- getImageData- putImageData;1409;1408;1415;1508;1335 | |||
</pre> | |||
|} | |||
==== Possible regression causes ==== | |||
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them. | |||
=== cpstartup === | === cpstartup === | ||
* contact: :mconley | * contact: :mconley | ||