MozillaWiki

Buildbot/Talos/Tests: Difference between revisions

Page Discussion

Buildbot/Talos/Tests (view source)

Revision as of 21:37, 3 January 2018

48 bytes added ,  3 January 2018
→‎Test Descriptions: Finish putting the test list in alphabetical order.
(→‎Test Descriptions: Checkpointing a re-ordering to alphabetical so that I don't get conflicts in case somebody else is editing this.)
(→‎Test Descriptions: Finish putting the test list in alphabetical order.)
Line 79: Line 79:


== Test Descriptions ==
== Test Descriptions ==
'''Please keep these in alphabetical order'''
=== a11y ===
=== a11y ===
* contact: :davidb, :tbsaunde, :jmaher
* contact: :davidb, :tbsaunde, :jmaher
Line 546: Line 549:
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.


=== Microbench ===
* contact: :benwa
* source: [[https://dxr.mozilla.org/mozilla-central/source/testing/gtest/mozilla/MozGTestBench.cpp MozGTestBench.cpp]]
* type: Custom GTest micro-benchmarking
* data: Time taken for a GTest function to execute
* summarization: Not a Talos test. This suite is provides a way to add low level platform performance regression tests for things that are not suited to be tested by Talos. See the [[https://wiki.mozilla.org/Buildbot/Talos/Sheriffing#Microbench_Policy Microbench Sheriffing Policy]] for some notes on how to treat regressions.
==== Possible regression causes ====
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
=== Other data ===


These are not part of the Talos code, but like Talos they are benchmarks that record data using the graphserver and are analyzed by the same scripts for regressions.


=== tp5o_scroll ===
==== Number of Constructors (num_ctors) ====
* contact: :jmaher
* source: [https://people.mozilla.org/~jmaher/taloszips/zips/tp5n.zip tp5n.zip]
* type: [https://wiki.mozilla.org/Buildbot/Talos#Page_Load_Tests PageLoader]
* data: we load each of the 51 tp5o pages 12 times, resulting in 51 sets of 12 data points.
* '''To run it locally''', you'd need [[#tp5n_pages_set|tp5n.zip]].
* summarization:
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] data point, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 11; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l470 source: test.py]
** suite: [https://wiki.mozilla.org/Buildbot/Talos/Data#geometric_mean geometric mean] of the 51 subtest results.


{| cellspacing="1" cellpadding="1" border="1"
This test runs at build time and measures the number of static initializers in the compiled code.  Reducing this number is helpful for [https://blog.mozilla.org/tglek/2010/05/27/startup-backward-constructors/ startup optimizations].
|-
 
| Talos test name
* https://hg.mozilla.org/build/tools/file/348853aee492/buildfarm/utils/count_ctors.py
| Description
** these are run for linux 32+64 opt and pgo builds.
|-
 
|tp5o_scroll
===== Possible regression causes =====
|Measures average frames interval while scrolling the pages of the tp5o set
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
|}


This test is identical to tscrollx, but it scrolls the 50 pages of the tp5o set (rather than 6 synthetic pages which tscrollx scrolls). For each page, the test waits 500ms after the page load event fires, then iterates 100 scroll steps of 10px each (or until the bottom of the page is reached - whichever comes first), then reports the average frame interval.
=== PerfStrip Tests ===


{| class="mw-collapsible mw-collapsed wikitable"
* contact: :milan
! Example data
* source: https://dxr.mozilla.org/mozilla-central/source/xpcom/tests/gtest/TestStrings.cpp
|-
* type: Microbench
| <pre>
* reporting: execution time in ms (lower is better) for 100k function calls
0;163.com/www.163.com/index.html;9.73;8.61;7.37;8.17;7.58;7.29;6.88;7.45;6.91;6.61;8.47;7.12
* data: each test run and measured 5 times
1;56.com/www.56.com/index.html;10.85;10.24;10.75;10.30;10.23;10.10;10.31;10.06;11.10;10.06;9.56;10.30
* summarization:
2;aljazeera.net/aljazeera.net/portal.html;9.23;7.15;7.50;7.26;7.73;7.05;7.14;7.66;7.23;7.93;7.26;7.18
 
3;amazon.com/www.amazon.com/Kindle-Wireless-Reader-Wifi-Graphite/dp/B002Y27P3M/507846.html;7.14;6.87;7.18;6.31;6.93;6.71;6.37;7.00;6.59;5.37;7.31;6.13
PerfStripWhitespace - call StripWhitespace() on 5 different test cases 20k times (each)
4;bbc.co.uk/www.bbc.co.uk/news/index.html;7.39;6.33;6.22;7.66;6.67;7.77;6.91;7.74;7.08;6.36;6.03;7.12
 
5;beatonna.livejournal.com/beatonna.livejournal.com/index.html;5.79;5.79;5.68;5.46;5.55;5.48;5.69;5.83;5.88;5.97;5.93;5.88
PerfStripCharsWhitespace - call StripChars("\f\t\r\n") on 5 different test cases 20k times (each)
6;bild.de/www.bild.de/index.html;8.65;7.63;7.17;6.36;7.44;7.68;8.63;6.71;8.34;7.15;7.82;7.70
 
7;cgi.ebay.com/cgi.ebay.com/ALL-NEW-KINDLE-3-eBOOK-WIRELESS-READING-DEVICE-W-WIFI-/130496077314@pt=LH_DefaultDomain_0&hash=item1e622c1e02.html;7.12;6.81;7.22;6.98;7.05;5.68;7.15;6.54;7.31;7.18;7.82;7.77
PerfStripCRLF - call StripCRLF() on 5 different test cases 20k times (each)
8;chemistry.about.com/chemistry.about.com/index.html;6.76;6.17;6.41;6.88;5.67;5.47;6.83;6.28;6.16;6.81;6.21;6.75
 
9;chinaz.com/chinaz.com/index.html;10.72;7.99;7.33;7.10;7.85;8.62;8.39;6.72;6.26;6.65;8.14;7.78
PerfStripCharsCRLF() - call StripChars("\r\n") on 5 different test cases 20k times (each)
10;cnn.com/www.cnn.com/index.html;7.73;6.80;6.08;8.27;9.24;7.81;7.69;7.05;8.17;7.70;7.90;6.81
 
11;dailymail.co.uk/www.dailymail.co.uk/ushome/index.html;6.37;8.28;7.19;8.00;8.09;7.43;6.90;7.24;7.77;7.29;7.38;6.14
==== Possible regression causes ====
12;dailymotion.com/www.dailymotion.com/us.html;9.53;9.80;9.29;9.03;9.10;8.64;8.62;8.71;8.77;9.81;9.64;8.96
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
13;digg.com/digg.com/news/story/New_logo_for_Mozilla_Firefox_browser.html;7.72;7.06;7.60;5.67;6.85;7.32;7.80;5.98;8.27;6.68;7.52;8.39
 
14;ezinearticles.com/ezinearticles.com/index.html@Migraine-Ocular---The-Eye-Migraines&id=4684133.html;7.14;7.11;8.09;7.17;6.87;7.12;7.65;7.74;7.26;7.36;6.91;6.95
=== rasterflood_svg ===
15;globo.com/www.globo.com/index.html;6.71;7.91;5.83;7.34;7.75;8.00;7.73;7.85;7.03;6.42;8.43;8.11
* contact: :dvander
16;google.com/www.google.com/search@q=mozilla.html;6.49;6.23;7.96;6.39;7.23;8.19;7.35;7.39;6.94;7.24;7.55;7.62
* source: [[https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/gfx/benchmarks/rasterflood_svg.html rasterflood_svg.html]]
17;goo.ne.jp/goo.ne.jp/index.html;8.56;7.18;7.15;7.03;6.85;7.62;7.66;6.99;7.84;7.51;7.23;7.18
* type: [https://wiki.mozilla.org/Buildbot/Talos#Page_Load_Tests PageLoader]
18;guardian.co.uk/www.guardian.co.uk/index.html;7.32;7.62;8.18;7.62;7.83;8.08;7.60;8.17;8.47;7.54;7.92;8.09
* data: we load the rasterflood_svg.html page ten times, measuring pageload each time, generating 10 data points.
19;homeway.com.cn/www.hexun.com/index.html;10.18;8.75;8.83;8.64;8.98;8.07;7.76;9.29;8.05;7.55;8.91;7.78
* summarization:
20;huffingtonpost.com/www.huffingtonpost.com/index.html;8.38;7.17;7.03;6.83;6.49;6.47;6.69;7.08;6.81;7.29;7.13;7.70
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] data point, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 9; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l986 source: test.py]
21;ifeng.com/ifeng.com/index.html;12.45;8.65;8.75;7.56;8.26;7.71;8.04;7.45;7.83;7.14;8.38;7.68
 
22;imdb.com/www.imdb.com/title/tt1099212/index.html;8.53;5.65;6.94;7.18;6.10;7.57;6.26;8.34;8.16;7.29;7.62;8.51
This page animates some complex SVG patterns in a requestAnimationFrame callback. However, it also churns the CPU during each callback, spinning an empty loop for 14ms each frame. The intent is that, if we consider the rasterization costs to be 0, then the animation should run close to 60fps. Otherwise it will lag. Since rasterization costs are not 0, the lower we can get them, the faster the test will run. The test runs in ASAP mode to maximize framerate. The result is how quickly the browser is able to render 600 frames of the animation.
23;imgur.com/imgur.com/gallery/index.html;8.10;7.20;7.50;7.88;7.27;6.97;8.13;7.14;7.59;7.39;8.01;8.82
 
24;indiatimes.com/www.indiatimes.com/index.html;8.00;6.74;7.37;8.52;7.03;8.45;7.08;8.47;9.26;7.89;7.17;6.74
Improvements (or regressions) to general painting performance or SVG are likely to affect this benchmark.
25;mail.ru/mail.ru/index.html;7.64;9.50;9.47;7.03;6.45;6.24;8.03;6.72;7.18;6.39;6.25;6.25
 
26;mashable.com/mashable.com/index.html;7.97;8.03;6.10;7.80;7.91;7.26;7.49;7.45;7.60;7.08;7.63;7.36
==== Possible regression causes ====
27;media.photobucket.com/media.photobucket.com/image/funny%20gif/findstuff22/Best%20Images/Funny/funny-gif1.jpg@o=1.html;290.00;195.00;217.00;199.00;204.00;196.00;198.00;206.00;209.00;208.00;192.00;196.00
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
28;myspace.com/www.myspace.com/albumart.html;14.40;13.45;13.29;13.62;13.42;14.15;13.86;14.34;14.69;14.10;13.82;14.13
29;naver.com/www.naver.com/index.html;9.15;8.31;9.40;9.89;7.29;8.43;8.87;8.77;8.96;8.24;8.16;8.21
30;noimpactman.typepad.com/noimpactman.typepad.com/index.html;7.27;7.14;7.70;7.86;7.43;6.95;7.30;7.58;7.51;7.95;7.43;7.05
31;page.renren.com/page.renren.com/index.html;7.94;8.13;6.76;7.77;6.93;6.60;7.62;7.61;6.88;7.56;7.55;7.48
32;people.com.cn/people.com.cn/index.html;11.92;9.22;8.49;8.55;8.34;8.49;6.91;9.92;8.69;8.63;7.69;9.34
33;rakuten.co.jp/www.rakuten.co.jp/index.html;11.10;7.13;8.68;7.85;8.37;7.91;6.74;8.27;8.55;8.93;7.15;9.02
34;reddit.com/www.reddit.com/index.html;6.38;7.95;6.84;7.04;6.96;7.15;8.05;7.71;8.13;7.13;6.60;7.53
35;reuters.com/www.reuters.com/index.html;7.51;7.25;6.60;6.98;7.41;6.45;7.61;7.46;6.11;7.15;7.05;6.94
36;slideshare.net/www.slideshare.net/jameswillamor/lolcats-in-popular-culture-a-historical-perspective.html;7.20;6.32;6.80;6.87;6.29;6.45;7.18;6.92;6.57;7.41;7.08;6.51
37;sohu.com/www.sohu.com/index.html;11.72;9.64;8.85;7.12;7.96;9.14;7.76;8.19;7.14;7.68;8.08;7.24
38;spiegel.de/www.spiegel.de/index.html;7.24;7.30;6.64;7.01;6.74;6.70;6.36;6.84;7.86;7.08;7.12;7.40
39;stackoverflow.com/stackoverflow.com/questions/184618/what-is-the-best-comment-in-source-code-you-have-ever-encountered.html;7.39;5.88;7.22;6.51;7.12;6.51;6.46;6.53;6.63;6.54;6.48;6.80
40;store.apple.com/store.apple.com/us@mco=Nzc1MjMwNA.html;6.23;7.17;7.39;8.98;7.99;8.03;9.12;8.37;8.56;7.61;8.06;7.55
41;thepiratebay.org/thepiratebay.org/top/201.html;9.08;8.93;8.09;7.49;7.30;7.80;7.54;7.65;7.91;7.53;8.37;8.04
42;tudou.com/www.tudou.com/index.html;10.06;9.38;8.68;7.37;8.57;9.11;8.20;7.91;8.78;9.64;8.11;8.47
43;uol.com.br/www.uol.com.br/index.html;9.04;9.49;9.48;9.31;8.68;8.41;9.16;8.91;9.49;8.37;9.77;8.73
44;w3.org/www.w3.org/standards/webdesign/htmlcss.html;6.62;5.98;6.87;6.47;7.22;6.05;6.42;6.50;7.47;7.18;5.82;7.11
45;wsj.com/online.wsj.com/home-page.html;7.49;8.57;6.84;8.12;7.60;7.24;8.16;8.22;6.81;8.28;8.11;8.58
46;xinhuanet.com/xinhuanet.com/index.html;13.66;9.21;10.09;9.56;8.99;10.29;10.24;8.91;11.23;10.82;9.64;10.11
47;xunlei.com/xunlei.com/index.html;8.99;8.16;8.82;8.37;7.01;8.48;7.98;8.69;8.10;8.10;7.10;6.41
48;yelp.com/www.yelp.com/biz/alexanders-steakhouse-cupertino.html;8.18;7.45;7.01;8.14;7.12;7.82;8.24;7.13;7.00;6.41;6.85;7.31
49;youku.com/www.youku.com/index.html;12.21;10.29;10.37;10.34;8.40;9.82;9.23;9.91;9.64;9.91;8.90;10.23
50;youtube.com/www.youtube.com/music.html;9.90;9.06;9.29;9.17;8.85;8.77;9.83;9.21;9.29;10.09;9.69;8.64
</pre>
|}


==== Possible regression causes ====
=== rasterflood_gradient ===
Some examples of things that cause regressions in this test are:
* contact: :dvander
* Increased displayport size (which causes a larger display list to be built)
* source: [[https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/gfx/benchmarks/rasterflood_gradient.html rasterflood_gradient.html]]
* Slowdown in the building of display list
* Slowdown in rasterization of content
 
=== tp5 ===
* contact: :jmaher
* source: [https://people.mozilla.org/~jmaher/taloszips/zips/tp5n.zip tp5n.zip]
* type: [https://wiki.mozilla.org/Buildbot/Talos#Page_Load_Tests PageLoader]
* type: [https://wiki.mozilla.org/Buildbot/Talos#Page_Load_Tests PageLoader]
* data: we load each of the 51 tp5o pages 25 times, resulting in 51 sets of 25 data points.
* data: we load the rasterflood_gradient.html page ten times, computing a score each time, generating 10 data points.
* '''To run it locally''', you'd need [[#tp5n_pages_set|tp5n.zip]].
* summarization:
* summarization:
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] '''5''' data points, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 20; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l449 source: test.py]
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] data point, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 9; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l986 source: test.py]
** suite: [https://wiki.mozilla.org/Buildbot/Talos/Data#geometric_mean geometric mean] of the 51 subtest results.


{| cellspacing="1" cellpadding="1" border="1"
This page animates some complex gradient patterns in a requestAnimationFrame callback. However, it also churns the CPU during each callback, spinning an empty loop for 14ms each frame. The intent is that, if we consider the rasterization costs to be 0, then the animation should run close to 60fps. Otherwise it will lag. Since rasterization costs are not 0, the lower we can get them, the faster the test will run. The test runs in ASAP mode to maximize framerate.
|-
| Talos test name
| Description
|-
|tp5o
|tp5 with limited pageset (48 pages as others have too much noise)
|}


Tests the time it takes Firefox to load the [https://people.mozilla.org/~jmaher/taloszips/zips/tp5n.zip tp5 web page test set]. The web set was culled from the Alexa top 500 April 8th, 2011 and consists of 51 pages.  
The test runs for 10 seconds, and the resulting score is how many frames we were able to render during that time. Higher is better. Improvements (or regressions) to general painting performance or gradient rendering will affect this benchmark.


Here are the broad steps we use to create the test set:
==== Possible regression causes ====
# Take the Alexa top 500 sites list
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
# Remove all sites with questionable or explicit content
 
# Remove duplicate site (for ex. many Google search front pages)
=== sessionrestore/sessionrestore_no_auto_restore/sessionrestore_many_windows ===
# Manually select to keep interesting pages (such as pages in different locales)
* contact: :Yoric, :mikedeboer
# Select a more representative page from any site presenting a simple search/login/etc. page
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/startup_test/sessionrestore talos/sessionrestore]
# Deal with Windows 255 char limit for cached pages
* bug: https://bugzilla.mozilla.org/show_bug.cgi?id=936630, https://bugzilla.mozilla.org/show_bug.cgi?id=1331937
# Limit test set to top 100 pages
* type: Startup
* measuring: time spent reading and restoring the session.
* reporting: interval in ms (lower is better).
* data: we load the session restore index page 10 times to collect 1 set of 10 data points.
* summarization:
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] data point, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 9 data points; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l305 source: test.py]
** suite: identical to subtest
 
Three tests measure the time spent reading and restoring the session from a valid sessionstore.js. Time starts ticking during event sessionRestoreInit and stops during event sessionRestored.


Note that the above steps did not eliminate all outside network access so we had to take further action to scrub all the pages so that there are 0 outside network accesses (this is done so that the tp test is as deterministic measurement of our rendering/layout/paint process as possible).
In ''sessionrestore'', this is tested with a configuration that requires the session to be restored. In ''sessionrestore_no_auto_restore'', this is tested with a configuration that requires the session to not be restored. Both of the above tests use a sessionstore.js file that contains one window and roughly 89 tabs. In ''sessionrestore_many_windows'', this is tested with a sessionstore.js that contains 3 windows and 130 tabs. The first window contains 50 tabs, 80 remaning tabs are divided equally between the second and the third window.


{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
Line 669: Line 643:
|-
|-
| <pre>
| <pre>
0;163.com/www.163.com/index.html;1035;512;542;519;505;514;551;513;554;793;487;528;528;498;503;530;527;490;521;535;521;496;498;564;520
[2362.0, 2147.0, 2171.0, 2134.0, 2116.0, 2145.0, 2141.0, 2141.0, 2136.0, 2080.0]
1;56.com/www.56.com/index.html;1081;583;580;577;597;580;623;558;572;592;598;580;564;583;596;600;579;580;566;573;566;581;571;600;586
</pre>
2;aljazeera.net/aljazeera.net/portal.html;688;347;401;382;346;362;347;372;337;
|}
 
==== Possible regression causes ====
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
 
=== speedometer ===
* contact: :jmaher, :rwood
* source: [[https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/speedometer/speedometer.manifest speedometer.manifest]]
* type: [https://wiki.mozilla.org/Buildbot/Talos#Page_Load_Tests PageLoader]
* measuring: responsiveness of web applications
* reporting: runs/
|}
|}


An svg-only number that measures SVG rendering performance, with animations or iterations of rendering. This is an ASAP test -- i.e. it iterates in unlimited frame-rate mode thus reflecting the maximum rendering throughput of each test. The reported value is the overall duration the sequence/animation took to complete. To turn on ASAP mode, we set these preferences:
==== Possible regression causes ====
preferences = {'layout.frame_rate': 0, 'docshell.event_starvation_delay_hint': 1}
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
 
=== Stylo gtest microbenchmarks ===
 
* contact: :bholley, :SimonSapin
* source: [https://dxr.mozilla.org/mozilla-central/source/layout/style/test/gtest]
* type: [https://wiki.mozilla.org/Buildbot/Talos/Tests#Microbench Microbench]
* reporting: intervals in ms (lower is better)
* data: each test is run and measured 5 times
* summarization: take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the 5 data points; [https://dxr.mozilla.org/mozilla-central/source/testing/gtest/mozilla/MozGTestBench.cpp#43-46 source: MozGTestBench.cpp]


{| class="mw-collapsible mw-collapsed wikitable"
! Example data
|-
| <pre>
0;hixie-001.xml;562;555;508;521;522;520;499;510;492;514;502;504;500;521;510;506;511;505;495;517;520;512;503;504;502
1;hixie-002.xml;510;613;536;530;536;522;498;505;500;504;498;529;498;509;493;512;501;506;504;499;496;505;508;511;503
2;hixie-003.xml;339;248;242;261;250;241;240;248;258;244;235;240;247;248;239;247;241;245;242;245;251;239;241;240;237
3;hixie-004.xml;709;540;538;536;540;536;552;539;535;535;543;533;536;535;545;537;537;537;537;539;538;535;536;538;536
4;hixie-005.xml;3096;3086;3003;3809;3213;3323;3143;3313;3192;3203;3225;3048;3069;3101;3189;3251;3172;3122;3266;3183;3159;3076;3014;3237;3100
5;hixie-006.xml;5586;5668;5565;5666;5668;5728;5886;5534;5484;5607;5678;5577;5745;5753;5532;5585;5506;5516;5648;5778;5894;5994;5794;5852;5810
6;hixie-007.xml;1823;1743;1739;1743;1744;1787;1802;1788;1782;1766;1787;1750;1748;1788;1766;1779;1767;1794;1758;1768;1781;1773;1765;1798;1805
</pre>
|}


==== Possible regression causes ====
Servo_StyleSheet_FromUTF8Bytes_Bench parses a sample stylesheet 20 times with Stylo’s CSS parser that is written in Rust. It starts from an in-memory UTF-8 string, so that I/O or UTF-16-to-UTF-8 conversion is not measured.
* Did you change the dimensions of the content area? Even a little? The tsvgx test seems to be sensitive to changes like this. See {{bug|1375479}}, for example. Usually, these sorts of "regressions" aren't real regressions - they just mean that we need to re-baseline our expectations from the test.


=== tsvg_static ===
Gecko_nsCSSParser_ParseSheet_Bench does the same with Gecko’s previous CSS parser that is written in C++, for comparison.
* contact: :jwatt, :jmaher, :dholbert, :neerja
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/svg_static/ svg_static]
* type: PageLoader
* data: we load the 5 svg pages 25 times, resulting in 5 sets of 25 data points
* summarization:
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] '''5''' data points, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 20; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l623 source: test.py]
** suite: [https://wiki.mozilla.org/Buildbot/Talos/Data#geometric_mean geometric mean] of the 5 subtest results.


{| cellspacing="1" cellpadding="1" border="1"
Servo_DeclarationBlock_SetPropertyById_Bench parses the string "10px" with Stylo’s CSS parser and sets it as the value of a property in a declaration block, a million times. This is similar to animations that are based on JavaScript code modifying Element.style instead of using CSS @keyframes.
|-
| Talos test name
| Description
|-
|tsvg_static
|An svg-only number that measures SVG rendering performance of some complex (but static) SVG content.
|}


{| class="mw-collapsible mw-collapsed wikitable"
Servo_DeclarationBlock_SetPropertyById_WithInitialSpace_Bench is the same, but with the string " 10px" with an initial space. That initial space is less typical of JS animations, but is almost always there in stylesheets or full declarations like "width: 10px". This microbenchmark was used to test the effect of some specific code changes. Regressions here may be acceptable if Servo_StyleSheet_FromUTF8Bytes_Bench is not affected.
! Example data
|-
| <pre>
0;gearflowers.svg;262;184;184;198;197;187;181;186;177;192;196;194;194;186;195;190;237;193;188;182;188;196;191;194;184
1;composite-scale.svg;69;52;48;49;57;51;52;87;52;49;49;51;58;53;64;57;49;65;67;58;53;59;56;68;50
2;composite-scale-opacity.svg;72;53;91;54;51;58;60;46;51;57;59;58;66;70;57;61;47;51;76;65;52;65;64;64;63
3;composite-scale-rotate.svg;70;76;89;62;62;78;57;77;79;82;74;56;61;79;73;64;75;74;81;82;76;58;77;61;62
4;composite-scale-rotate-opacity.svg;91;60;67;84;62;66;78;69;65;68;62;73;68;63;64;71;79;77;63;80;85;65;82;76;81
</pre>
|}


==== Possible regression causes ====
==== Possible regression causes ====
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.


=== tsvgr_opacity ===
=== tabpaint ===
* contact: :jwatt, :jmaher, :avih
* contact: :mconley
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/svg_opacity/svg_opacity.manifest]
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/tabpaint tabpaint]
* type: PageLoader
* bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1253382
* data: we load the 2 svg opacity pages 25 times, resulting in 2 sets of 25 data points
* type: Pageloader
* summarization:
* measuring:  
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] '''5''' data points, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 20; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l623 source: test.py]
** The time it takes to paint the content of a newly opened tab when the tab is opened from the parent (ex: by hitting Ctrl-T)
** suite: [https://wiki.mozilla.org/Buildbot/Talos/Data#geometric_mean geometric mean] of the 2 subtest results.
** The time it takes to paint the content of a newly opened tab when the tab is opened from content (ex: by clicking on a target="_blank" link)
 
* '''NOT''' measuring:
{| cellspacing="1" cellpadding="1" border="1"
** The time it takes to animate the tabs. That's the responsibility of the TART test. tabpaint is strictly concerned with the painting of the web content.
* data: we load the tabpaint trigger page 20 times, each run produces two values (the time it takes to paint content when opened from the parent, and the time it takes to paint content when opened from content), resulting in 2 sets of 20 data points.
** Example:
{| class="mw-collapsible mw-collapsed wikitable"
! Example data
|-
|-
| Talos test name
| <pre>
| Description
0;tabpaint-from-parent;105;76;66;64;64;69;65;63;70;68;64;60;65;63;54;61;64;67;61;64
|-
1; tabpaint-from-content;129;68;72;72;70;78;86;85;82;79;120;92;76;80;74;82;76;89;77;85
|tsvgr_opacity
</pre>
|[[#Row_Major_vs._Column_Major|Row Major]] and 25 cycles/page.
|}
|}


Renders many semi-transparent, partially overlapping SVG rectangles, and measures time to completion of this rendering.
* summarization:
 
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] data point, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 19 data points
Note that this test also tends to reflect changes in network efficiency and navigation bar rendering issues:
** suite: geometric_mean(subtests)
 
* Most of the page load tests measure from before the location is changed, until onload + mozafterpaint, therefore any changes in chrome performance from the location change, or network performance (the pages load from a local web server) would affect page load times. SVG opacity is rather quick by itself, so any such chrome/network/etc performance changes would affect this test more than other page load tests (relatively, in percentages).
 
{| class="mw-collapsible mw-collapsed wikitable"
! Example data
|-
| <pre>
0;big-optimizable-group-opacity-2500.svg;170;171;205;249;249;244;192;252;192;431;182;250;189;249;151;168;209;194;247;250;193;250;255;247;247
1;small-group-opacity-2500.svg;585;436;387;441;512;438;440;380;443;391;450;386;459;383;445;388;450;436;485;443;383;438;528;444;441
</pre>
|}


==== Possible regression causes ====
==== Possible regression causes ====
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.


=== tpaint ===
=== TART/CART ===
* contact: :jimm, :jmaher
* contact: :avih, :jmaher, :MattN
* source: [[https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/startup_test/tpaint.html tpaint-window.html]]
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/tart tart]
* type: Startup
* type: PageLoader
* data: we load the tpaint test window  20 times, resulting in 1 set of 20 data points.
* measuring: Desktop Firefox UI animation speed and smoothness
* reporting: intervals in ms (lower is better) - see below for details
* data: there are 30 reported subtests from TART which we load 25 times, resulting in 30 sets of 25 data points.
* summarization:
* summarization:
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] '''5''' data points, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 15; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l190 source: test.py]
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] data point, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 24 data points; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l305 source: test.py]
** suite: identical to subtest
** suite: [https://wiki.mozilla.org/Buildbot/Talos/Data#geometric_mean geometric mean] of the 30 subtest results.


{| cellspacing="1" cellpadding="1" border="1"
TART is the '''Tab Animation Regression Test''' and CART is the '''Customize Animation Regression Test'''.
|-
|Talos test name
|Description
|-
|tpaint
|twinopen but measuring the time after we receive the [[Buildbot/Talos/Tests#Paint_Tests|MozAfterPaint and OnLoad event]].
|}


Tests the amount of time it takes the open a new window. This test does not include startup time. Multiple test windows are opened in succession, results reported are the average amount of time required to create and display a window in the running instance of the browser. (Measures ctrl-n performance.)
TART tests tab animation on these cases:
 
* Simple: single new tab of about:blank open/close without affecting (shrinking/expanding) other tabs.
{| class="mw-collapsible mw-collapsed wikitable"
* icon: same as above with favicons and long title instead of about:blank.
* Newtab: newtab open with thumbnails preview - without affecting other tabs, with and without preload.
* Fade: opens a tab, then measures fadeout/fadein (tab animation without the overhead of opening/closing a tab).
** Case 1 is tested with DPI scaling of 1.
** Case 2 is tested with DPI scaling of 1.0 and 2.0.
** Case 3 is tested with the default scaling of the test system.
** Case 4 is tested with DPI scaling of 2.0 with the "icon" tab (favicon and long title).
** Each animation produces 3 test results:
*** error: difference between the designated duration and the actual completion duration from the trigger.
*** half: average frame interval over the 2nd half of the animation.
*** all: average frame interval over all recorded intervals.
*** And the run logs also include the explicit intervals from which these 3 values were derived.
 
CART uses the same framework to measure performance of the Australize "customize" animation (for entering the toolbar/menu customization view). Subtests include:
* Customize-enter animation (full and css-animation-only part).
* Customize-exit animation
 
TART/CART can be used as a stand-alone addon:
* Set the browser to ASAP mode (preferences layout.frame_rate=0, docshell.event_starvation_delay_hint=1). This makes the browser refresh the screen as fast as possible instead of limiting it to 60hz, thus allows higher resolution measurements. Requires restart to take effect.
* [https://hg.mozilla.org/mozilla-central/raw-file/tip/testing/talos/talos/generate-tart-xpi.html Install the latest addon xpi], or zip the addon dir of the source code, rename the extension to xpi and install.
* Visit chrome://tart/content/tart.html
* Select subtests to run. By default the selected tests are all the TART tests. CART is the "Customize" test.
 
{| class="mw-collapsible mw-collapsed wikitable"
! Example data
! Example data
|-
|-
| <pre>
| <pre>
[209.219, 222.180, 225.299, 225.970, 228.090, 229.450, 230.625, 236.315, 239.804, 242.795, 244.5, 244.770, 250.524, 251.785, 253.074, 255.349, 264.729, 266.014, 269.399, 326.190]
0;simple-open-DPI1.half.TART;2.35;2.42;2.63;2.47;2.71;2.38;2.37;2.41;2.48;2.70;2.44;2.41;2.51;2.43;2.41;2.56;2.76;2.49;2.36;2.40;2.70;2.53;2.35;2.46;2.47
</pre>
1;simple-open-DPI1.all.TART;2.80;2.95;3.12;2.92;3.46;2.87;2.92;2.99;2.89;3.24;2.94;2.95;3.25;2.92;3.02;3.00;3.21;3.31;2.84;2.87;3.10;3.13;3.10;2.94;2.95
|}
2;simple-open-DPI1.error.TART;33.60;36.33;35.93;35.97;38.17;34.77;36.00;35.01;36.25;36.22;35.24;35.76;36.64;36.31;34.74;34.40;34.34;41.48;35.04;34.83;34.27;34.04;34.37;35.22;36.52
 
3;simple-close-DPI1.half.TART;1.95;1.88;1.91;1.94;2.00;1.97;1.88;1.76;1.84;2.18;1.99;1.83;2.04;1.93;1.81;1.77;1.79;1.90;1.82;1.84;1.78;1.75;1.76;1.89;1.81
==== Possible regression causes ====
4;simple-close-DPI1.all.TART;2.19;2.08;2.07;2.32;2.65;2.32;2.26;1.96;2.02;2.26;2.05;2.16;2.19;2.11;2.04;1.98;2.05;2.02;2.01;2.11;1.97;1.97;2.05;2.01;2.12
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
5;simple-close-DPI1.error.TART;21.35;23.87;22.60;22.02;22.97;22.35;22.15;22.79;21.81;21.90;22.26;22.58;23.15;22.43;22.76;23.36;21.86;22.70;22.96;22.70;22.28;22.03;21.78;22.33;22.23
 
6;icon-open-DPI1.half.TART;2.42;2.33;2.50;2.58;2.36;2.51;2.60;2.35;2.52;2.51;2.59;2.34;3.29;2.63;2.46;2.57;2.53;2.50;2.39;2.51;2.44;2.66;2.72;2.36;2.52
=== tscrollx ===
7;icon-open-DPI1.all.TART;3.12;2.94;3.42;3.23;3.10;3.21;3.33;3.14;3.24;3.32;3.46;2.90;3.65;3.19;3.27;3.47;3.32;3.13;2.95;3.23;3.21;3.33;3.47;3.15;3.32
* contact: :jrmuizel, :jmaher, :avih
8;icon-open-DPI1.error.TART;38.39;37.96;37.03;38.85;37.03;37.17;37.19;37.56;36.67;36.33;36.89;36.85;37.54;38.46;35.38;37.52;36.68;36.48;36.03;35.71;37.12;37.08;37.74;38.09;35.85
* source: [[https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/scroll scroll.manifest]]
9;icon-close-DPI1.half.TART;1.94;1.93;1.79;1.89;1.83;1.83;1.90;1.89;1.75;1.76;1.77;1.74;1.81;1.86;1.95;1.99;1.73;1.83;1.97;1.80;1.94;1.84;2.01;1.88;2.03
* type: PageLoader
10;icon-close-DPI1.all.TART;2.14;2.14;1.98;2.03;2.02;2.25;2.29;2.13;1.97;2.01;1.94;2.01;1.99;2.05;2.11;2.09;2.02;2.02;2.12;2.02;2.20;2.11;2.19;2.07;2.27
* measuring: Scroll performance
11;icon-close-DPI1.error.TART;24.51;25.03;25.17;24.54;23.86;23.70;24.02;23.61;24.10;24.53;23.92;23.75;23.73;23.78;23.42;25.40;24.21;24.55;23.96;24.96;24.41;24.96;24.16;24.20;23.65
* reporting: Average frame interval (1/FPS). Lower is better.
12;icon-open-DPI2.half.TART;2.60;2.60;2.60;2.53;2.51;2.53;2.59;2.43;2.66;2.60;2.47;2.61;2.64;2.43;2.49;2.63;2.61;2.60;2.52;3.06;2.65;2.74;2.69;2.44;2.43
* data: we load 6 pages 25 times each, collecting 6 sets of 25 data points
13;icon-open-DPI2.all.TART;3.45;3.22;3.38;3.47;3.10;3.31;3.47;3.13;3.37;3.14;3.28;3.20;3.40;3.15;3.14;3.23;3.41;3.16;3.26;3.55;3.29;3.49;3.44;3.11;3.22
* summarization:
14;icon-open-DPI2.error.TART;40.20;37.86;37.53;41.46;37.03;38.77;37.48;36.97;37.28;37.72;36.09;36.71;38.89;38.21;37.37;38.91;36.79;36.10;37.60;36.99;37.56;35.76;38.92;37.46;37.52
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] data point, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 24; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l623 source: test.py]
15;icon-close-DPI2.half.TART;2.27;1.97;1.83;1.86;2.08;1.88;1.80;1.90;1.77;1.89;2.06;1.89;1.89;1.86;2.01;1.79;1.78;1.83;1.89;1.85;1.74;1.82;1.84;1.81;1.79
** suite: [https://wiki.mozilla.org/Buildbot/Talos/Data#geometric_mean geometric mean] of the 6 subtest results.
16;icon-close-DPI2.all.TART;2.33;2.13;2.18;2.03;2.33;2.03;1.95;2.06;1.96;2.13;2.25;2.10;2.13;2.03;2.18;2.00;2.05;2.01;2.08;2.05;1.96;2.04;2.10;2.04;2.08
 
17;icon-close-DPI2.error.TART;22.99;23.02;24.32;23.58;23.05;23.34;22.92;23.22;22.90;23.33;23.33;23.05;22.80;23.45;24.05;22.39;22.14;22.97;22.85;22.13;22.89;22.98;23.69;22.99;23.08
{| cellspacing="1" cellpadding="1" border="1"
18;iconFade-close-DPI2.half.TART;2.14;1.84;1.78;1.84;1.66;2.07;1.81;3.82;1.68;1.85;1.62;2.54;2.06;1.85;2.17;1.80;1.71;1.67;2.11;1.73;2.94;2.14;1.93;1.72;2.05
|-
19;iconFade-close-DPI2.all.TART;2.17;1.76;1.80;1.89;1.70;1.93;1.80;3.38;1.78;1.90;1.70;2.50;1.94;1.81;2.29;1.82;1.79;1.76;2.23;1.80;2.85;2.06;1.84;1.83;2.09
| Talos test name
20;iconFade-close-DPI2.error.TART;4.67;4.11;3.69;4.51;4.46;3.88;4.54;3.68;4.56;3.82;4.32;4.87;4.42;3.72;3.72;4.54;4.93;4.46;4.64;3.39;4.09;3.28;3.58;4.11;3.80
| Description
21;iconFade-open-DPI2.half.TART;2.37;2.61;2.37;2.62;2.54;2.84;2.57;2.44;4.33;2.57;2.59;2.67;2.58;2.48;2.38;2.39;2.50;2.39;2.50;2.57;2.52;2.55;2.47;2.69;2.41
|-
22;iconFade-open-DPI2.all.TART;2.45;2.64;2.39;2.60;2.57;2.60;2.61;2.59;3.14;2.55;2.54;2.66;2.57;2.48;2.47;2.46;2.55;2.45;2.51;2.61;2.54;2.58;2.50;2.54;2.40
|tscrollx
23;iconFade-open-DPI2.error.TART;3.64;4.67;4.31;5.79;6.43;3.64;4.82;8.68;5.78;4.38;3.80;3.98;4.64;653.63;4.63;3.76;4.23;5.01;5.48;4.99;3.48;5.10;5.02;6.14;5.58
|[https://groups.google.com/d/topic/mozilla.dev.platform/RICw5SJhNMo/discussion Replacing tscroll,tsvg with tscrollx,tsvgx]
24;newtab-open-preload-no.half.TART;5.02;2.90;3.06;3.03;2.94;2.94;3.08;3.12;3.60;3.19;2.82;2.96;3.67;7.85;2.79;3.12;3.18;2.92;2.86;2.96;2.96;3.00;2.90;2.97;2.94
25;newtab-open-preload-no.all.TART;7.11;4.66;5.03;4.68;4.50;4.58;4.76;4.76;5.67;4.96;4.36;4.51;5.21;11.16;4.38;4.69;4.77;4.45;4.45;4.70;4.51;4.61;4.54;4.69;4.60
26;newtab-open-preload-no.error.TART;40.82;40.85;37.38;37.40;36.30;36.47;36.89;37.63;37.12;38.65;36.73;36.95;36.11;38.59;37.39;37.77;37.93;37.54;37.46;38.29;36.58;38.25;38.32;37.92;36.93
27;newtab-open-preload-yes.half.TART;3.14;2.96;2.97;8.37;2.98;3.00;2.96;3.05;3.12;3.48;3.07;3.23;3.05;2.88;2.92;3.06;2.90;3.01;3.19;2.90;3.18;3.11;3.04;3.16;3.21
28;newtab-open-preload-yes.all.TART;5.10;4.60;4.63;8.94;5.01;4.69;4.63;4.67;4.93;5.43;4.78;5.12;4.77;4.65;4.50;4.78;4.75;4.63;4.76;4.45;4.86;4.88;4.69;4.86;4.92
29;newtab-open-preload-yes.error.TART;35.90;37.24;38.57;40.60;36.04;38.12;38.78;36.73;36.91;36.69;38.12;36.69;37.79;35.80;36.11;38.01;36.59;38.85;37.14;37.30;38.02;38.95;37.64;37.86;36.43
</pre>
|}
|}


==== Possible regression causes ====
* CART
** Regressions might be reported for patches that add new buttons to the menu panel by default, since this can add new rows to the panel and make it scrollable, which increases the amount of work to do during the transition. See {{bug|1230671}} for details.


This test scrolls several pages where each represent a different known "hard" case to scroll (* needinfo), and measures the average frames interval (1/FPS) on each. The ASAP test (tscrollx) iterates in unlimited frame-rate mode thus reflecting the maximum scroll throughput per page. To turn on ASAP mode, we set these preferences:
=== TestStandardURL ===
preferences = {'layout.frame_rate': 0, 'docshell.event_starvation_delay_hint': 1}
 
* contact: :valentin
* source:
* type:
* reporting:
* data:
* summarization:
 
TestStandardURL.Perf - Runs 10K iterations on the most important getters and setters of nsStandardURL
 
TestStandardURL.NormalizePerf - Runs 20K x 5 iterations on nsStandardURL::NormalizeIPv4 on input that are valid IPv4 strings


{| class="mw-collapsible mw-collapsed wikitable"
TestStandardURL.NormalizePerfFails - Runs 20K x 5 iterations on nsStandardURL::NormalizeIPv4 on input that is not valid IPv4
! Example data
|-
| <pre>
0;tiled.html;5.41;5.57;5.34;5.64;5.53;5.48;5.44;5.49;5.50;5.50;5.49;5.66;5.50;5.37;5.57;5.54;5.46;5.31;5.41;5.57;5.50;5.52;5.71;5.31;5.44
fixed.html;10.404609053497941;10.47;10.66;10.45;10.73;10.79;10.64;10.64;10.82;10.43;10.92;10.47;10.47;10.64;10.74;10.67;10.40;10.83;10.77;10.54;10.38;10.70;10.44;10.38;10.56
downscale.html;5.493209876543211;5.27;5.50;5.50;5.51;5.46;5.58;5.58;5.51;5.49;5.49;5.47;9.09;5.56;5.61;5.50;5.47;5.59;5.47;5.49;5.60;5.61;5.58;5.40;5.43
downscale.html;10.676522633744854;10.82;10.79;10.41;10.75;10.91;10.52;10.61;10.50;10.55;10.80;10.17;10.68;10.41;10.42;10.41;10.58;10.28;10.56;10.66;10.68;10.47;10.60;10.61;10.26
4;iframe.svg;13.82;14.87;14.78;14.35;14.73;14.50;14.15;14.46;14.80;14.48;15.10;14.93;14.77;14.52;14.08;15.01;14.58;14.52;15.23;14.35;14.72;14.28;14.30;14.27;14.96
5;reader.htm;10.72;10.62;10.23;10.48;10.42;10.64;10.40;10.40;10.14;10.60;10.51;10.36;10.57;10.41;10.52;10.75;10.19;10.72;10.44;9.75;10.49;10.07;10.54;10.46;10.44
</pre>
|}


==== Possible regression causes ====
==== Possible regression causes ====
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.


=== tresize ===
=== tp5 ===
* contact: :jimm, :jmaher, :avih
* contact: :jmaher
* source: [[https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/startup_test/tresize/addon/content/tresize-test.html tresize-test.html]]
* source: [https://people.mozilla.org/~jmaher/taloszips/zips/tp5n.zip tp5n.zip]
* type: StartupTest
* type: [https://wiki.mozilla.org/Buildbot/Talos#Page_Load_Tests PageLoader]
* measuring: Time to do XUL resize, in ms (lower is better).
* data: we load each of the 51 tp5o pages 25 times, resulting in 51 sets of 25 data points.
* data: we run the tresize test page 20 times, resulting in 1 set of 20 data points.
* '''To run it locally''', you'd need [[#tp5n_pages_set|tp5n.zip]].
* summarization:
* summarization:
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] '''5''' data points, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 15 data points; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l205 source: test.py]
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] '''5''' data points, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 20; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l449 source: test.py]
** suite: same as subtest result
** suite: [https://wiki.mozilla.org/Buildbot/Talos/Data#geometric_mean geometric mean] of the 51 subtest results.


A purer form of paint measurement than tpaint. This test opens a single window positioned at 10,10 and sized to 300,300, then resizes the window outward |max| times measuring the amount of time it takes to repaint each resize. Dumps the resulting dataset and average to stdout or logfile.
{| cellspacing="1" cellpadding="1" border="1"
|-
| Talos test name
| Description
|-
|tp5o
|tp5 with limited pageset (48 pages as others have too much noise)
|}


In {{bug|1102479}} tresize was rewritten to work in e10s mode which involved a full rewrite of the test.
Tests the time it takes Firefox to load the [https://people.mozilla.org/~jmaher/taloszips/zips/tp5n.zip tp5 web page test set]. The web set was culled from the Alexa top 500 April 8th, 2011 and consists of 51 pages.  


To run resize locally without talos, please [https://hg.mozilla.org/mozilla-central/raw-file/tip/testing/talos/talos/generate-tresize-xpi.html install the addon] to run the test locally.
Here are the broad steps we use to create the test set:
# Take the Alexa top 500 sites list
# Remove all sites with questionable or explicit content
# Remove duplicate site (for ex. many Google search front pages)
# Manually select to keep interesting pages (such as pages in different locales)
# Select a more representative page from any site presenting a simple search/login/etc. page
# Deal with Windows 255 char limit for cached pages
# Limit test set to top 100 pages
 
Note that the above steps did not eliminate all outside network access so we had to take further action to scrub all the pages so that there are 0 outside network accesses (this is done so that the tp test is as deterministic measurement of our rendering/layout/paint process as possible).


{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
Line 1,015: Line 1,015:
|-
|-
| <pre>
| <pre>
[23.2565333333333, 23.763383333333362, 22.58369999999999, 22.802766666666653, 22.304050000000025, 23.010383333333326, 22.865466666666677, 24.233716666666705, 24.110983333333365, 22.21390000000004, 23.910333333333316, 23.409816666666647, 19.873049999999992, 21.103966666666686, 20.389749999999978, 20.777349999999984, 20.326283333333365, 22.341616666666667, 20.29813333333336, 20.769600000000104]
0;163.com/www.163.com/index.html;1035;512;542;519;505;514;551;513;554;793;487;528;528;498;503;530;527;490;521;535;521;496;498;564;520
</pre>
1;56.com/www.56.com/index.html;1081;583;580;577;597;580;623;558;572;592;598;580;564;583;596;600;579;580;566;573;566;581;571;600;586
|}
2;aljazeera.net/aljazeera.net/portal.html;688;347;401;382;346;362;347;372;337;345;365;337;380;338;355;360;356;366;367;352;350;366;346;375;382
 
3;amazon.com/www.amazon.com/Kindle-Wireless-Reader-Wifi-Graphite/dp/B002Y27P3M/507846.html;1392;878;901;852;886;867;877;864;862;877;866;888;3308;870;863;869;873;850;851;850;857;873;869;860;855
==== Possible regression causes ====
4;bbc.co.uk/www.bbc.co.uk/news/index.html;455;271;272;271;279;289;276;285;277;291;281;286;278;286;274;285;276;285;287;286;276;288;279;272;278
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
5;beatonna.livejournal.com/beatonna.livejournal.com/index.html;290;123;123;129;120;121;124;125;119;125;120;150;121;147;121;121;113;121;119;122;117;112;127;117;139
 
6;bild.de/www.bild.de/index.html;1314;937;946;931;922;918;920;937;934;930;947;928;936;933;933;928;930;941;951;946;947;938;925;939;938
=== xperf ===
7;cgi.ebay.com/cgi.ebay.com/ALL-NEW-KINDLE-3-eBOOK-WIRELESS-READING-DEVICE-W-WIFI-/130496077314@pt=LH_DefaultDomain_0&hash=item1e622c1e02.html;495;324;330;328;321;308;315;308;321;313;327;330;317;339;333;322;312;370;336;327;310;312;312;355;330
* contact: :aklotz, :jmaher
8;chemistry.about.com/chemistry.about.com/index.html;238;156;156;154;158;161;152;151;152;167;179;152;154;156;161;161;157;167;151;167;154;149;178;153;160
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/xtalos xperf instrumentation]
9;chinaz.com/chinaz.com/index.html;347;223;255;234;245;233;264;234;244;228;260;224;258;223;280;220;243;225;251;226;258;232;258;232;247
* type: Pageloader (tp5n) / Startup
10;cnn.com/www.cnn.com/index.html;551;384;436;394;391;375;371;407;371;374;398;372;368;388;376;380;386;377;363;383;384;370;388;381;374
* measuring: IO counters from windows (currently, only startup IO is in scope)
11;dailymail.co.uk/www.dailymail.co.uk/ushome/index.html;984;606;551;561;545;542;576;564;543;560;566;557;561;544;545;576;548;539;568;567;557;560;545;544;578
* reporting: Summary of read/write counters for disk, network (lower is better)
12;dailymotion.com/www.dailymotion.com/us.html;473;271;286;272;285;288;290;290;280;268;286;269;287;275;289;282;293;287;304;261;289;284;281;277;286
 
13;digg.com/digg.com/news/story/New_logo_for_Mozilla_Firefox_browser.html;410;321;304;303;322;300;319;321;320;306;323;313;312;305;312;338;317;338;301;325;297;302;309;305;300
Talos will turn orange for 'x' jobs on windows 7 if your changeset accesses files which are not predefined in the [[https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/xtalos/xperf_whitelist.json whitelist]] during startup; specifically, before the "[https://hg.mozilla.org/mozilla-central/file/0eebc33d8593/toolkit/components/startup/nsAppStartup.cpp#l631 sessionstore-windows-restored]" Firefox event. If your job turns orange, you will see a list of files in Treeherder (or in the log file) which have been accessed unexpectedly (similar to this):
14;ezinearticles.com/ezinearticles.com/index.html@Migraine-Ocular---The-Eye-Migraines&id=4684133.html;234;177;163;163;186;176;185;175;167;156;162;199;163;190;173;181;175;178;165;159;182;170;183;169;158
* TEST-UNEXPECTED-FAIL : xperf: File '{profile}\secmod.db' was accessed and we were not expecting it. DiskReadCount: 6, DiskWriteCount: 0, DiskReadBytes: 16904, DiskWriteBytes: 0
15;globo.com/www.globo.com/index.html;684;468;466;485;482;445;433;467;467;450;487;466;440;484;444;451;511;443;429;469;468;430;485;459;447
* TEST-UNEXPECTED-FAIL : xperf: File '{profile}\cert8.db' was accessed and we were not expecting it. DiskReadCount: 4, DiskWriteCount: 0, DiskReadBytes: 33288, DiskWriteBytes: 0
16;google.com/www.google.com/search@q=mozilla.html;150;100;102;101;97;104;99;116;107;100;98;137;102;102;99;106;98;112;100;102;105;104;107;96;100
* TEST-UNEXPECTED-FAIL : xperf: File 'c:\$logfile' was accessed and we were not expecting it. DiskReadCount: 0, DiskWriteCount: 2, DiskReadBytes: 0, DiskWriteBytes: 32768 TEST-UNEXPECTED-FAIL : xperf: File '{profile}\secmod.db' was accessed and we were not expecting it. DiskReadCount: 6, DiskWriteCount: 0, DiskReadBytes: 16904, DiskWriteBytes: 0
17;goo.ne.jp/goo.ne.jp/index.html;328;125;132;132;143;121;122;120;132;145;166;139;144;125;128;152;128;145;130;132;154;126;142;133;139
* TEST-UNEXPECTED-FAIL : xperf: File '{profile}\cert8.db' was accessed and we were not expecting it. DiskReadCount: 4, DiskWriteCount: 0, DiskReadBytes: 33288, DiskWriteBytes: 0
18;guardian.co.uk/www.guardian.co.uk/index.html;462;311;296;322;309;305;303;288;301;308;301;304;319;309;295;305;294;308;304;322;310;314;302;303;292
* TEST-UNEXPECTED-FAIL : xperf: File 'c:\$logfile' was accessed and we were not expecting it. DiskReadCount: 0, DiskWriteCount: 2, DiskReadBytes: 0, DiskWriteBytes: 32768
19;homeway.com.cn/www.hexun.com/index.html;584;456;396;357;417;374;376;406;363;392;400;378;378;402;390;373;398;393;366;385;383;361;418;386;351
 
20;huffingtonpost.com/www.huffingtonpost.com/index.html;811;609;575;596;568;585;589;571;568;600;571;588;585;570;574;616;576;564;598;594;589;590;572;572;612
In the case that these files are expected to be accessed during startup by your changeset, then we can add them to the [https://bugzilla.mozilla.org/enter_bug.cgi?product=Testing&component=Talos whitelist].
21;ifeng.com/ifeng.com/index.html;829;438;478;462;448;465;469;470;429;463;465;432;482;444;476;453;460;476;461;484;467;510;447;477;443
 
22;imdb.com/www.imdb.com/title/tt1099212/index.html;476;337;358;332;414;379;344;420;354;363;387;345;358;371;341;385;359;379;353;349;392;349;358;378;347
Xperf runs tp5 while collecting xperf metrics for disk IO and network IO. The providers we listen for are:
23;imgur.com/imgur.com/gallery/index.html;419;205;224;231;207;222;206;231;204;219;209;210;210;208;202;215;203;210;203;212;218;219;202;224;230
* [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/xperf.config#l10 'PROC_THREAD', 'LOADER', 'HARD_FAULTS', 'FILENAME', 'FILE_IO', 'FILE_IO_INIT']
24;indiatimes.com/www.indiatimes.com/index.html;530;339;348;384;376;381;353;350;403;333;356;393;350;328;393;329;389;346;394;349;382;332;409;327;354
 
25;mail.ru/mail.ru/index.html;500;256;308;251;271;270;270;246;273;252;279;249;301;252;251;256;271;246;267;254;265;248;277;247;275
The values we collect during stackwalk are:
26;mashable.com/mashable.com/index.html;699;497;439;508;440;448;512;446;431;500;445;427;495;455;458;499;440;432;522;443;447;488;445;461;489
* [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/xperf.config#l11 'FileRead', 'FileWrite', 'FileFlush']
27;media.photobucket.com/media.photobucket.com/image/funny%20gif/findstuff22/Best%20Images/Funny/funny-gif1.jpg@o=1.html;294;203;195;189;213;186;195;186;204;188;190;220;204;202;195;204;192;204;191;187;204;199;191;192;211
 
28;myspace.com/www.myspace.com/albumart.html;595;446;455;420;433;425;416;429;452;411;435;439;389;434;418;402;422;426;396;438;423;391;434;438;395
==== Possible regression causes ====
29;naver.com/www.naver.com/index.html;626;368;338;386;342;377;371;352;379;348;362;357;359;354;386;338;394;330;326;372;345;392;336;336;368
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
30;noimpactman.typepad.com/noimpactman.typepad.com/index.html;431;333;288;292;285;313;277;289;282;292;276;293;270;294;289;281;275;302;285;290;280;285;278;284;283
 
31;page.renren.com/page.renren.com/index.html;373;232;228;228;213;227;224;227;226;216;234;226;230;212;213;221;224;230;212;218;217;221;212;222;230
 
32;people.com.cn/people.com.cn/index.html;579;318;305;339;307;341;325;326;307;309;315;314;318;317;321;309;307;299;312;313;305;326;318;384;310
=== speedometer ===
33;rakuten.co.jp/www.rakuten.co.jp/index.html;717;385;371;388;381;348;394;358;396;368;343;386;348;388;393;388;360;339;398;357;392;378;395;386;367
* contact: :jmaher, :rwood
34;reddit.com/www.reddit.com/index.html;340;254;248;255;241;241;248;275;251;250;250;252;243;274;240;269;254;249;242;257;271;253;243;278;252
* source: [[https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/speedometer/speedometer.manifest speedometer.manifest]]
35;reuters.com/www.reuters.com/index.html;513;404;355;358;379;343;354;385;379;354;418;363;342;412;355;351;402;375;354;400;362;355;380;373;336
* type: [https://wiki.mozilla.org/Buildbot/Talos#Page_Load_Tests PageLoader]
36;slideshare.net/www.slideshare.net/jameswillamor/lolcats-in-popular-culture-a-historical-perspective.html;397;279;270;283;285;283;291;286;289;284;275;281;288;284;280;279;290;301;290;270;292;282;289;267;278
* measuring: responsiveness of web applications
37;sohu.com/www.sohu.com/index.html;727;414;479;414;431;485;418;440;488;431;432;464;442;407;488;435;416;465;445;414;480;416;403;463;429
* reporting: runs/minute score
38;spiegel.de/www.spiegel.de/index.html;543;430;391;380;440;387;375;430;380;397;415;383;434;420;384;399;421;392;384;418;388;380;427;403;392
* data: there are 16 subtests in Speedometer; each of these are made up of 9 internal benchmarks.
39;stackoverflow.com/stackoverflow.com/questions/184618/what-is-the-best-comment-in-source-code-you-have-ever-encountered.html;503;377;356;438;370;388;409;367;366;407;375;363;393;360;363;396;376;391;426;363;378;408;400;359;408
* summarization:
40;store.apple.com/store.apple.com/us@mco=Nzc1MjMwNA.html;488;327;344;343;333;329;328;348;361;342;362;332;389;333;382;331;382;343;405;343;326;325;329;323;340
** subtest: For all of the 16 subtests, we collect the sum of all their internal benchmark results.
41;thepiratebay.org/thepiratebay.org/top/201.html;412;274;317;260;256;269;266;261;258;289;245;284;256;277;251;302;276;307;268;268;247;285;260;271;257
** suite: geometric [https://wiki.mozilla.org/Buildbot/Talos/Data#mean mean] of the 16 sums
42;tudou.com/www.tudou.com/index.html;522;304;281;283;287;285;288;307;279;288;282;303;292;288;290;287;311;271;279;274;294;272;290;269;290
43;uol.com.br/www.uol.com.br/index.html;668;387;450;411;395;452;386;431;452;394;385;436;413;414;440;401;412;439;408;430;426;415;382;433;387
44;w3.org/www.w3.org/standards/webdesign/htmlcss.html;225;143;129;151;181;141;147;137;159;179;142;153;136;139;191;140;151;143;141;181;140;154;142;143;183
45;wsj.com/online.wsj.com/home-page.html;634;466;512;463;467;507;461;432;492;494;491;507;466;477;495;455;451;495;461;463;494;468;444;497;442
46;xinhuanet.com/xinhuanet.com/index.html;991;663;727;659;647;639;644;656;666;658;670;648;676;653;652;654;641;636;664;668;655;657;646;674;633
47;xunlei.com/xunlei.com/index.html;802;625;624;636;641;652;659;642;623;635;628;606;667;688;683;694;672;640;628;620;653;626;633;654;643
48;yelp.com/www.yelp.com/biz/alexanders-steakhouse-cupertino.html;752;475;502;472;477;512;489;478;501;472;454;517;487;474;521;467;450;513;491;464;536;507;455;511;481
49;youku.com/www.youku.com/index.html;844;448;498;441;417;497;478;439;467;436;447;465;438;461;466;446;452;496;457;446;486;449;467;499;442
50;youtube.com/www.youtube.com/music.html;443;338;253;289;238;296;254;290;242;302;237;290;253;305;253;293;251;311;244;293;255;291;246;316;249
</pre>
|}


This is the [http://browserbench.org/Speedometer/ Speedometer] javascript benchmark taken verbatim and slightly modified to fit into our pageloader extension and talos harness.
==== File IO ====
 
===== Possible regression causes =====
* '''nonmain_startup_fileio opt (with or without e10s) windows7-32''' – {{bug|1274018}} This test seems to consistently report a higher result for mozilla-central compared to Try even for an identical revision due to extension signing checks. In other words, if you are comparing Try and Mozilla-Central you may see a false-positive regression on perfherder. Graphs: [https://treeherder.mozilla.org/perf.html#/graphs?timerange=604800&series=%5Bmozilla-central,e5f5eaa174ef22fdd6b6e150e8c450aa827c2ff6,1,1%5D&series=%5Btry,e5f5eaa174ef22fdd6b6e150e8c450aa827c2ff6,1,1%5D non-e10s] [https://treeherder.mozilla.org/perf.html#/graphs?series=%5B%22mozilla-central%22,%222f3af3833d55ff371ecf01c41aeee1939ef3a782%22,1,1%5D&series=%5B%22try%22,%222f3af3833d55ff371ecf01c41aeee1939ef3a782%22,1,1%5D&timerange=604800 e10s]
 
==== Private Bytes ====
 
A memory metric tracked during tp4 test runs. This metric is sampled every 20 seconds.
 
For windows, a [https://technet.microsoft.com/en-us/library/cc780836%28WS.10%29.aspx description from Microsoft TechNet].
 
===== Possible regression causes =====
* Changing JavaScript GC policies or tuning parameters.
* If you fix a regression for this test and have some tips to share, this is a good place for them.


{| class="mw-collapsible mw-collapsed wikitable"
==== RSS (Resident Set Size)====
! Example data
 
|-
A memory metric tracked during tp5 test runs. This metric is sampled every 20 seconds. This metric is collected on linux/mac only.
| <pre>
 
0;VanillaJS-TodoMVC/Adding100Items/Sync;153.055;150.4950000000008;149.16500000000087;150.8000000000029;150.92499999998836;147.53000000000003;154.8199999999997;156.20000000000437;147.0749999999971;150.47500000000582;151.63;158.02000000000044;145.54499999999825;156.3050000000003;146.7100000000064;147.505;148.5699999999979;148.9350000000013;147.61000000000058;146.41999999999825;149.64;151.875;189.75500000000466;147.04499999999825;151.0350000000035
[https://en.wikipedia.org/wiki/Resident_set_size Description from wikipedia].
1;VanillaJS-TodoMVC/Adding100Items/Async;29.99000000000001;17.80500000000029;27.924999999995634;44.51000000000204;23.229999999995925;33.84500000000003;22.600000000000364;60.93999999999505;45.669999999998254;17.235000000000582;22.125;54.36499999999796;69.95000000000437;16.56500000000233;16.52499999999418;26.65500000000003;31.31000000000131;40.7300000000032;18.455000000001746;34.945000000006985;25.029999999999973;28.229999999999563;20.709999999999127;15.940000000002328;15.960000000006403
 
2;VanillaJS-TodoMVC/Adding100Items;183.04500000000002;168.3000000000011;177.0899999999965;195.31000000000495;174.15499999998428;181.37500000000006;177.42000000000007;217.13999999999942;192.74499999999534;167.7100000000064;173.755;212.3849999999984;215.49500000000262;172.87000000000262;163.23500000000058;174.16000000000003;179.8799999999992;189.6650000000045;166.06500000000233;181.36500000000524;174.66999999999996;180.10499999999956;210.46500000000378;162.98500000000058;166.9950000000099
===== Possible regression causes =====
3;VanillaJS-TodoMVC/CompletingAllItems/Sync;50.01500000000004;50.154999999998836;45.95499999999447;49.95499999999447;49.345000000001164;46.67500000000001;48.17000000000007;50.075000000004366;49.6050000000032;47.5;49.20999999999998;49.86499999999796;45.79000000000087;48.45999999999913;47.27499999999418;46.964999999999975;51.279999999998836;56.42000000000553;47.49000000000524;47.09000000001106;47.66999999999996;46.97999999999956;150.63000000000466;46.42500000000291;46.06500000000233
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
4;VanillaJS-TodoMVC/CompletingAllItems/Async;15.05499999999995;20.144999999998618;8.50499999999738;14.57499999999709;16.014999999999418;20.524999999999977;42.18000000000029;16.62000000000262;16.735000000000582;18.305000000007567;18.670000000000016;10.06499999999869;15.57999999999447;20.24500000000262;25.460000000006403;20.364999999999952;21.18500000000131;16.310000000004948;21.584999999999127;10.729999999995925;27.314999999999998;28.56499999999869;24.595000000001164;18.9800000000032;14.94999999999709
 
5;VanillaJS-TodoMVC/CompletingAllItems;65.07;70.29999999999745;54.45999999999185;64.52999999999156;65.36000000000058;67.19999999999999;90.35000000000036;66.69500000000698;66.34000000000378;65.80500000000757;67.88;59.92999999999665;61.36999999999534;68.70500000000175;72.73500000000058;67.32999999999993;72.46500000000015;72.73000000001048;69.07500000000437;57.820000000006985;74.98499999999996;75.54499999999825;175.22500000000582;65.40500000000611;61.01499999999942
==== Xres (X Resource Monitoring) ====
6;VanillaJS-TodoMVC/DeletingAllItems/Sync;46.02499999999998;34.530000000000655;31.715000000003783;34.575000000004366;38.93000000000757;43.485000000000014;33.00500000000102;44.59499999999389;34.38500000000204;33.10000000000582;37.875;34.594999999997526;33.169999999998254;34.974999999998545;35.61999999999534;35.89500000000004;36.159999999999854;38.654999999998836;33.770000000004075;34.904999999998836;42.299999999999955;36.13999999999942;32.56999999999971;37.970000000001164;33.69499999999243
 
7;VanillaJS-TodoMVC/DeletingAllItems/Async;15.475000000000023;11.94000000000051;17.70499999999447;20.19000000000233;9.509999999994761;7.425000000000011;14.040000000000873;10.104999999995925;13.430000000000291;11.985000000000582;6.984999999999957;23.75500000000102;63.45999999999913;8.014999999999418;12.059999999997672;21.40500000000003;13.279999999998836;11.364999999997963;7.864999999997963;33.154999999998836;12.420000000000016;22.409999999999854;8.090000000003783;12.994999999995343;7.125
A memory metric tracked during tp5 test runs. This metric is sampled every 20 seconds. This metric is collected on linux only.
8;VanillaJS-TodoMVC/DeletingAllItems;61.5;46.470000000001164;49.419999999998254;54.765000000006694;48.44000000000233;50.910000000000025;47.04500000000189;54.699999999989814;47.81500000000233;45.0850000000064;44.85999999999996;58.349999999998545;96.62999999999738;42.98999999999796;47.679999999993015;57.30000000000007;49.43999999999869;50.0199999999968;41.63500000000204;68.05999999999767;54.71999999999997;58.54999999999927;40.66000000000349;50.96499999999651;40.81999999999243
 
9;VanillaJS-TodoMVC;309.615;285.0699999999997;280.9699999999866;314.6050000000032;287.9549999999872;299.48500000000007;314.8150000000023;338.5349999999962;306.90000000000146;278.6000000000204;286.49499999999995;330.6649999999936;373.49499999999534;284.5650000000023;283.6499999999942;298.79;301.78499999999804;312.4150000000118;276.77500000000873;307.2450000000099;304.3749999999999;314.1999999999971;426.3500000000131;279.3550000000032;268.83000000000175
[https://linux.die.net/man/3/xres xres man page].
10;Vanilla-ES2015-TodoMVC/Adding100Items/Sync;187.5;191.14999999999782;198.6999999999971;196.77500000000146;193.25;192.91499999999996;197.4350000000013;194.09999999999854;193.60499999999593;195.09500000000116;197.52000000000004;200.14500000000044;241.61499999999796;193.61999999999534;191.75500000000466;193.32000000000005;195.8100000000013;200.0699999999997;192.95000000000437;475.40499999999884;192.67999999999995;196.77000000000044;197.66500000000087;194.8550000000032;197.4250000000029
 
11;Vanilla-ES2015-TodoMVC/Adding100Items/Async;22.949999999999932;15.715000000000146;18.2300000000032;30.5199999999968;21.53499999998894;23.80499999999995;34.409999999999854;16.494999999995343;21.619999999995343;19.130000000004657;21.230000000000018;17.354999999999563;22.25;16.18000000000029;21.02499999999418;16.92999999999995;16.280000000002474;19.965000000003783;27.185000000004948;18.630000000004657;27.039999999999964;17.125;16.510000000002037;22.06999999999971;19.514999999999418
===== Possible regression causes =====
12;Vanilla-ES2015-TodoMVC/Adding100Items;210.44999999999993;206.86499999999796;216.9300000000003;227.29499999999825;214.78499999998894;216.7199999999999;231.84500000000116;210.5949999999939;215.22499999999127;214.22500000000582;218.75000000000006;217.5;263.86499999999796;209.79999999999563;212.77999999999884;210.25;212.09000000000378;220.0350000000035;220.1350000000093;494.0350000000035;219.7199999999999;213.89500000000044;214.1750000000029;216.9250000000029;216.94000000000233
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
13;Vanilla-ES2015-TodoMVC/CompletingAllItems/Sync;54.25999999999999;50.36999999999898;55.58999999999651;52.38500000000204;56.31500000000233;52.174999999999955;50.875;50.349999999998545;56.61000000000058;51.570000000006985;52.495000000000005;50.13000000000102;62.71999999999389;51.18000000000029;53.44999999999709;51.29499999999996;50.66500000000087;87.94000000000233;52.83999999999651;60.330000000001746;51.69500000000005;50.80500000000029;52.70999999999913;54.79000000000087;54.13999999999942
 
14;Vanilla-ES2015-TodoMVC/CompletingAllItems/Async;27.379999999999995;17.260000000002037;32.075000000004366;11.264999999999418;23.94999999999709;17.154999999999973;13.764999999999418;15.525000000001455;19.63499999999476;16.214999999996508;25.055000000000064;20.340000000000146;22.485000000000582;18.849999999998545;21.829999999987194;16.75999999999999;18.82999999999811;23.034999999996217;19.775000000001455;24.289999999993597;33.664999999999964;18.400000000001455;27.7699999999968;18.450000000004366;17.855000000010477
==== Working Set (tp5_memset) ====
15;Vanilla-ES2015-TodoMVC/CompletingAllItems;81.63999999999999;67.63000000000102;87.66500000000087;63.650000000001455;80.26499999999942;69.32999999999993;64.63999999999942;65.875;76.24499999999534;67.78500000000349;77.55000000000007;70.47000000000116;85.20499999999447;70.02999999999884;75.27999999998428;68.05499999999995;69.49499999999898;110.97499999999854;72.61499999999796;84.61999999999534;85.36000000000001;69.20500000000175;80.47999999999593;73.24000000000524;71.9950000000099
 
16;Vanilla-ES2015-TodoMVC/DeletingItems/Sync;34.174999999999955;42.539999999997235;36.05500000000029;36.06500000000233;47.629999999990105;41.799999999999955;35.669999999998254;45.78500000000349;35.50499999999738;45.53500000000349;39.504999999999995;34.61000000000058;49.24500000000262;44.79500000000553;41.169999999998254;43.86500000000001;35.275000000001455;48.044999999998254;41.51000000000204;44.86500000000524;40.72000000000003;35.04000000000087;35.51499999999942;35.62999999999738;39.90000000000873
A memory metric tracked during tp5 test runs. This metric is sampled every 20 seconds. This metric is collected on windows only. [https://technet.microsoft.com/en-us/library/cc780836%28WS.10%29.aspx Description from Microsoft TechNet].
17;Vanilla-ES2015-TodoMVC/DeletingItems/Async;10.754999999999995;12.934999999997672;12.819999999999709;10.430000000000291;13.205000000001746;14.090000000000032;39.77000000000044;12.014999999999418;12.299999999995634;7.349999999991269;13.724999999999909;11.579999999998108;13.019999999996799;7.219999999993888;14.464999999996508;13.370000000000005;7.424999999999272;7.0350000000034925;11.20499999999447;11.330000000001746;20.844999999999914;6.985000000000582;9.305000000000291;7.255000000004657;12.240000000005239
 
18;Vanilla-ES2015-TodoMVC/DeletingItems;44.92999999999995;55.47499999999491;48.875;46.49500000000262;60.83499999999185;55.889999999999986;75.43999999999869;57.80000000000291;47.804999999993015;52.88499999999476;53.229999999999905;46.18999999999869;62.26499999999942;52.01499999999942;55.63499999999476;57.235000000000014;42.70000000000073;55.080000000001746;52.71499999999651;56.195000000006985;61.56499999999994;42.025000000001455;44.81999999999971;42.88500000000204;52.14000000001397
===== Possible regression causes =====
19;Vanilla-ES2015-TodoMVC;337.01999999999987;329.9699999999939;353.47000000000116;337.4400000000023;355.8849999999802;341.9399999999998;371.9249999999993;334.2699999999968;339.2749999999796;334.8950000000041;349.53000000000003;334.15999999999985;411.33499999999185;331.8449999999939;343.6949999999779;335.53999999999996;324.2850000000035;386.0900000000038;345.4650000000038;634.8500000000058;366.64499999999987;325.12500000000364;339.47499999999854;333.0500000000102;341.0750000000262
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
20;Vanilla-ES2015-Babel-Webpack-TodoMVC/Adding100Items/Sync;186.9849999999999;192.72999999999956;196.3249999999971;198.64500000000407;204.4100000000035;195.62;357.52999999999884;201.65000000000146;191.4300000000003;199.69500000000698;199.63;200.86000000000058;274.52999999999884;210.28499999999622;196.20500000000175;194.58000000000004;192.73499999999694;212.0550000000003;204.0699999999997;223.20500000000175;196.0999999999999;194.08000000000175;198.52000000000407;224.71500000000378;191.57000000000698
 
21;Vanilla-ES2015-Babel-Webpack-TodoMVC/Adding100Items/Async;19.97499999999991;16.31499999999869;22.955000000001746;27.419999999998254;16.214999999996508;26.11999999999989;16.2300000000032;23.294999999998254;25.089999999996508;19.769999999989523;25.125;20.020000000000437;30.464999999996508;22.110000000000582;16.944999999992433;17.940000000000055;22.455000000001746;25.614999999997963;25.155000000006112;30.044999999998254;33.51499999999987;17.349999999998545;16.6050000000032;18.30500000000029;28.964999999996508
==== Modified Page List Bytes ====
22;Vanilla-ES2015-Babel-Webpack-TodoMVC/Adding100Items;206.9599999999998;209.04499999999825;219.27999999999884;226.06500000000233;220.625;221.7399999999999;373.76000000000204;224.9449999999997;216.5199999999968;219.4649999999965;224.755;220.88000000000102;304.99499999999534;232.3949999999968;213.14999999999418;212.5200000000001;215.1899999999987;237.66999999999825;229.22500000000582;253.25;229.61499999999978;211.4300000000003;215.12500000000728;243.02000000000407;220.5350000000035
 
23;Vanilla-ES2015-Babel-Webpack-TodoMVC/CompletingAllItems/Sync;52.31000000000017;51.87000000000262;54.81999999999971;60.59499999999389;107.30499999999302;60.569999999999936;59.724999999998545;62.32499999999709;52.86999999999534;52.17499999998836;65.69999999999982;54.22999999999956;56.705000000001746;86.8900000000067;60.57499999999709;53.044999999999845;54.87999999999738;52.54000000000087;61.580000000001746;149.7399999999907;56.49000000000001;53.3650000000016;53.224999999998545;70.97000000000116;51.845000000001164
A memory metric tracked during tp5 test runs. This metric is sampled every 20 seconds. This metric is collected on Windows7 only. [https://msdn.microsoft.com/en-us/library/aa965225%28VS.85%29.aspx Description from Microsoft MSDN].
24;Vanilla-ES2015-Babel-Webpack-TodoMVC/CompletingAllItems/Async;17.24000000000001;18.75;16.349999999998545;9.584999999999127;22.639999999999418;10.330000000000155;13.540000000000873;17.80500000000029;15.205000000001746;17.789999999993597;19.37000000000012;21.68500000000131;9.959999999999127;16.854999999995925;31.464999999996508;18.829999999999927;17.92000000000189;17.344999999993888;18.2699999999968;17.110000000000582;21.3900000000001;27.0099999999984;19.364999999997963;19.924999999995634;18.964999999996508
 
25;Vanilla-ES2015-Babel-Webpack-TodoMVC/CompletingAllItems;69.55000000000018;70.62000000000262;71.16999999999825;70.17999999999302;129.94499999999243;70.90000000000009;73.26499999999942;80.12999999999738;68.07499999999709;69.96499999998196;85.06999999999994;75.91500000000087;66.66500000000087;103.74500000000262;92.0399999999936;71.87499999999977;72.79999999999927;69.88499999999476;79.84999999999854;166.84999999999127;77.88000000000011;80.375;72.58999999999651;90.8949999999968;70.80999999999767
===== Possible regression causes =====
26;Vanilla-ES2015-Babel-Webpack-TodoMVC/DeletingItems/Sync;41.649999999999864;48.38999999999942;39.984999999993306;36.25500000000466;44.76499999999942;43.409999999999854;41.56999999999971;35.03500000000349;36.43000000000029;40.05500000000757;41.25;41.08499999999913;40.86000000000058;80.55500000000029;47.75999999999476;37.92000000000007;35.22999999999956;66.125;38.7100000000064;38.48500000000058;46.309999999999945;40.19500000000335;39.7300000000032;36.674999999995634;38.05000000000291
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
27;Vanilla-ES2015-Babel-Webpack-TodoMVC/DeletingItems/Async;12.345000000000027;13.670000000001892;12.884999999994761;14.830000000001746;11.220000000001164;9.495000000000118;11.325000000000728;7.55000000000291;9.055000000000291;7.069999999992433;11.184999999999945;14.785000000003492;25.885000000002037;7.165000000000873;16.36500000000524;9.680000000000064;6.805000000000291;6.870000000002619;13.439999999995052;12.884999999994761;13.634999999999991;11.994999999998981;11.334999999999127;23.784999999996217;12.620000000009895
 
28;Vanilla-ES2015-Babel-Webpack-TodoMVC/DeletingItems;53.99499999999989;62.06000000000131;52.86999999998807;51.0850000000064;55.98500000000058;52.90499999999997;52.89500000000044;42.5850000000064;45.48500000000058;47.125;52.434999999999945;55.87000000000262;66.74500000000262;87.72000000000116;64.125;47.600000000000136;42.034999999999854;72.99500000000262;52.150000000001455;51.36999999999534;59.944999999999936;52.19000000000233;51.06500000000233;60.45999999999185;50.670000000012806
==== % CPU ====
29;Vanilla-ES2015-Babel-Webpack-TodoMVC;330.5049999999999;341.7250000000022;343.31999999998516;347.33000000000175;406.554999999993;345.54499999999996;499.9200000000019;347.6600000000035;330.07999999999447;336.55499999997846;362.2599999999999;352.6650000000045;438.40499999999884;423.8600000000006;369.3149999999878;331.995;330.0249999999978;380.54999999999563;361.2250000000058;471.4699999999866;367.4399999999998;343.9950000000026;338.7800000000061;394.3749999999927;342.01500000001397
 
30;React-TodoMVC/Adding100Items/Sync;299.2650000000001;299.6800000000003;321.70000000000437;308.51000000000204;313.6149999999907;321.855;306.9500000000007;305.2149999999965;335.4499999999971;338.52999999999884;316.16999999999985;317.7599999999984;312.92000000000553;315.4950000000026;357.29000000000815;311.1600000000001;338.21500000000015;314.3249999999971;335.6449999999968;314.5450000000128;308.28999999999996;309.1949999999997;325.72499999999854;324.3950000000041;316.22000000000116
Cpu usage tracked during tp5 test runs. This metric is sampled every 20 seconds. This metric is collected on windows only.
31;React-TodoMVC/Adding100Items/Async;19.685000000000173;23.674999999999272;23.1050000000032;29.470000000001164;21.860000000000582;23.174999999999955;26.100000000002183;21.089999999996508;34.87999999999738;21.919999999998254;23.924999999999955;57.73999999999796;29.94499999999971;23.220000000001164;22.664999999993597;24.52500000000009;32.8949999999968;23.095000000001164;24.8550000000032;38.354999999995925;26.585000000000036;29.56500000000233;50.549999999995634;20.409999999996217;30.554999999993015
 
32;React-TodoMVC/Adding100Items;318.9500000000003;323.35499999999956;344.80500000000757;337.9800000000032;335.47499999999127;345.03;333.0500000000029;326.304999999993;370.32999999999447;360.4499999999971;340.0949999999998;375.49999999999636;342.86500000000524;338.7150000000038;379.95500000000175;335.6850000000002;371.10999999999694;337.41999999999825;360.5;352.90000000000873;334.875;338.76000000000204;376.2749999999942;344.8050000000003;346.7749999999942
===== Possible regression causes =====
33;React-TodoMVC/CompletingAllItems/Sync;179.42999999999984;183.98500000000058;189.8199999999997;198.58499999999913;198.70500000000175;198.81999999999994;295.0750000000007;192.4449999999997;198.77000000000407;299.9349999999977;197.39999999999986;220.69000000000233;196.75;188.38500000000204;191.0899999999965;206.9849999999999;201.47499999999854;193.12000000000262;208.1800000000003;240.68499999999767;199.67499999999995;219.02499999999782;192.25999999999476;188.15000000000146;197.7850000000035
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
34;React-TodoMVC/CompletingAllItems/Async;17.795000000000073;15.280000000002474;16.080000000001746;18.264999999999418;15.634999999994761;19.519999999999982;17.11999999999898;18.12999999999738;22.50499999999738;20.66500000000815;16.795000000000073;31.209999999999127;16.94000000000233;16.709999999999127;16.625;20.575000000000045;44.47999999999956;18.139999999999418;17.68000000000029;22.094999999986612;17.97499999999991;22.895000000000437;57.08499999999913;22.529999999998836;17.995000000009895
 
35;React-TodoMVC/CompletingAllItems;197.2249999999999;199.26500000000306;205.90000000000146;216.84999999999854;214.3399999999965;218.33999999999992;312.1949999999997;210.5749999999971;221.27500000000146;320.6000000000058;214.19499999999994;251.90000000000146;213.69000000000233;205.09500000000116;207.7149999999965;227.55999999999995;245.9549999999981;211.26000000000204;225.86000000000058;262.7799999999843;217.64999999999986;241.91999999999825;249.3449999999939;210.6800000000003;215.7800000000134
==== Responsiveness ====
36;React-TodoMVC/DeletingAllItems/Sync;121.0949999999998;114.28499999999985;122.61499999999796;118.03000000000611;115.91500000000815;128.47500000000014;114.81499999999869;113.52500000000146;155.77500000000146;113.59500000000116;123.64499999999975;97.13999999999942;117.53500000000349;117.76499999999942;119.67999999999302;120.27999999999997;137.02500000000146;125.30000000000291;124.55000000000291;119.55500000000757;117.54999999999995;120.625;123.20500000000175;126.48999999999796;126.08999999999651
contact: :jimm, :jmaher
37;React-TodoMVC/DeletingAllItems/Async;7.394999999999982;8.580000000001746;12.364999999997963;6.610000000000582;10.989999999990687;6.735000000000127;9.794999999998254;10.340000000003783;6.625;9.069999999992433;6.654999999999745;8.81000000000131;14.569999999999709;8.619999999995343;218.33000000000175;9.275000000000091;15.8849999999984;6.260000000002037;17.69000000000233;25.835000000006403;13.28500000000031;7.970000000001164;58.229999999995925;10.485000000000582;8.740000000005239
 
38;React-TodoMVC/DeletingAllItems;128.48999999999978;122.8650000000016;134.97999999999593;124.6400000000067;126.90499999999884;135.21000000000026;124.60999999999694;123.86500000000524;162.40000000000146;122.6649999999936;130.2999999999995;105.95000000000073;132.1050000000032;126.38499999999476;338.00999999999476;129.55500000000006;152.90999999999985;131.56000000000495;142.24000000000524;145.39000000001397;130.83500000000026;128.59500000000116;181.43499999999767;136.97499999999854;134.83000000000175
Measures the delay for the event loop to process a [[Performance/Snappy#Current Infrastructure|tracer event]]. For more details, see {{bug|631571}}.
39;React-TodoMVC;644.665;645.4850000000042;685.685000000005;679.4700000000084;676.7199999999866;698.5800000000002;769.8549999999996;660.7449999999953;754.0049999999974;803.7149999999965;684.5899999999992;733.3499999999985;688.6600000000108;670.1949999999997;925.679999999993;692.8000000000002;769.9749999999949;680.2400000000052;728.6000000000058;761.070000000007;683.3600000000001;709.2750000000015;807.0549999999857;692.4599999999991;697.3850000000093
 
40;React-Redux-TodoMVC/Adding100Items/Sync;434.4200000000001;453.82000000000335;714.3499999999985;445.4150000000009;449.7250000000058;480.75999999999976;448.96500000000015;436.7850000000035;446.4799999999959;456.6399999999994;446.56500000000005;434.54499999999825;486.66999999999825;442.75500000000466;444.13000000000466;437.69000000000005;444.40499999999884;453.66999999999825;447.2649999999994;446.32500000001164;441.125;455.244999999999;446.1399999999994;443.4300000000003;485.91999999999825
The score on this benchmark is proportional to the sum of squares of all event delays that exceed a 20ms threshold. Lower is better.
41;React-Redux-TodoMVC/Adding100Items/Async;21.19499999999971;17.854999999999563;18.285000000003492;35.544999999998254;189.95500000000175;27.829999999999927;45.93500000000131;23.00999999999476;20.25;20.22500000000582;26.75;48.29500000000189;23.854999999995925;57.48999999999796;58.42500000000291;20.610000000000127;17.8650000000016;25.535000000003492;26.360000000000582;31.895000000004075;38.89499999999998;29.784999999999854;72.77999999999884;29.99000000000524;18.00999999999476
 
42;React-Redux-TodoMVC/Adding100Items;455.6149999999998;471.6750000000029;732.635000000002;480.9599999999991;639.6800000000076;508.5899999999997;494.90000000000146;459.79499999999825;466.7299999999959;476.86500000000524;473.31500000000005;482.84000000000015;510.5249999999942;500.2450000000026;502.55500000000757;458.3000000000002;462.27000000000044;479.20500000000175;473.625;478.2200000000157;480.02;485.02999999999884;518.9199999999983;473.42000000000553;503.929999999993
We collect 8000+ data points from the browser during the test and apply [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/output.py#l95 this formula] to the results:
43;React-Redux-TodoMVC/CompletingAllItems/Sync;586.6949999999997;655.2200000000012;728.6900000000023;610.3300000000017;619.2949999999983;603.2900000000004;701.1849999999977;594.1750000000029;610.1949999999997;607.3999999999942;614.3800000000001;690.5900000000001;660.6000000000058;602.8550000000032;619.5099999999948;608.9550000000004;612.0949999999975;620.5999999999985;606.1599999999962;602.7949999999983;609.75;609.8400000000001;615.5;622.4400000000023;691.9250000000029
<pre>
44;React-Redux-TodoMVC/CompletingAllItems/Async;19.159999999999854;14.919999999998254;20.37999999999738;21.19499999999971;72.06500000000233;30.22499999999991;17.139999999999418;19.06999999999971;22.095000000001164;330.41500000000815;25.050000000000182;41.955000000001746;56.71499999999651;77.06999999999971;68.59500000000116;22.360000000000127;15.390000000003056;17.700000000004366;28.55500000000029;17.75999999999476;61.07500000000027;29.12000000000262;142.95999999999913;29.160000000003492;20.264999999999418
return sum([float(x)*float(x) / 1000000.0 for x in val_list])
45;React-Redux-TodoMVC/CompletingAllItems;605.8549999999996;670.1399999999994;749.0699999999997;631.5250000000015;691.3600000000006;633.5150000000003;718.3249999999971;613.2450000000026;632.2900000000009;937.8150000000023;639.4300000000003;732.5450000000019;717.3150000000023;679.9250000000029;688.1049999999959;631.3150000000005;627.4850000000006;638.3000000000029;634.7149999999965;620.554999999993;670.8250000000003;638.9600000000028;758.4599999999991;651.6000000000058;712.1900000000023
</pre>
46;React-Redux-TodoMVC/DeletingItems/Sync;307.10000000000036;407.27999999999884;301.90499999999884;304.2299999999959;322.7699999999895;305.7249999999999;298.40499999999884;315.1650000000009;312.3950000000041;320.1100000000006;308.3699999999999;289.6349999999984;397.7549999999974;305.20500000000175;314.8099999999977;310.00999999999976;308.1949999999997;307.86999999999534;309.7900000000009;306.6399999999994;313.05999999999995;308.6100000000006;332.34999999999854;317.83000000000175;305.7399999999907
 
47;React-Redux-TodoMVC/DeletingItems/Async;67.1550000000002;149.01000000000204;7.5;9.934999999997672;57.125;7.349999999999909;17.1150000000016;8.505000000004657;9.594999999993888;42.50500000000466;8.2800000000002;14.270000000000437;18.435000000004948;58.92500000000291;59.125;7.7849999999998545;6.400000000001455;6.360000000000582;18.169999999998254;8.095000000001164;20.855000000000018;8.270000000000437;7.049999999995634;8.789999999993597;15.194999999992433
===== Possible regression causes =====
48;React-Redux-TodoMVC/DeletingItems;374.25500000000056;556.2900000000009;309.40499999999884;314.1649999999936;379.8949999999895;313.0749999999998;315.52000000000044;323.67000000000553;321.98999999999796;362.61500000000524;316.6500000000001;303.90499999999884;416.1900000000023;364.13000000000466;373.9349999999977;317.7949999999996;314.59500000000116;314.2299999999959;327.9599999999991;314.7350000000006;333.91499999999996;316.880000000001;339.3999999999942;326.61999999999534;320.9349999999831
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
49;React-Redux-TodoMVC;1435.725;1698.1050000000032;1791.1100000000006;1426.6499999999942;1710.9349999999977;1455.1799999999998;1528.744999999999;1396.7100000000064;1421.0099999999948;1777.2950000000128;1429.3950000000004;1519.2900000000009;1644.0299999999988;1544.3000000000102;1564.5950000000012;1407.4100000000003;1404.3500000000022;1431.7350000000006;1436.2999999999956;1413.5100000000093;1484.7600000000002;1440.8700000000026;1616.7799999999916;1451.6400000000067;1537.0549999999785
 
50;EmberJS-TodoMVC/Adding100Items/Sync;573.5649999999996;615.864999999998;579.5349999999962;588.4250000000029;588.3950000000041;594.2149999999997;588.3050000000003;572.5450000000055;598.5299999999988;583.4650000000111;587.6000000000004;574.5449999999983;591.25;584.4000000000015;592.8699999999953;610.5000000000005;593.7050000000017;583.4349999999977;608.7149999999965;594.8600000000006;614.6350000000002;580;640.8450000000012;602.8950000000041;592.4750000000058
=== tp5o_scroll ===
51;EmberJS-TodoMVC/Adding100Items/Async;19.300000000000182;33.465000000000146;44.48500000000058;26.875;56.979999999995925;36.18499999999949;19.534999999999854;70.46500000000378;21.580000000001746;77.61500000000524;75.49499999999989;24.81999999999971;19.07999999999447;21.80999999999767;25.05000000000291;34.89499999999953;21.354999999999563;21.19000000000233;22.205000000001746;30.74000000000524;42.600000000000364;20.724999999998545;25.910000000003492;23.610000000000582;69.11999999999534
* contact: :jmaher
52;EmberJS-TodoMVC/Adding100Items;592.8649999999998;649.3299999999981;624.0199999999968;615.3000000000029;645.375;630.3999999999992;607.8400000000001;643.0100000000093;620.1100000000006;661.0800000000163;663.0950000000003;599.364999999998;610.3299999999945;606.2099999999991;617.9199999999983;645.395;615.0600000000013;604.625;630.9199999999983;625.6000000000058;657.2350000000006;600.7249999999985;666.7550000000047;626.5050000000047;661.5950000000012
* source: [https://people.mozilla.org/~jmaher/taloszips/zips/tp5n.zip tp5n.zip]
53;EmberJS-TodoMVC/CompletingAllItems/Sync;241.21999999999935;250.63500000000204;234.49499999999534;250.66999999999825;257.6600000000035;230.34499999999935;254.08000000000175;254.91999999999825;252.30999999999767;240.6750000000029;252.625;239.49499999999898;239.08499999999913;259.1800000000003;248.43000000000757;259.77000000000044;237.9350000000013;242.84999999999854;239.86000000000058;238.9100000000035;260.1950000000006;242.99499999999898;367.45000000000437;248.7800000000061;238.5500000000029
* type: [https://wiki.mozilla.org/Buildbot/Talos#Page_Load_Tests PageLoader]
54;EmberJS-TodoMVC/CompletingAllItems/Async;17.145000000000437;47.31000000000131;18.535000000003492;30.69999999999709;22.245000000009895;20.625;30.234999999996944;67.3599999999933;21.360000000000582;273.29499999999825;64.89000000000033;25.74499999999898;15.069999999999709;17.505000000004657;28.735000000000582;40.039999999999964;20.7599999999984;18.150000000001455;22.659999999996217;19.330000000001746;14.860000000000582;20.80500000000029;28.6050000000032;19.470000000001164;71.45999999999185
* data: we load each of the 51 tp5o pages 12 times, resulting in 51 sets of 12 data points.
55;EmberJS-TodoMVC/CompletingAllItems;258.3649999999998;297.94500000000335;253.02999999999884;281.36999999999534;279.9050000000134;250.96999999999935;284.3149999999987;322.27999999999156;273.66999999999825;513.9700000000012;317.5150000000003;265.23999999999796;254.15499999999884;276.68500000000495;277.16500000000815;299.8100000000004;258.6949999999997;261;262.5199999999968;258.24000000000524;275.0550000000012;263.7999999999993;396.05500000000757;268.2500000000073;310.00999999999476
* '''To run it locally''', you'd need [[#tp5n_pages_set|tp5n.zip]].
56;EmberJS-TodoMVC/DeletingItems/Sync;140.9949999999999;259.15999999999985;144.56500000000233;153.38999999999942;152.125;143.3099999999995;160.1150000000016;146.71500000000378;152.93499999999767;146.0500000000029;140.0699999999997;139.61999999999898;141.51499999999942;149.23500000000058;149.72499999999127;151.09999999999945;142.37000000000262;149.84500000000116;144.82999999999447;150.30499999999302;144.23499999999967;147.44000000000233;152.25;143.18999999999505;141.86000000000058
* summarization:
57;EmberJS-TodoMVC/DeletingItems/Async;7.585000000000036;17.81999999999971;7.2700000000040745;7.105000000003201;6.654999999998836;7.539999999999964;6.444999999999709;75.83000000000175;7.494999999995343;7.30000000000291;39.409999999999854;6.2849999999998545;6.700000000004366;12.659999999996217;6.915000000008149;14.329999999999927;6.474999999998545;10.844999999993888;12.365000000005239;9.130000000004657;16.81500000000051;6.415000000000873;12.659999999996217;9.470000000001164;62.0399999999936
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] data point, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 11; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l470 source: test.py]
58;EmberJS-TodoMVC/DeletingItems;148.57999999999993;276.97999999999956;151.8350000000064;160.49500000000262;158.77999999999884;150.84999999999945;166.5600000000013;222.54500000000553;160.42999999999302;153.35000000000582;179.47999999999956;145.90499999999884;148.21500000000378;161.8949999999968;156.63999999999942;165.42999999999938;148.84500000000116;160.68999999999505;157.1949999999997;159.43499999999767;161.05000000000018;153.8550000000032;164.90999999999622;152.65999999999622;203.89999999999418
** suite: [https://wiki.mozilla.org/Buildbot/Talos/Data#geometric_mean geometric mean] of the 51 subtest results.
59;EmberJS-TodoMVC;999.8099999999995;1224.255000000001;1028.885000000002;1057.1650000000009;1084.0600000000122;1032.219999999998;1058.7150000000001;1187.8350000000064;1054.2099999999919;1328.4000000000233;1160.0900000000001;1010.5099999999948;1012.6999999999971;1044.7900000000009;1051.7250000000058;1110.6349999999998;1022.6000000000022;1026.314999999995;1050.6349999999948;1043.2750000000087;1093.340000000002;1018.380000000001;1227.7200000000084;1047.4150000000081;1175.50499999999
 
60;EmberJS-Debug-TodoMVC/Adding100Items/Sync;1462.4049999999997;1356.1599999999999;1356.0099999999948;1344.5;1392.8500000000058;1335.9300000000003;1409.7899999999972;1513.6649999999936;1342.2000000000044;1743.5799999999872;1419.125;1340.6850000000013;1370.5449999999983;1414.4449999999997;1743.75;1373.9099999999999;1337.130000000001;1865.7150000000038;1688.5950000000012;1387.9399999999878;1354.625;1354.5300000000025;1356.2849999999962;1370.6949999999997;1499.270000000004
{| cellspacing="1" cellpadding="1" border="1"
61;EmberJS-Debug-TodoMVC/Adding100Items/Async;30.920000000000073;107.78499999999985;23.82499999999709;83.31499999999505;28.47500000000582;37.8100000000004;31.885000000002037;25.2300000000032;24.18499999999767;25.035000000003492;25.295000000000073;71.46500000000015;37.3949999999968;33.86000000000058;23.379999999990105;28.460000000000036;72.59500000000116;26.99500000000262;22.924999999995634;24.385000000009313;31.954999999999927;71.53999999999724;24.360000000000582;31.584999999999127;33.68499999999767
|-
62;EmberJS-Debug-TodoMVC/Adding100Items;1493.3249999999998;1463.9449999999997;1379.8349999999919;1427.814999999995;1421.3250000000116;1373.7400000000007;1441.6749999999993;1538.8949999999968;1366.385000000002;1768.6149999999907;1444.42;1412.1500000000015;1407.939999999995;1448.3050000000003;1767.12999999999;1402.37;1409.7250000000022;1892.7100000000064;1711.5199999999968;1412.324999999997;1386.58;1426.0699999999997;1380.6449999999968;1402.2799999999988;1532.9550000000017
| Talos test name
63;EmberJS-Debug-TodoMVC/CompletingAllItems/Sync;635.4200000000001;721.135000000002;633.885000000002;631.6300000000047;730.1199999999953;622.8800000000001;641.8450000000012;640.25;689.114999999998;642.8450000000012;648.4650000000001;978.3799999999974;629.5750000000044;644.114999999998;655.8399999999965;631.125;612.005000000001;675.560000000005;780.9150000000009;636.1300000000047;671.8450000000003;627.5750000000007;640.8349999999991;632.6600000000035;650.1050000000105
| Description
64;EmberJS-Debug-TodoMVC/CompletingAllItems/Async;19.219999999999345;20.219999999997526;14.944999999999709;355.7850000000035;30.05999999999767;14.875;26.845000000001164;38.19999999999709;22.169999999998254;33.8350000000064;12.659999999999854;42.29500000000189;18.31500000000233;19.895000000004075;33.845000000001164;21.449999999999818;292.46000000000276;34.21499999999651;32.46499999999651;18.389999999999418;34.76500000000033;266.02000000000044;27.375;15.400000000001455;21.004999999990105
|-
65;EmberJS-Debug-TodoMVC/CompletingAllItems;654.6399999999994;741.3549999999996;648.8300000000017;987.4150000000081;760.179999999993;637.7550000000001;668.6900000000023;678.4499999999971;711.2849999999962;676.6800000000076;661.125;1020.6749999999993;647.8900000000067;664.010000000002;689.6849999999977;652.5749999999998;904.4650000000038;709.7750000000015;813.3799999999974;654.5200000000041;706.6100000000006;893.5950000000012;668.2099999999991;648.060000000005;671.1100000000006
|tp5o_scroll
66;EmberJS-Debug-TodoMVC/DeletingItems/Sync;796.5100000000002;881.880000000001;839.4449999999997;838.7900000000009;804.2599999999948;822.0099999999993;1158.4050000000025;809.6849999999977;922.9850000000006;837.7299999999959;854.1950000000006;900.8899999999994;814.5950000000012;863.2050000000017;850.3199999999924;801.0650000000005;815.6500000000015;829.364999999998;851.0900000000038;821.9799999999959;838.9949999999999;820.0900000000001;999.2700000000041;793.5199999999968;826.6999999999971
|Measures average frames interval while scrolling the pages of the tp5o set
67;EmberJS-Debug-TodoMVC/DeletingItems/Async;8.329999999999927;7.240000000001601;18.540000000000873;9.509999999994761;9.77499999999418;16.68499999999949;15.830000000001746;21.985000000000582;14.779999999998836;9.714999999996508;10.454999999999927;7.049999999999272;16.515000000006694;24.239999999997963;9.764999999999418;8.534999999999854;10.385000000002037;8.904999999998836;7.25;18.705000000001746;10.969999999999345;10.095000000001164;176.27500000000146;10.779999999998836;19.97500000000582
|}
68;EmberJS-Debug-TodoMVC/DeletingItems;804.8400000000001;889.1200000000026;857.9850000000006;848.2999999999956;814.0349999999889;838.6949999999988;1174.2350000000042;831.6699999999983;937.7649999999994;847.4449999999924;864.6500000000005;907.9399999999987;831.1100000000079;887.4449999999997;860.0849999999919;809.6000000000004;826.0350000000035;838.2699999999968;858.3400000000038;840.6849999999977;849.9649999999992;830.1850000000013;1175.5450000000055;804.2999999999956;846.6750000000029
 
69;EmberJS-Debug-TodoMVC;2952.8049999999994;3094.420000000002;2886.649999999994;3263.529999999999;2995.5399999999936;2850.1899999999996;3284.600000000006;3049.014999999992;3015.4349999999977;3292.7399999999907;2970.1950000000006;3340.7649999999994;2886.9400000000096;2999.760000000002;3316.8999999999796;2864.545;3140.2250000000095;3440.7550000000047;3383.239999999998;2907.529999999999;2943.1549999999997;3149.850000000002;3224.4000000000015;2854.6399999999994;3050.7400000000052
This test is identical to tscrollx, but it scrolls the 50 pages of the tp5o set (rather than 6 synthetic pages which tscrollx scrolls). For each page, the test waits 500ms after the page load event fires, then iterates 100 scroll steps of 10px each (or until the bottom of the page is reached - whichever comes first), then reports the average frame interval.
70;BackboneJS-TodoMVC/Adding100Items/Sync;178.04500000000007;175.88999999999942;185.27500000000146;183.20500000000175;172.60999999998603;177.9300000000003;190.15000000000146;193.98500000000058;174.48500000000058;186.7850000000035;190.6900000000005;189.8199999999997;181.61499999999796;184.1600000000035;175.14999999999418;184.60499999999956;190.16500000000087;185.9300000000003;187.65000000000146;195.61500000000524;180.80499999999847;186.12999999999738;186.80999999999767;184.83499999999913;183.89000000001397
 
71;BackboneJS-TodoMVC/Adding100Items/Async;33.94000000000051;35.13999999999942;35.520000000004075;27.264999999999418;34.729999999995925;25.155000000000655;30.99500000000262;28.44499999999971;40.720000000001164;44.03500000000349;29.139999999999418;36.909999999999854;23.715000000003783;37.43499999999767;42.38999999999942;29.44499999999971;45.95999999999913;39.51000000000204;38.145000000004075;31.460000000006403;25.840000000000146;37.345000000001164;31.81999999999971;24.139999999999418;35.544999999998254
{| class="mw-collapsible mw-collapsed wikitable"
72;BackboneJS-TodoMVC/Adding100Items;211.98500000000058;211.02999999999884;220.79500000000553;210.47000000000116;207.33999999998196;203.08500000000095;221.14500000000407;222.4300000000003;215.20500000000175;230.82000000000698;219.82999999999993;226.72999999999956;205.33000000000175;221.59500000000116;217.5399999999936;214.04999999999927;236.125;225.44000000000233;225.79500000000553;227.07500000001164;206.64499999999862;223.47499999999854;218.62999999999738;208.97499999999854;219.43500000001222
! Example data
73;BackboneJS-TodoMVC/CompletingAllItems/Sync;78.02499999999964;77.74499999999898;79.38500000000204;80.9800000000032;77.71500000001106;73.79500000000007;78.48500000000058;83.44999999999709;78.04999999999563;78.29499999999825;80.26500000000124;83.8650000000016;75.65999999999622;79.08000000000175;76.41999999999825;76.4950000000008;81.92499999999927;92.11000000000058;82.89999999999418;80.04499999999825;82.36499999999978;83.08499999999913;75.66000000000349;74.625;80.64500000000407
|-
74;BackboneJS-TodoMVC/CompletingAllItems/Async;22.94499999999971;51.60499999999956;59.74000000000524;43.06000000000495;45.23999999999069;29.829999999999927;32.17000000000189;26.69999999999709;49.099999999998545;48.94999999999709;24.219999999999345;40.69499999999971;141.06500000000233;20.9800000000032;47.43499999999767;32.6150000000016;51.91500000000087;31.005000000004657;29.375;33.85000000000582;28.93000000000029;33.27000000000044;23.090000000003783;38.04000000000087;36.25
| <pre>
75;BackboneJS-TodoMVC/CompletingAllItems;100.96999999999935;129.34999999999854;139.12500000000728;124.04000000000815;122.95500000000175;103.625;110.65500000000247;110.14999999999418;127.14999999999418;127.24499999999534;104.48500000000058;124.56000000000131;216.72499999999854;100.06000000000495;123.85499999999593;109.1100000000024;133.84000000000015;123.11500000000524;112.27499999999418;113.89500000000407;111.29500000000007;116.35499999999956;98.75000000000728;112.66500000000087;116.89500000000407
0;163.com/www.163.com/index.html;9.73;8.61;7.37;8.17;7.58;7.29;6.88;7.45;6.91;6.61;8.47;7.12
76;BackboneJS-TodoMVC/DeletingAllItems/Sync;80.18000000000029;79.99500000000262;90.93499999999767;88.20500000000175;87.04499999999825;77.03999999999905;79.93000000000029;91.87000000000262;86.34500000000116;86.5399999999936;92.58500000000095;93.15999999999985;79.00500000000466;85.45999999999913;78.54999999998836;83.94000000000051;81.30500000000029;94.04500000000553;84.15499999999884;95.00999999999476;86.03999999999905;86.375;87.94999999999709;90.36499999999796;92.02000000000407
1;56.com/www.56.com/index.html;10.85;10.24;10.75;10.30;10.23;10.10;10.31;10.06;11.10;10.06;9.56;10.30
77;BackboneJS-TodoMVC/DeletingAllItems/Async;11.604999999999563;11.270000000000437;12.105000000003201;19.189999999995052;26.95999999999185;15.539999999999054;10.014999999999418;13.319999999999709;37.6449999999968;27.870000000009895;8.25;35.42000000000189;31.68000000000029;10.42000000000553;28.11500000000524;17.485000000000582;7.31000000000131;32.56999999999971;9.349999999998545;8.839999999996508;12.854999999999563;13.489999999997963;11.235000000000582;34.31999999999971;10.94999999999709
2;aljazeera.net/aljazeera.net/portal.html;9.23;7.15;7.50;7.26;7.73;7.05;7.14;7.66;7.23;7.93;7.26;7.18
78;BackboneJS-TodoMVC/DeletingAllItems;91.78499999999985;91.26500000000306;103.04000000000087;107.3949999999968;114.0049999999901;92.57999999999811;89.94499999999971;105.19000000000233;123.98999999999796;114.41000000000349;100.83500000000095;128.58000000000175;110.68500000000495;95.88000000000466;106.6649999999936;101.42500000000109;88.6150000000016;126.61500000000524;93.50499999999738;103.84999999999127;98.89499999999862;99.86499999999796;99.18499999999767;124.68499999999767;102.97000000000116
3;amazon.com/www.amazon.com/Kindle-Wireless-Reader-Wifi-Graphite/dp/B002Y27P3M/507846.html;7.14;6.87;7.18;6.31;6.93;6.71;6.37;7.00;6.59;5.37;7.31;6.13
79;BackboneJS-TodoMVC;404.7399999999998;431.64500000000044;462.9600000000137;441.9050000000061;444.2999999999738;399.28999999999905;421.74500000000626;437.7699999999968;466.3449999999939;472.4750000000058;425.15000000000146;479.8700000000026;532.7400000000052;417.53500000001077;448.0599999999831;424.58500000000276;458.58000000000175;475.1700000000128;431.5749999999971;444.820000000007;416.8349999999973;439.69499999999607;416.5650000000023;446.3249999999971;439.30000000001746
4;bbc.co.uk/www.bbc.co.uk/news/index.html;7.39;6.33;6.22;7.66;6.67;7.77;6.91;7.74;7.08;6.36;6.03;7.12
80;AngularJS-TodoMVC/Adding100Items/Sync;534.8099999999995;542.2049999999981;530.8900000000067;512.1500000000015;552.2149999999965;522.1750000000011;534.4650000000001;548.7200000000012;528.864999999998;521.6599999999889;551.2250000000004;529.6650000000009;633.2699999999968;536.8500000000058;533.0899999999965;545.8649999999998;543.5349999999999;541.6599999999962;542.6800000000003;579.1549999999988;521.4400000000005;556.6999999999971;521.3050000000003;542.9449999999997;535.8399999999965
5;beatonna.livejournal.com/beatonna.livejournal.com/index.html;5.79;5.79;5.68;5.46;5.55;5.48;5.69;5.83;5.88;5.97;5.93;5.88
81;AngularJS-TodoMVC/Adding100Items/Async;26.270000000000437;24.110000000000582;25.56999999999971;33.25999999999476;34.43000000000757;26.69000000000051;67.20000000000073;24.025000000001455;72.19000000000233;25.010000000009313;31.30500000000029;32.42500000000291;22.025000000001455;33.61000000000058;25.81500000000233;31.75;24.854999999999563;25.93499999999767;24.529999999998836;25.955000000001746;32.43499999999949;34.585000000002765;33.154999999998836;24.69499999999971;23.85000000000582
6;bild.de/www.bild.de/index.html;8.65;7.63;7.17;6.36;7.44;7.68;8.63;6.71;8.34;7.15;7.82;7.70
82;AngularJS-TodoMVC/Adding100Items;561.0799999999999;566.3149999999987;556.4600000000064;545.4099999999962;586.6450000000041;548.8650000000016;601.6650000000009;572.7450000000026;601.0550000000003;546.6699999999983;582.5300000000007;562.0900000000038;655.2949999999983;570.4600000000064;558.9049999999988;577.6149999999998;568.3899999999994;567.5949999999939;567.2099999999991;605.1100000000006;553.875;591.2849999999999;554.4599999999991;567.6399999999994;559.6900000000023
7;cgi.ebay.com/cgi.ebay.com/ALL-NEW-KINDLE-3-eBOOK-WIRELESS-READING-DEVICE-W-WIFI-/130496077314@pt=LH_DefaultDomain_0&hash=item1e622c1e02.html;7.12;6.81;7.22;6.98;7.05;5.68;7.15;6.54;7.31;7.18;7.82;7.77
83;AngularJS-TodoMVC/CompletingAllItems/Sync;137.70999999999913;129.64500000000044;125.17499999999563;123.04000000000087;136.40000000000873;125.68499999999949;138.84000000000015;128.31499999999505;127.34499999999389;134.2149999999965;126.07500000000073;123.875;130.29499999999825;122.90500000000611;128.7899999999936;133.15500000000065;162.12000000000262;126.34000000000378;127.30000000000291;126.90499999999884;123.61000000000058;130.1349999999984;124.62000000000262;129.3050000000003;135.80499999999302
8;chemistry.about.com/chemistry.about.com/index.html;6.76;6.17;6.41;6.88;5.67;5.47;6.83;6.28;6.16;6.81;6.21;6.75
84;AngularJS-TodoMVC/CompletingAllItems/Async;29.600000000000364;17.400000000001455;15.165000000000873;26.075000000004366;18.654999999998836;19.395000000000437;80.04000000000087;28.639999999999418;184.66500000000087;15.104999999995925;16.974999999998545;21.104999999999563;37.30000000000291;27.75499999999738;15.955000000001746;17.655000000000655;21.125;19.79500000000553;21.015000000006694;16.68499999999767;32.51500000000124;29.49499999999898;17.904999999998836;19.845000000001164;21.839999999996508
9;chinaz.com/chinaz.com/index.html;10.72;7.99;7.33;7.10;7.85;8.62;8.39;6.72;6.26;6.65;8.14;7.78
85;AngularJS-TodoMVC/CompletingAllItems;167.3099999999995;147.0450000000019;140.3399999999965;149.11500000000524;155.05500000000757;145.07999999999993;218.88000000000102;156.95499999999447;312.00999999999476;149.31999999999243;143.04999999999927;144.97999999999956;167.59500000000116;150.6600000000035;144.74499999999534;150.8100000000013;183.24500000000262;146.1350000000093;148.3150000000096;143.5899999999965;156.12500000000182;159.62999999999738;142.52500000000146;149.15000000000146;157.64499999998952
10;cnn.com/www.cnn.com/index.html;7.73;6.80;6.08;8.27;9.24;7.81;7.69;7.05;8.17;7.70;7.90;6.81
86;AngularJS-TodoMVC/DeletingAllItems/Sync;296.66999999999825;300.005000000001;301.68999999999505;311.7450000000026;432.0249999999942;296.8099999999995;302.66999999999825;321.77999999999884;324.04499999999825;310.36500000000524;307.0849999999991;306.28499999999985;422.8050000000003;300.15000000000146;442.3099999999977;517.5999999999985;324.90499999999884;309.09999999999854;309.02999999999884;300.36500000000524;360.6700000000001;313.7549999999974;306.8349999999991;418.5650000000023;318.1750000000029
11;dailymail.co.uk/www.dailymail.co.uk/ushome/index.html;6.37;8.28;7.19;8.00;8.09;7.43;6.90;7.24;7.77;7.29;7.38;6.14
87;AngularJS-TodoMVC/DeletingAllItems/Async;37.100000000000364;15.204999999998108;15.384999999994761;22.544999999998254;16.354999999995925;19.390000000001237;44.340000000000146;12.92500000000291;23.525000000001455;16.220000000001164;17.954999999999927;12.794999999998254;17.290000000000873;23.50499999999738;20.405000000013388;17.74499999999898;13.965000000000146;20.31999999999971;17.039999999993597;23.955000000001746;17.079999999999927;28.770000000000437;23.06500000000233;15.509999999994761;15.360000000000582
12;dailymotion.com/www.dailymotion.com/us.html;9.53;9.80;9.29;9.03;9.10;8.64;8.62;8.71;8.77;9.81;9.64;8.96
88;AngularJS-TodoMVC/DeletingAllItems;333.7699999999986;315.2099999999991;317.0749999999898;334.2900000000009;448.3799999999901;316.2000000000007;347.0099999999984;334.70500000000175;347.5699999999997;326.5850000000064;325.03999999999905;319.0799999999981;440.09500000000116;323.65499999999884;462.71500000001106;535.3449999999975;338.869999999999;329.41999999999825;326.06999999999243;324.320000000007;377.75;342.5249999999978;329.90000000000146;434.0749999999971;333.5350000000035
13;digg.com/digg.com/news/story/New_logo_for_Mozilla_Firefox_browser.html;7.72;7.06;7.60;5.67;6.85;7.32;7.80;5.98;8.27;6.68;7.52;8.39
89;AngularJS-TodoMVC;1062.159999999998;1028.5699999999997;1013.8749999999927;1028.8150000000023;1190.0800000000017;1010.1450000000023;1167.5550000000003;1064.4049999999988;1260.6349999999948;1022.5749999999971;1050.619999999999;1026.1500000000015;1262.9850000000006;1044.7750000000087;1166.3650000000052;1263.7699999999986;1090.505000000001;1043.1500000000015;1041.5950000000012;1073.020000000004;1087.7500000000018;1093.439999999995;1026.885000000002;1150.864999999998;1050.8699999999953
14;ezinearticles.com/ezinearticles.com/index.html@Migraine-Ocular---The-Eye-Migraines&id=4684133.html;7.14;7.11;8.09;7.17;6.87;7.12;7.65;7.74;7.26;7.36;6.91;6.95
90;Angular2-TypeScript-TodoMVC/Adding100Items/Sync;151.8700000000008;165.75500000000102;167.17499999999563;156.56500000000233;166.3649999999907;177.4499999999989;159.91499999999724;168.3249999999971;155.2850000000035;161.69499999999243;157.34500000000116;156;170.5699999999997;159.18999999999505;162.8350000000064;178.82999999999993;165.00499999999738;173.05999999999767;166.6999999999971;171.56500000000233;156.51999999999862;152.53499999999985;156.3550000000032;167.18999999999505;162.31000000001222
15;globo.com/www.globo.com/index.html;6.71;7.91;5.83;7.34;7.75;8.00;7.73;7.85;7.03;6.42;8.43;8.11
91;Angular2-TypeScript-TodoMVC/Adding100Items/Async;21.670000000000073;30.794999999998254;15.560000000004948;27.5;26.42500000000291;49.414999999999054;18.985000000000582;27.69499999999971;14.845000000001164;30.464999999996508;18.549999999999272;31.69499999999971;26.955000000001746;18.145000000004075;32.75999999999476;19.350000000000364;25.514999999999418;26.580000000001746;21.56999999999971;19.07499999999709;17.039999999999054;16.25500000000102;15.489999999997963;15.854999999995925;17.90000000000873
16;google.com/www.google.com/search@q=mozilla.html;6.49;6.23;7.96;6.39;7.23;8.19;7.35;7.39;6.94;7.24;7.55;7.62
92;Angular2-TypeScript-TodoMVC/Adding100Items;173.54000000000087;196.54999999999927;182.73500000000058;184.06500000000233;192.7899999999936;226.86499999999796;178.89999999999782;196.0199999999968;170.13000000000466;192.15999999998894;175.89500000000044;187.6949999999997;197.52500000000146;177.33499999999913;195.59500000000116;198.1800000000003;190.5199999999968;199.63999999999942;188.2699999999968;190.63999999999942;173.55999999999767;168.79000000000087;171.84500000000116;183.04499999999098;180.21000000002095
17;goo.ne.jp/goo.ne.jp/index.html;8.56;7.18;7.15;7.03;6.85;7.62;7.66;6.99;7.84;7.51;7.23;7.18
93;Angular2-TypeScript-TodoMVC/CompletingAllItems/Sync;24.655000000000655;26.37000000000262;26.215000000003783;24.525000000001455;25.730000000010477;24.19999999999891;25.665000000000873;25.654999999998836;25.139999999999418;27;27.145000000000437;30.360000000000582;26.20499999999447;29.24000000000524;28.27499999999418;26.045000000000073;27.00499999999738;26.205000000001746;25.409999999996217;26.82499999999709;24.9950000000008;25.18500000000131;25.094999999993888;26.334999999999127;26.739999999990687
18;guardian.co.uk/www.guardian.co.uk/index.html;7.32;7.62;8.18;7.62;7.83;8.08;7.60;8.17;8.47;7.54;7.92;8.09
94;Angular2-TypeScript-TodoMVC/CompletingAllItems/Async;14.154999999998836;22.1150000000016;15.710000000006403;9.665000000000873;15.805000000007567;21.82499999999891;14.25499999999738;9.875;18.87000000000262;19.644999999989523;16.420000000000073;9.2400000000016;11.209999999999127;24.24000000000524;13.77499999999418;23.86499999999978;24.169999999998254;18.32499999999709;24.56999999999971;19.154999999998836;18.485000000000582;19.010000000002037;19.794999999998254;28.970000000001164;14.85000000000582
19;homeway.com.cn/www.hexun.com/index.html;10.18;8.75;8.83;8.64;8.98;8.07;7.76;9.29;8.05;7.55;8.91;7.78
95;Angular2-TypeScript-TodoMVC/CompletingAllItems;38.80999999999949;48.48500000000422;41.925000000010186;34.19000000000233;41.535000000018044;46.02499999999782;39.919999999998254;35.529999999998836;44.01000000000204;46.64499999998952;43.56500000000051;39.60000000000218;37.4149999999936;53.48000000001048;42.04999999998836;49.909999999999854;51.174999999995634;44.529999999998836;49.979999999995925;45.979999999995925;43.48000000000138;44.19500000000335;44.88999999999214;55.30500000000029;41.58999999999651
20;huffingtonpost.com/www.huffingtonpost.com/index.html;8.38;7.17;7.03;6.83;6.49;6.47;6.69;7.08;6.81;7.29;7.13;7.70
96;Angular2-TypeScript-TodoMVC/DeletingItems/Sync;62.524999999999636;62.18999999999869;57.23500000000058;51.775000000001455;61.55999999999767;51.585000000000946;60.965000000000146;56.98500000000058;55.200000000004366;63.029999999998836;55.98999999999978;54.030000000002474;68.03500000000349;54.26499999999942;55.20999999999185;62.225000000000364;64.97999999999956;55.41000000000349;63.70999999999913;60.50500000000466;56.469999999999345;61.86999999999898;59.18000000000029;55.23999999999796;55.00500000000466
21;ifeng.com/ifeng.com/index.html;12.45;8.65;8.75;7.56;8.26;7.71;8.04;7.45;7.83;7.14;8.38;7.68
97;Angular2-TypeScript-TodoMVC/DeletingItems/Async;20.05999999999949;20.399999999997817;9.549999999995634;7.2849999999962165;19.910000000003492;10.739999999999782;8.825000000000728;10.780000000006112;8.434999999997672;9.539999999993597;6.8399999999983265;20.95499999999811;19.229999999995925;7.059999999997672;16.729999999995925;10.56500000000051;20.49499999999898;37.36499999999796;13.785000000003492;10.460000000006403;10.715000000000146;9.3849999999984;9.380000000004657;6.169999999998254;6.8549999999959255
22;imdb.com/www.imdb.com/title/tt1099212/index.html;8.53;5.65;6.94;7.18;6.10;7.57;6.26;8.34;8.16;7.29;7.62;8.51
98;Angular2-TypeScript-TodoMVC/DeletingItems;82.58499999999913;82.58999999999651;66.78499999999622;59.05999999999767;81.47000000000116;62.32500000000073;69.79000000000087;67.7650000000067;63.63500000000204;72.56999999999243;62.82999999999811;74.98500000000058;87.26499999999942;61.32499999999709;71.93999999998778;72.79000000000087;85.47499999999854;92.77500000000146;77.49500000000262;70.96500000001106;67.18499999999949;71.25499999999738;68.56000000000495;61.40999999999622;61.86000000000058
23;imgur.com/imgur.com/gallery/index.html;8.10;7.20;7.50;7.88;7.27;6.97;8.13;7.14;7.59;7.39;8.01;8.82
99;Angular2-TypeScript-TodoMVC;294.9349999999995;327.625;291.445000000007;277.3150000000023;315.7950000000128;335.2149999999965;288.60999999999694;299.3150000000023;277.77500000000873;311.3749999999709;282.28999999999905;302.2800000000025;322.20499999999447;292.1400000000067;309.5849999999773;320.880000000001;327.169999999991;336.9449999999997;315.74499999999534;307.5850000000064;284.22499999999854;284.2400000000016;285.29499999999825;299.7599999999875;283.66000000001804
24;indiatimes.com/www.indiatimes.com/index.html;8.00;6.74;7.37;8.52;7.03;8.45;7.08;8.47;9.26;7.89;7.17;6.74
100;VueJS-TodoMVC/Adding100Items/Sync;21.265000000001237;24.090000000000146;21.3949999999968;22.220000000001164;21.464999999996508;20.780000000000655;25.360000000000582;22.43000000000029;21.150000000001455;22.260000000009313;21.715000000000146;22.93499999999767;22.455000000001746;21.255000000004657;24.639999999999418;21.19499999999971;21.6349999999984;20.904999999998836;21.30999999999767;21.86500000000524;21.204999999999927;21.600000000002183;22.474999999998545;22.709999999999127;22.94999999999709
25;mail.ru/mail.ru/index.html;7.64;9.50;9.47;7.03;6.45;6.24;8.03;6.72;7.18;6.39;6.25;6.25
101;VueJS-TodoMVC/Adding100Items/Async;98.95000000000073;100.3650000000016;91.74500000000262;102.06999999999971;95.78500000000349;117.88500000000022;111.46500000000015;105.63999999999942;104.54999999999563;98.93499999999767;105.22500000000036;107.55999999999767;123.34000000000378;94.50499999999738;114.52000000000407;94.4950000000008;112.09500000000116;134.85499999999593;104.92500000000291;95.13000000000466;95.39499999999862;91.72000000000116;95.15499999999884;104.37999999999738;105.38999999999942
26;mashable.com/mashable.com/index.html;7.97;8.03;6.10;7.80;7.91;7.26;7.49;7.45;7.60;7.08;7.63;7.36
102;VueJS-TodoMVC/Adding100Items;120.21500000000196;124.45500000000175;113.13999999999942;124.29000000000087;117.25;138.66500000000087;136.82500000000073;128.0699999999997;125.69999999999709;121.19500000000698;126.94000000000051;130.49499999999534;145.79500000000553;115.76000000000204;139.1600000000035;115.69000000000051;133.72999999999956;155.75999999999476;126.23500000000058;116.9950000000099;116.59999999999854;113.32000000000335;117.62999999999738;127.08999999999651;128.3399999999965
27;media.photobucket.com/media.photobucket.com/image/funny%20gif/findstuff22/Best%20Images/Funny/funny-gif1.jpg@o=1.html;290.00;195.00;217.00;199.00;204.00;196.00;198.00;206.00;209.00;208.00;192.00;196.00
103;VueJS-TodoMVC/CompletingAllItems/Sync;7.700000000000728;7.920000000001892;8.084999999999127;7.904999999998836;7.875;7.770000000000437;8.470000000001164;8.239999999997963;9.080000000001746;9.32499999999709;8.199999999998909;11.145000000000437;8.694999999999709;8.05000000000291;7.955000000001746;8.284999999999854;8.755000000001019;13.940000000002328;7.915000000000873;7.724999999991269;8.764999999999418;7.67500000000291;8.230000000003201;9.794999999998254;7.975000000005821
28;myspace.com/www.myspace.com/albumart.html;14.40;13.45;13.29;13.62;13.42;14.15;13.86;14.34;14.69;14.10;13.82;14.13
104;VueJS-TodoMVC/CompletingAllItems/Async;76.39500000000044;76.3849999999984;77.89500000000407;74.52999999999884;80.47000000000116;70.84499999999935;83.55500000000029;82.67999999999302;70.48500000000058;89.43000000000757;87.96999999999935;65.97999999999956;69.34500000000116;76.38999999999942;73.79499999999825;77.125;78.13999999999942;70.52500000000146;109.8949999999968;74.17999999999302;90.55999999999949;73.16500000000087;87.32499999999709;74.79499999999825;101.33999999999651
29;naver.com/www.naver.com/index.html;9.15;8.31;9.40;9.89;7.29;8.43;8.87;8.77;8.96;8.24;8.16;8.21
105;VueJS-TodoMVC/CompletingAllItems;84.09500000000116;84.30500000000029;85.9800000000032;82.43499999999767;88.34500000000116;78.61499999999978;92.02500000000146;90.91999999999098;79.56500000000233;98.75500000000466;96.16999999999825;77.125;78.04000000000087;84.44000000000233;81.75;85.40999999999985;86.89500000000044;84.46500000000378;117.80999999999767;81.90499999998428;99.32499999999891;80.84000000000378;95.55500000000029;84.58999999999651;109.31500000000233
30;noimpactman.typepad.com/noimpactman.typepad.com/index.html;7.27;7.14;7.70;7.86;7.43;6.95;7.30;7.58;7.51;7.95;7.43;7.05
106;VueJS-TodoMVC/DeletingAllItems/Sync;11.359999999998763;17.534999999999854;14.515000000006694;18.864999999997963;12.355000000010477;15.965000000000146;23.279999999998836;12.25;12.775000000001455;10.629999999990105;12.2549999999992;12.534999999999854;10.784999999996217;12.784999999996217;18.69000000000233;10.43499999999949;11.56499999999869;10.569999999999709;11.060000000004948;10.200000000011642;17.375;13.044999999998254;11.74500000000262;14.735000000000582;10.460000000006403
31;page.renren.com/page.renren.com/index.html;7.94;8.13;6.76;7.77;6.93;6.60;7.62;7.61;6.88;7.56;7.55;7.48
107;VueJS-TodoMVC/DeletingAllItems/Async;22.30999999999949;22.849999999998545;22.139999999999418;22.93000000000029;21.92499999998836;23.644999999998618;27.610000000000582;22.839999999996508;36.18499999999767;23.47499999999127;24.05500000000029;40.654999999998836;27.025000000001455;24.885000000002037;34.23500000000058;24.8799999999992;30.8849999999984;22.290000000000873;27.104999999995925;26.569999999992433;27.834999999999127;21.599999999998545;26.375;23.875;22.519999999989523
32;people.com.cn/people.com.cn/index.html;11.92;9.22;8.49;8.55;8.34;8.49;6.91;9.92;8.69;8.63;7.69;9.34
108;VueJS-TodoMVC/DeletingAllItems;33.669999999998254;40.3849999999984;36.65500000000611;41.794999999998254;34.279999999998836;39.60999999999876;50.88999999999942;35.08999999999651;48.95999999999913;34.10499999998137;36.30999999999949;53.18999999999869;37.80999999999767;37.669999999998254;52.92500000000291;35.31499999999869;42.44999999999709;32.86000000000058;38.16500000000087;36.770000000004075;45.20999999999913;34.6449999999968;38.12000000000262;38.61000000000058;32.979999999995925
33;rakuten.co.jp/www.rakuten.co.jp/index.html;11.10;7.13;8.68;7.85;8.37;7.91;6.74;8.27;8.55;8.93;7.15;9.02
109;VueJS-TodoMVC;237.98000000000138;249.14500000000044;235.77500000000873;248.5199999999968;239.875;256.8899999999994;279.7400000000016;254.0799999999872;254.22499999999854;254.05499999999302;259.41999999999825;260.80999999999403;261.6450000000041;237.87000000000262;273.8350000000064;236.41499999999905;263.0749999999971;273.0849999999991;282.2099999999991;235.66999999999825;261.1349999999966;228.80500000000393;251.3050000000003;250.2899999999936;270.63499999999476
34;reddit.com/www.reddit.com/index.html;6.38;7.95;6.84;7.04;6.96;7.15;8.05;7.71;8.13;7.13;6.60;7.53
110;jQuery-TodoMVC/Adding100Items/Sync;364.2950000000001;372.13500000000204;326.41999999999825;343.97499999999854;346.06999999999243;349.35000000000036;343.2400000000016;347.3249999999971;343.7099999999991;337.55500000000757;401.91499999999905;339.9850000000006;330.63499999999476;344.8249999999971;339.3099999999977;338.9499999999989;355.02000000000044;356.1800000000003;346.34999999999854;500.0850000000064;334.15999999999985;368.5849999999991;336.5650000000023;338.5900000000038;462.3350000000064
35;reuters.com/www.reuters.com/index.html;7.51;7.25;6.60;6.98;7.41;6.45;7.61;7.46;6.11;7.15;7.05;6.94
111;jQuery-TodoMVC/Adding100Items/Async;25.354999999999563;19.159999999999854;52.104999999995925;18.764999999999418;28.97500000000582;35.68000000000029;29.465000000000146;24.200000000004366;28.640000000006694;35.81500000000233;21.625;18.149999999997817;18.05000000000291;26.084999999999127;26.889999999999418;48;23.020000000000437;18.464999999996508;28.340000000003783;19.07499999999709;37.23499999999876;18.915000000000873;45.6449999999968;21.335000000006403;19.710000000006403
36;slideshare.net/www.slideshare.net/jameswillamor/lolcats-in-popular-culture-a-historical-perspective.html;7.20;6.32;6.80;6.87;6.29;6.45;7.18;6.92;6.57;7.41;7.08;6.51
112;jQuery-TodoMVC/Adding100Items;389.64999999999964;391.2950000000019;378.5249999999942;362.73999999999796;375.04499999999825;385.03000000000065;372.70500000000175;371.52500000000146;372.3500000000058;373.3700000000099;423.53999999999905;358.1349999999984;348.6849999999977;370.9099999999962;366.1999999999971;386.9499999999989;378.0400000000009;374.6449999999968;374.6900000000023;519.1600000000035;371.3949999999986;387.5;382.2099999999991;359.9250000000102;482.0450000000128
37;sohu.com/www.sohu.com/index.html;11.72;9.64;8.85;7.12;7.96;9.14;7.76;8.19;7.14;7.68;8.08;7.24
113;jQuery-TodoMVC/CompletingAllItems/Sync;656.9349999999995;604.6800000000003;593.7649999999994;597.5199999999968;621.6349999999948;606.6700000000001;601.1849999999977;600.9549999999945;617.3400000000038;606.3999999999942;579.9499999999989;576.2250000000022;583.4550000000017;590.2900000000009;653.4099999999889;830.505000000001;640.1500000000015;598.9599999999991;602.9200000000055;594.8450000000012;848.0150000000012;586.7950000000019;609.2299999999959;620.6150000000052;585.179999999993
38;spiegel.de/www.spiegel.de/index.html;7.24;7.30;6.64;7.01;6.74;6.70;6.36;6.84;7.86;7.08;7.12;7.40
114;jQuery-TodoMVC/CompletingAllItems/Async;23.94499999999971;31.025000000001455;76.33999999999651;20.334999999999127;19.030000000013388;32.43000000000029;34.18999999999869;22.80000000000291;28.0199999999968;36.4600000000064;19.225000000000364;18.49499999999898;18.195000000006985;32.73500000000058;18.419999999998254;18.405000000000655;17.919999999998254;27.974999999998545;56.395000000004075;18.630000000004657;26.270000000000437;19.474999999998545;70.51499999999942;21.81999999999971;17.55000000000291
39;stackoverflow.com/stackoverflow.com/questions/184618/what-is-the-best-comment-in-source-code-you-have-ever-encountered.html;7.39;5.88;7.22;6.51;7.12;6.51;6.46;6.53;6.63;6.54;6.48;6.80
115;jQuery-TodoMVC/CompletingAllItems;680.8799999999992;635.7050000000017;670.1049999999959;617.8549999999959;640.6650000000081;639.1000000000004;635.3749999999964;623.7549999999974;645.3600000000006;642.8600000000006;599.1749999999993;594.7200000000012;601.6500000000087;623.0250000000015;671.8299999999872;848.9100000000017;658.0699999999997;626.9349999999977;659.3150000000096;613.4750000000058;874.2850000000017;606.2700000000004;679.7449999999953;642.435000000005;602.7299999999959
40;store.apple.com/store.apple.com/us@mco=Nzc1MjMwNA.html;6.23;7.17;7.39;8.98;7.99;8.03;9.12;8.37;8.56;7.61;8.06;7.55
116;jQuery-TodoMVC/DeletingAllItems/Sync;301.5;298.84999999999854;298.86999999999534;306.2150000000038;366.2350000000006;317.85499999999956;339.4350000000013;305.125;326.0599999999977;317.9600000000064;296.65999999999985;299.02999999999884;299.00999999999476;351.54500000000553;327.0199999999895;299.0400000000009;300.0750000000007;311.8700000000026;353.4599999999991;318;407.8850000000002;296.5599999999977;319.70499999999447;381.875;307.4349999999977
41;thepiratebay.org/thepiratebay.org/top/201.html;9.08;8.93;8.09;7.49;7.30;7.80;7.54;7.65;7.91;7.53;8.37;8.04
117;jQuery-TodoMVC/DeletingAllItems/Async;8.530000000000655;10.094999999997526;56.26000000000204;8.590000000003783;12.694999999992433;5.755000000001019;10.030000000002474;7.915000000000873;5.889999999999418;16.430000000007567;7.7849999999998545;6.005000000001019;9.404999999998836;17.755000000004657;10.75;6.3700000000008;10.375;15.770000000004075;6.629999999997381;6.44999999999709;11.139999999999418;13.659999999999854;57.419999999998254;6.099999999998545;10.599999999991269
42;tudou.com/www.tudou.com/index.html;10.06;9.38;8.68;7.37;8.57;9.11;8.20;7.91;8.78;9.64;8.11;8.47
118;jQuery-TodoMVC/DeletingAllItems;310.03000000000065;308.94499999999607;355.1299999999974;314.80500000000757;378.929999999993;323.6100000000006;349.4650000000038;313.0400000000009;331.9499999999971;334.39000000001397;304.4449999999997;305.03499999999985;308.4149999999936;369.3000000000102;337.7699999999895;305.4100000000017;310.4500000000007;327.6400000000067;360.0899999999965;324.4499999999971;419.02499999999964;310.2199999999975;377.1249999999927;387.97499999999854;318.03499999998894
43;uol.com.br/www.uol.com.br/index.html;9.04;9.49;9.48;9.31;8.68;8.41;9.16;8.91;9.49;8.37;9.77;8.73
119;jQuery-TodoMVC;1380.5599999999995;1335.9449999999997;1403.7599999999875;1295.4000000000015;1394.6399999999994;1347.7400000000016;1357.545000000002;1308.3199999999997;1349.6600000000035;1350.6200000000244;1327.159999999998;1257.8899999999994;1258.75;1363.2350000000079;1375.7999999999738;1541.2700000000023;1346.5600000000013;1329.2200000000012;1394.0950000000084;1457.0850000000064;1664.705;1303.989999999998;1439.0799999999872;1390.3350000000137;1402.8099999999977
44;w3.org/www.w3.org/standards/webdesign/htmlcss.html;6.62;5.98;6.87;6.47;7.22;6.05;6.42;6.50;7.47;7.18;5.82;7.11
120;Preact-TodoMVC/Adding100Items/Sync;15.204999999999927;24.610000000000582;24.43000000000029;22.330000000001746;26.70999999999185;15.7549999999992;24.389999999999418;20.215000000003783;23.49000000000524;15.22500000000582;14.985000000000582;15.555000000000291;18.325000000004366;25.8949999999968;14.770000000004075;15.525000000001455;21.465000000000146;15.970000000001164;18.94000000000233;16.019999999989523;15.725000000000364;25.885000000002037;14.945000000006985;14.875;24.110000000000582
45;wsj.com/online.wsj.com/home-page.html;7.49;8.57;6.84;8.12;7.60;7.24;8.16;8.22;6.81;8.28;8.11;8.58
121;Preact-TodoMVC/Adding100Items/Async;58.0049999999992;58.48999999999796;108.47999999999593;60.31499999999505;62.870000000009895;63.85499999999956;57.7599999999984;63.88500000000204;81.09999999999854;63.2899999999936;62.034999999999854;73.15000000000146;77.92500000000291;60.73500000000058;58.53499999998894;58.33499999999913;64.88500000000204;68.36000000000058;61.92000000000553;76.46499999999651;88.75;63.775000000001455;111.65000000000146;71.19499999999971;88.68499999999767
46;xinhuanet.com/xinhuanet.com/index.html;13.66;9.21;10.09;9.56;8.99;10.29;10.24;8.91;11.23;10.82;9.64;10.11
122;Preact-TodoMVC/Adding100Items;73.20999999999913;83.09999999999854;132.90999999999622;82.6449999999968;89.58000000000175;79.60999999999876;82.14999999999782;84.10000000000582;104.59000000000378;78.51499999999942;77.02000000000044;88.70500000000175;96.25000000000728;86.62999999999738;73.30499999999302;73.86000000000058;86.35000000000218;84.33000000000175;80.86000000000786;92.48499999998603;104.47500000000036;89.66000000000349;126.59500000000844;86.06999999999971;112.79499999999825
47;xunlei.com/xunlei.com/index.html;8.99;8.16;8.82;8.37;7.01;8.48;7.98;8.69;8.10;8.10;7.10;6.41
123;Preact-TodoMVC/CompletingAllItems/Sync;9.610000000000582;9.340000000000146;9.404999999998836;11.009999999994761;10.214999999996508;11.520000000000437;10.18999999999869;9.444999999999709;10.395000000004075;10.845000000001164;10.354999999999563;10.075000000000728;10.494999999995343;10.235000000000582;9.375;10.18999999999869;9.950000000000728;9.764999999999418;9.989999999997963;9.940000000002328;12.574999999998909;9.354999999999563;9.480000000003201;11.569999999999709;12.665000000008149
48;yelp.com/www.yelp.com/biz/alexanders-steakhouse-cupertino.html;8.18;7.45;7.01;8.14;7.12;7.82;8.24;7.13;7.00;6.41;6.85;7.31
124;Preact-TodoMVC/CompletingAllItems/Async;35.335000000000946;30.215000000000146;32.42500000000291;43.705000000001746;33.904999999998836;50.94499999999971;42.91500000000087;30.9800000000032;33.31500000000233;35.13499999999476;32.76999999999862;33.340000000000146;30.229999999995925;38.93499999999767;41.625;31.44000000000051;36.75;37.83499999999913;33.17500000000291;27.83499999999185;32.849999999998545;30.600000000002183;28.55999999999767;33.00500000000466;30.19000000000233
49;youku.com/www.youku.com/index.html;12.21;10.29;10.37;10.34;8.40;9.82;9.23;9.91;9.64;9.91;8.90;10.23
125;Preact-TodoMVC/CompletingAllItems;44.94500000000153;39.55500000000029;41.830000000001746;54.71499999999651;44.11999999999534;62.465000000000146;53.10499999999956;40.42500000000291;43.7100000000064;45.979999999995925;43.12499999999818;43.41500000000087;40.72499999999127;49.169999999998254;51;41.6299999999992;46.70000000000073;47.599999999998545;43.16500000000087;37.77499999999418;45.42499999999745;39.955000000001746;38.04000000000087;44.575000000004366;42.85500000001048
50;youtube.com/www.youtube.com/music.html;9.90;9.06;9.29;9.17;8.85;8.77;9.83;9.21;9.29;10.09;9.69;8.64
126;Preact-TodoMVC/DeletingItems/Sync;12.155000000000655;6.9099999999998545;6.889999999999418;6.150000000001455;7.319999999992433;14.270000000000437;14.140000000003056;8.020000000004075;7.344999999993888;7.7850000000034925;12.819999999999709;7.31499999999869;7.309999999997672;8.909999999996217;11.135000000009313;6.424999999999272;6.595000000001164;6.290000000000873;12.190000000002328;6.7850000000034925;7.014999999999418;6.229999999999563;6.3950000000040745;10.594999999993888;8.770000000004075
</pre>
127;Preact-TodoMVC/DeletingItems/Async;12.410000000001673;17.700000000000728;67.28000000000611;16.655000000006112;18.470000000001164;17.704999999999927;28.700000000000728;15.00499999999738;16.50499999999738;19.535000000003492;14.350000000000364;15.364999999997963;19.150000000001455;16.205000000001746;11;12.385000000000218;11.770000000000437;12.12000000000262;20.299999999995634;6.809999999997672;6.764999999999418;16.030000000002474;6.440000000002328;11.989999999997963;18.69999999999709
|}
128;Preact-TodoMVC/DeletingItems;24.56500000000233;24.610000000000582;74.17000000000553;22.805000000007567;25.789999999993597;31.975000000000364;42.84000000000378;23.025000000001455;23.84999999999127;27.320000000006985;27.170000000000073;22.679999999996653;26.459999999999127;25.114999999997963;22.135000000009313;18.80999999999949;18.3650000000016;18.410000000003492;32.48999999999796;13.595000000001164;13.779999999998836;22.260000000002037;12.835000000006403;22.58499999999185;27.470000000001164
 
129;Preact-TodoMVC;142.72000000000298;147.26499999999942;248.9100000000035;160.16500000000087;159.4899999999907;174.04999999999927;178.09500000000116;147.5500000000102;172.15000000000146;151.81500000000233;147.3149999999987;154.79999999999927;163.43499999999767;160.9149999999936;146.44000000000233;134.29999999999927;151.4150000000045;150.34000000000378;156.5150000000067;143.85499999998137;163.67999999999665;151.87500000000728;177.47000000001572;153.22999999999593;183.1200000000099
==== Possible regression causes ====
130;Inferno-TodoMVC/Adding100Items/Sync;126.0099999999984;133.8100000000013;139.45499999999447;128.51000000000204;132.47500000000582;135.21999999999935;144.1349999999984;136.52000000000407;130.47000000000116;137.07000000000698;132.65999999999985;134.09999999999854;133.9449999999997;142.1449999999968;145.49000000000524;142.53499999999985;128.76500000000306;139.93499999999767;138.20999999999913;131.15499999999884;130.96500000000015;127.7300000000032;146.9300000000003;136.5300000000061;132
Some examples of things that cause regressions in this test are:
131;Inferno-TodoMVC/Adding100Items/Async;16.965000000000146;34.715000000000146;36.424999999995634;21.299999999995634;18.195000000006985;22.854999999999563;27.040000000000873;39.15999999999622;30.614999999997963;32.354999999995925;33.590000000000146;30.589999999996508;29.345000000001164;21.709999999999127;19.389999999999418;25.73999999999978;17.19499999999971;18.125;34.23999999999796;27.504999999990105;27.75;18.724999999998545;17.3949999999968;17.034999999996217;30.10000000000582
* Increased displayport size (which causes a larger display list to be built)
132;Inferno-TodoMVC/Adding100Items;142.97499999999854;168.52500000000146;175.8799999999901;149.80999999999767;150.6700000000128;158.0749999999989;171.17499999999927;175.6800000000003;161.08499999999913;169.4250000000029;166.25;164.68999999999505;163.29000000000087;163.85499999999593;164.88000000000466;168.27499999999964;145.96000000000276;158.05999999999767;172.4499999999971;158.65999999998894;158.71500000000015;146.45500000000175;164.3249999999971;153.56500000000233;162.10000000000582
* Slowdown in the building of display list
133;Inferno-TodoMVC/CompletingAllItems/Sync;144.375;160.22000000000116;173.22000000000116;152.77999999999884;156.7100000000064;154.8150000000005;129.1899999999987;154.75;130.63500000000204;152.72000000000116;130.91500000000087;149.90499999999884;156.22499999999854;162.76000000000204;145.4649999999965;129.0650000000005;157.0649999999987;154.5699999999997;156.8050000000003;151.60499999999593;129.4449999999997;146.60999999999694;120.52500000000146;159.7699999999968;147.44000000000233
* Slowdown in rasterization of content
134;Inferno-TodoMVC/CompletingAllItems/Async;24.460000000000946;42.05500000000029;35.12000000000262;129.40499999999884;17.910000000003492;59.75;25.775000000001455;40.45499999999447;19.285000000003492;9.584999999991851;32.30500000000029;31.165000000000873;27.80500000000029;168.125;6.255000000004657;13.364999999999782;12.784999999999854;6.3400000000037835;9.064999999995052;20.414999999993597;13.635000000000218;11.829999999998108;20.935000000004948;23.17500000000291;12.494999999995343
 
135;Inferno-TodoMVC/CompletingAllItems;168.83500000000095;202.27500000000146;208.34000000000378;282.1849999999977;174.6200000000099;214.5650000000005;154.96500000000015;195.20499999999447;149.92000000000553;162.30499999999302;163.22000000000116;181.0699999999997;184.02999999999884;330.88500000000204;151.72000000000116;142.4300000000003;169.84999999999854;160.9100000000035;165.86999999999534;172.01999999998952;143.07999999999993;158.43999999999505;141.4600000000064;182.9449999999997;159.93499999999767
=== tp6 ===
136;Inferno-TodoMVC/DeletingItems/Sync;598.1900000000005;566.8400000000001;628.6800000000003;575.2350000000006;556.7149999999965;806.4950000000008;576.1649999999972;602.3050000000003;575.4649999999965;606.6499999999942;790.4950000000008;577.8100000000013;595.8099999999977;599.0749999999971;590.5399999999936;583.6100000000006;564.9550000000017;605.8250000000044;591.7150000000038;586.0399999999936;566.1049999999996;562.375;584.5250000000015;571.695000000007;573.9599999999919
* contact: :rwood, :jmaher, :bsmedberg
137;Inferno-TodoMVC/DeletingItems/Async;38.784999999999854;74.45999999999913;7;43.71499999999651;7.605000000010477;17.295000000000073;10.544999999998254;58.729999999995925;6.365000000005239;35.04000000000815;6.779999999998836;16.770000000000437;21.400000000001455;47.04000000000087;16.105000000010477;7.364999999999782;9.43999999999869;31.81500000000233;10.12999999999738;9.880000000004657;15.295000000000073;36.5;16.459999999999127;37.74499999999534;17.31500000000233
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/quantum_pageload tp6]
138;Inferno-TodoMVC/DeletingItems;636.9750000000004;641.2999999999993;635.6800000000003;618.9499999999971;564.320000000007;823.7900000000009;586.7099999999955;661.0349999999962;581.8300000000017;641.6900000000023;797.2749999999996;594.5800000000017;617.2099999999991;646.114999999998;606.6450000000041;590.9750000000004;574.3950000000004;637.6400000000067;601.8450000000012;595.9199999999983;581.3999999999996;598.875;600.9850000000006;609.4400000000023;591.2749999999942
* type: [https://wiki.mozilla.org/Buildbot/Talos/Tests#Page_Load_Tests PageLoader]
139;Inferno-TodoMVC;948.7849999999999;1012.1000000000022;1019.8999999999942;1050.9449999999924;889.6100000000297;1196.4300000000003;912.8499999999949;1031.919999999991;892.8350000000064;973.4199999999983;1126.7450000000008;940.3399999999965;964.5299999999988;1140.854999999996;923.2450000000099;901.6800000000003;890.2050000000017;956.6100000000079;940.1649999999936;926.5999999999767;883.1949999999997;903.7699999999968;906.7700000000041;945.9500000000044;913.3099999999977
* reporting: intervals in ms (lower is better)
140;Elm-TodoMVC/Adding100Items/Sync;305.84000000000015;310.3500000000022;312.47000000000116;317.6200000000026;299.5650000000023;304.28000000000065;323.7350000000006;354.11500000000524;312.3050000000003;322.0749999999971;327.6700000000001;357.6650000000009;314.6949999999997;304.36499999999796;308.27999999999884;319.7000000000007;305.7649999999994;314.4650000000038;322.31499999999505;327.8799999999901;319.6399999999994;300.70500000000175;296.4050000000061;309.54499999999825;318.41999999999825
* data: we load each of the tp6 pages 25 times
141;Elm-TodoMVC/Adding100Items/Async;20.350000000000364;35.904999999998836;47.724999999998545;51.83499999999913;49.104999999995925;43.92500000000109;31.385000000002037;18.729999999995925;22.214999999996508;29.739999999990687;20.290000000000873;25.534999999999854;22.19999999999709;73;38.21499999999651;18.13500000000022;200.40499999999884;23.43000000000029;23.114999999997963;19.64999999999418;20.51000000000022;46.3650000000016;35.18999999999505;29.510000000002037;21.44000000000233
* summarization:
142;Elm-TodoMVC/Adding100Items;326.1900000000005;346.255000000001;360.1949999999997;369.45500000000175;348.66999999999825;348.20500000000175;355.1200000000026;372.84500000000116;334.5199999999968;351.8149999999878;347.96000000000095;383.2000000000007;336.8949999999968;377.36499999999796;346.49499999999534;337.83500000000095;506.16999999999825;337.8950000000041;345.429999999993;347.5299999999843;340.14999999999964;347.07000000000335;331.59500000000116;339.0550000000003;339.8600000000006
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first five] data points, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 20; [http://searchfox.org/mozilla-central/rev/8a61c71153a79cda2e1ae7d477564347c607cc5f/testing/talos/talos/test.py#246 source: test.py]
143;Elm-TodoMVC/CompletingAllItems/Sync;290.65499999999884;304.84999999999854;376.7350000000006;312.4150000000009;309.61500000000524;301.2999999999993;425.2400000000016;469.5400000000009;318.375;316.54499999999825;421.89000000000124;305.83500000000276;297.90499999999884;309.02999999999884;291.0399999999936;314.1650000000009;299.8099999999977;316.5599999999977;299.09500000000116;338.91999999999825;297.6749999999993;305.6850000000013;416.2450000000026;300.70500000000175;311.2749999999942
** suite: identical to subtest; each tp6_* test is reported individually (they're NOT aggregated)
144;Elm-TodoMVC/CompletingAllItems/Async;15.274999999999636;41.62999999999738;27.685000000004948;28.19999999999709;36.92500000000291;17.729999999999563;17.095000000001164;20.964999999996508;26.470000000001164;20.729999999995925;18.639999999999418;17.7599999999984;21.68000000000029;77.58499999999185;63.63499999999476;27.05000000000109;54.43999999999869;19.75499999999738;14.440000000002328;27.61500000000524;23.05000000000109;75.04000000000087;18.715000000003783;25.139999999999418;30.45000000001164
 
145;Elm-TodoMVC/CompletingAllItems;305.9299999999985;346.4799999999959;404.42000000000553;340.61499999999796;346.54000000000815;319.02999999999884;442.33500000000276;490.5049999999974;344.84500000000116;337.2749999999942;440.53000000000065;323.59500000000116;319.5849999999991;386.6149999999907;354.67499999998836;341.21500000000196;354.24999999999636;336.31499999999505;313.5350000000035;366.5350000000035;320.72500000000036;380.7250000000022;434.9600000000064;325.84500000000116;341.7250000000058
The talos tp6 (also known as 'quantum-pageload') test suite loads various pages and reports their load times. Each test page/URL is reported individually. The result values (ms) is the 'time to first non-blank paint'.
146;Elm-TodoMVC/DeletingItems/Sync;249.29500000000007;267.8899999999994;264.2450000000026;261.1499999999942;282.54999999998836;267.60499999999956;278.6100000000006;262.0049999999974;268.0149999999994;262.63499999999476;256.97999999999956;280.65000000000146;278.5349999999962;271.31999999999243;254.02499999999418;292.1700000000001;260.4750000000022;278.66999999999825;257.79999999999563;296.9100000000035;266.1399999999994;259.3999999999978;442;264.8350000000064;272.5650000000023
 
147;Elm-TodoMVC/DeletingItems/Async;15.635000000000218;14.850000000002183;16.50499999999738;15.275000000008731;10.84000000001106;8.090000000000146;43.93500000000131;9.20499999999447;12.75;18.56500000000233;9.905000000000655;10.24500000000262;8.930000000000291;231;20.570000000006985;13.965000000000146;49.875;10.839999999996508;8.269999999996799;8.19999999999709;13.699999999998909;60.405000000002474;13.544999999998254;15.009999999994761;10.479999999995925
Instead of loading the page sets from a local file, a tool called [http://docs.mitmproxy.org/en/latest/introduction.html mitmproxy] is used to playback page archives via an https proxy. When the tp6 suite is run via ./mach talos-test, mozharness automatically downloads and installs the mitmproxy tool and pageset archives.
148;Elm-TodoMVC/DeletingItems;264.9300000000003;282.7400000000016;280.75;276.4250000000029;293.3899999999994;275.6949999999997;322.5450000000019;271.20999999999185;280.7649999999994;281.1999999999971;266.8850000000002;290.8950000000041;287.4649999999965;502.31999999999243;274.59500000000116;306.1350000000002;310.3500000000022;289.50999999999476;266.06999999999243;305.1100000000006;279.8399999999983;319.8050000000003;455.54499999999825;279.84500000000116;283.04499999999825
 
149;Elm-TodoMVC;897.0499999999993;975.4749999999985;1045.3650000000052;986.4950000000026;988.6000000000058;942.9300000000003;1120.0000000000073;1134.5599999999904;960.1299999999974;970.289999999979;1055.3750000000018;997.690000000006;943.9449999999924;1266.299999999981;975.7649999999849;985.1850000000031;1170.7699999999968;963.7199999999939;925.0349999999889;1019.1749999999884;940.7149999999983;1047.6000000000058;1222.1000000000058;944.7450000000026;964.6300000000047
The mitmproxy tool is started, the proxy setup in Firefox, and the mitmproxy https security certificate is loaded into Firefox, all automatically. The pagesets are made available for playback via the proxy, at which point each of the tp6 test pages are ready for load and measurement.
150;Flight-TodoMVC/Adding100Items/Sync;136.78000000000065;143.5600000000013;134.66999999999825;138.38000000000466;138.74000000000524;135.96500000000015;131.82500000000437;140.47500000000582;132.15000000000873;132.5850000000064;132.70499999999993;133.35499999999956;134.71500000000378;137.02499999999418;136.7850000000035;132.9300000000003;139.55999999999767;131.11499999999796;139.84000000001106;134.65999999998894;136.50500000000102;142.97000000000116;140.9300000000003;132.61999999999534;139.9100000000035
151;Flight-TodoMVC/Adding100Items/Async;37.56999999999971;23.700000000000728;20.825000000004366;31.185000000012224;44.86000000000058;21.670000000000073;45.76499999999942;28.32499999999709;32.75;18.80999999999767;26.960000000000946;54.659999999999854;20.80500000000029;32.74000000000524;31.56500000000233;22.735000000000582;20.270000000000437;38.63999999999942;18.235000000000582;22.895000000004075;23.23999999999978;17.80999999999767;38.55000000000291;28.904999999998836;18.414999999993597
152;Flight-TodoMVC/Adding100Items;174.35000000000036;167.26000000000204;155.49500000000262;169.56500000001688;183.60000000000582;157.63500000000022;177.59000000000378;168.8000000000029;164.90000000000873;151.39500000000407;159.66500000000087;188.01499999999942;155.52000000000407;169.76499999999942;168.35000000000582;155.66500000000087;159.8299999999981;169.75499999999738;158.07500000001164;157.55499999999302;159.7450000000008;160.77999999999884;179.4800000000032;161.52499999999418;158.3249999999971
153;Flight-TodoMVC/CompletingAllItems/Sync;130.51500000000124;135.38000000000102;115.40499999999884;198.59999999999127;150.05499999999302;123.46500000000015;126.97499999999854;125.84000000000378;130.18499999999767;120.81999999999243;119.02499999999964;129.78499999999985;121.94000000000233;119.05000000000291;129.3000000000029;121.40999999999985;119.27999999999884;121.24500000000262;132.65499999999884;128.9499999999971;126.07500000000073;143.8849999999984;124.23500000000058;126.56000000001222;117.94499999999243
154;Flight-TodoMVC/CompletingAllItems/Async;19.484999999998763;60.54000000000087;20.715000000003783;34.07499999999709;30.610000000000582;20.11499999999978;25.95499999999447;21.010000000002037;37.58499999999185;25.375;32.32500000000073;54.095000000001164;32.63500000000204;19.40999999998894;29.645000000004075;22.534999999999854;20.20499999999811;48.359999999993306;21.875;26.57500000001164;24.894999999998618;18.040000000000873;47.65999999999622;27.92500000000291;23.369999999995343
155;Flight-TodoMVC/CompletingAllItems;150;195.9200000000019;136.12000000000262;232.67499999998836;180.6649999999936;143.57999999999993;152.92999999999302;146.85000000000582;167.76999999998952;146.19499999999243;151.35000000000036;183.88000000000102;154.57500000000437;138.45999999999185;158.94500000000698;143.9449999999997;139.48499999999694;169.60499999999593;154.52999999999884;155.52500000000873;150.96999999999935;161.92499999999927;171.8949999999968;154.48500000001513;141.31499999998778
156;Flight-TodoMVC/DeletingItems/Sync;104.86499999999978;120.29000000000087;110.06999999999971;107.19999999999709;101.55000000000291;99.44500000000153;108.51499999999942;106.78500000000349;102.66000000000349;107.36500000000524;105.46500000000015;103.22000000000116;109.83499999999913;100.46999999998661;106.92500000000291;100.94499999999971;98.97000000000116;119.27500000000146;96.86500000000524;105.19499999999243;102.6200000000008;110.17499999999927;109.18999999999505;99.49499999999534;113.125
157;Flight-TodoMVC/DeletingItems/Async;11.319999999999709;20.919999999998254;9.834999999999127;16.089999999996508;18.03499999998894;7.440000000000509;9.959999999999127;8.00499999999738;31.070000000006985;7.184999999997672;7.369999999998981;50.099999999998545;10.92500000000291;7.444999999992433;23.264999999999418;12.06500000000051;14.510000000002037;13.180000000000291;8.565000000002328;7.150000000008731;7.4099999999998545;22.349999999998545;18.510000000002037;19.014999999999418;10.134999999994761
158;Flight-TodoMVC/DeletingItems;116.18499999999949;141.20999999999913;119.90499999999884;123.2899999999936;119.58499999999185;106.88500000000204;118.47499999999854;114.79000000000087;133.73000000001048;114.55000000000291;112.83499999999913;153.3199999999997;120.76000000000204;107.91499999997905;130.19000000000233;113.01000000000022;113.4800000000032;132.45500000000175;105.43000000000757;112.34500000000116;110.03000000000065;132.52499999999782;127.69999999999709;118.50999999999476;123.25999999999476
159;Flight-TodoMVC;440.53499999999985;504.39000000000306;411.5200000000041;525.5299999999988;483.84999999999127;408.1000000000022;448.99499999999534;430.4400000000096;466.40000000000873;412.1399999999994;423.85000000000036;525.2150000000001;430.8550000000105;416.1399999999703;457.48500000001513;412.6200000000008;412.79499999999825;471.81499999999505;418.03500000001804;425.4250000000029;420.7450000000008;455.2299999999959;479.0749999999971;434.5200000000041;422.8999999999796
</pre>
|}


==== Possible regression causes ====
The tp6 suite currently consists of these individual pageload tests:
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.


=== V8, version 7===
{| cellspacing="1" cellpadding="1" border="1"
* contact: :h4writer, :jmaher
|-
* source: [[https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/v8_7 v8.manifest]]
| Talos test name
* type: PageLoader
| Description
* measuring: ???
|-
* reporting: weighted score (higher is better)
| tp6_amazon
* data: there are 10 subtests in kraken, each subtest is an internal benchmark and generates a single replicate.
| Measures page load of an amazon search result for the keyword "laptop"
* summarization:
|-
** subtest: For all of the 10 data points, we take filter with [https://wiki.mozilla.org/Buildbot/Talos/Data#v8_subtest v8_subtest] to apply the reference value for the subtest to generate a single number. [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l489 source: test.py]
| tp6_facebook
** suite: [https://wiki.mozilla.org/Buildbot/Talos/Data#v8_metric v8 metric] is basically the geometric mean of the 10 subtests and multiples the output by 100.
| Measures page load of a facebook timeline (post-login)
 
|-
this is the V8 (version 7) javascript benchmark taken verbatim and slightly modified to fit into our pageloader extension and talos harness.
| tp6_google
 
| Measures page load of a google search result for the keywords "barack obama"
The previous version of this test is V8 version 5 which was run on selective branches and operating systems.
|-
| tp6_youtube
| Measures page load of the youtube home page
|}


{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! Example data
! Example data for tp6_youtube
|-
|-
| <pre>
| <pre>
0;Richards;152.99877600979192
https://www.youtube.co/;409;441;354;337;326;375;353;650;354;712;667;336;791;497;342;355;611;334;
1;DeltaBlue;210.7481559536354
692;320;714;434;582;610;333
2;Encrypt;291.29041654529567
</pre>
3;Decrypt;5358.288770053476
|}
4;RayTrace;1833.3333333333333
5;Earley;678.42605156038
6;Boyer;10265.30612244898
7;RegExp;28333.333333333332
8;Splay;528.2620179609086
9;NavierStokes;5055.555555555556
</pre>
|}


==== Possible regression causes ====
==== Possible regression causes ====
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
Some examples of things that cause regressions in this test are:
* Please update this when regressions are found


=== TART/CART ===
=== tpaint ===
* contact: :avih, :jmaher, :MattN
* contact: :jimm, :jmaher
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/tart tart]
* source: [[https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/startup_test/tpaint.html tpaint-window.html]]
* type: PageLoader
* type: Startup
* measuring: Desktop Firefox UI animation speed and smoothness
* data: we load the tpaint test window  20 times, resulting in 1 set of 20 data points.
* reporting: intervals in ms (lower is better) - see below for details
* data: there are 30 reported subtests from TART which we load 25 times, resulting in 30 sets of 25 data points.
* summarization:
* summarization:
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] data point, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 24 data points; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l305 source: test.py]
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] '''5''' data points, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 15; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l190 source: test.py]
** suite: [https://wiki.mozilla.org/Buildbot/Talos/Data#geometric_mean geometric mean] of the 30 subtest results.
** suite: identical to subtest


TART is the '''Tab Animation Regression Test''' and CART is the '''Customize Animation Regression Test'''.
{| cellspacing="1" cellpadding="1" border="1"
|-
|Talos test name
|Description
|-
|tpaint
|twinopen but measuring the time after we receive the [[Buildbot/Talos/Tests#Paint_Tests|MozAfterPaint and OnLoad event]].
|}


TART tests tab animation on these cases:
Tests the amount of time it takes the open a new window. This test does not include startup time. Multiple test windows are opened in succession, results reported are the average amount of time required to create and display a window in the running instance of the browser. (Measures ctrl-n performance.)
* Simple: single new tab of about:blank open/close without affecting (shrinking/expanding) other tabs.
* icon: same as above with favicons and long title instead of about:blank.
* Newtab: newtab open with thumbnails preview - without affecting other tabs, with and without preload.
* Fade: opens a tab, then measures fadeout/fadein (tab animation without the overhead of opening/closing a tab).
** Case 1 is tested with DPI scaling of 1.
** Case 2 is tested with DPI scaling of 1.0 and 2.0.
** Case 3 is tested with the default scaling of the test system.
** Case 4 is tested with DPI scaling of 2.0 with the "icon" tab (favicon and long title).
** Each animation produces 3 test results:
*** error: difference between the designated duration and the actual completion duration from the trigger.
*** half: average frame interval over the 2nd half of the animation.
*** all: average frame interval over all recorded intervals.
*** And the run logs also include the explicit intervals from which these 3 values were derived.


CART uses the same framework to measure performance of the Australize "customize" animation (for entering the toolbar/menu customization view). Subtests include:
{| class="mw-collapsible mw-collapsed wikitable"
* Customize-enter animation (full and css-animation-only part).
! Example data
* Customize-exit animation
|-
| <pre>
[209.219, 222.180, 225.299, 225.970, 228.090, 229.450, 230.625, 236.315, 239.804, 242.795, 244.5, 244.770, 250.524, 251.785, 253.074, 255.349, 264.729, 266.014, 269.399, 326.190]
</pre>
|}


TART/CART can be used as a stand-alone addon:
==== Possible regression causes ====
* Set the browser to ASAP mode (preferences layout.frame_rate=0, docshell.event_starvation_delay_hint=1). This makes the browser refresh the screen as fast as possible instead of limiting it to 60hz, thus allows higher resolution measurements. Requires restart to take effect.
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
* [https://hg.mozilla.org/mozilla-central/raw-file/tip/testing/talos/talos/generate-tart-xpi.html Install the latest addon xpi], or zip the addon dir of the source code, rename the extension to xpi and install.
* Visit chrome://tart/content/tart.html
* Select subtests to run. By default the selected tests are all the TART tests. CART is the "Customize" test.


{| class="mw-collapsible mw-collapsed wikitable"
=== tps ===
! Example data
* contact: :blassey, :mconley
|-
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/tabswitch tabswitch]
| <pre>
* bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1166132
0;simple-open-DPI1.half.TART;2.35;2.42;2.63;2.47;2.71;2.38;2.37;2.41;2.48;2.70;2.44;2.41;2.51;2.43;2.41;2.56;2.76;2.49;2.36;2.40;2.70;2.53;2.35;2.46;2.47
* type: Pageloader
1;simple-open-DPI1.all.TART;2.80;2.95;3.12;2.92;3.46;2.87;2.92;2.99;2.89;3.24;2.94;2.95;3.25;2.92;3.02;3.00;3.21;3.31;2.84;2.87;3.10;3.13;3.10;2.94;2.95
* measuring:
2;simple-open-DPI1.error.TART;33.60;36.33;35.93;35.97;38.17;34.77;36.00;35.01;36.25;36.22;35.24;35.76;36.64;36.31;34.74;34.40;34.34;41.48;35.04;34.83;34.27;34.04;34.37;35.22;36.52
**  The time between switching a tab and the first paint to the content area
3;simple-close-DPI1.half.TART;1.95;1.88;1.91;1.94;2.00;1.97;1.88;1.76;1.84;2.18;1.99;1.83;2.04;1.93;1.81;1.77;1.79;1.90;1.82;1.84;1.78;1.75;1.76;1.89;1.81
* reporting:
4;simple-close-DPI1.all.TART;2.19;2.08;2.07;2.32;2.65;2.32;2.26;1.96;2.02;2.26;2.05;2.16;2.19;2.11;2.04;1.98;2.05;2.02;2.01;2.11;1.97;1.97;2.05;2.01;2.12
* data: we load 50 web pages 5 times each, resulting in 50 sets of 5 data points.
5;simple-close-DPI1.error.TART;21.35;23.87;22.60;22.02;22.97;22.35;22.15;22.79;21.81;21.90;22.26;22.58;23.15;22.43;22.76;23.36;21.86;22.70;22.96;22.70;22.28;22.03;21.78;22.33;22.23
* '''To run it locally''', you'd need [[#tp5n_pages_set|tp5n.zip]].
6;icon-open-DPI1.half.TART;2.42;2.33;2.50;2.58;2.36;2.51;2.60;2.35;2.52;2.51;2.59;2.34;3.29;2.63;2.46;2.57;2.53;2.50;2.39;2.51;2.44;2.66;2.72;2.36;2.52
* summarization:
7;icon-open-DPI1.all.TART;3.12;2.94;3.42;3.23;3.10;3.21;3.33;3.14;3.24;3.32;3.46;2.90;3.65;3.19;3.27;3.47;3.32;3.13;2.95;3.23;3.21;3.33;3.47;3.15;3.32
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] data point, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 4 data points; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l305 source: test.py]
8;icon-open-DPI1.error.TART;38.39;37.96;37.03;38.85;37.03;37.17;37.19;37.56;36.67;36.33;36.89;36.85;37.54;38.46;35.38;37.52;36.68;36.48;36.03;35.71;37.12;37.08;37.74;38.09;35.85
** suite: [https://wiki.mozilla.org/Buildbot/Talos/Data#geometric_mean geometric mean] of the 50 subtest results.
9;icon-close-DPI1.half.TART;1.94;1.93;1.79;1.89;1.83;1.83;1.90;1.89;1.75;1.76;1.77;1.74;1.81;1.86;1.95;1.99;1.73;1.83;1.97;1.80;1.94;1.84;2.01;1.88;2.03
 
10;icon-close-DPI1.all.TART;2.14;2.14;1.98;2.03;2.02;2.25;2.29;2.13;1.97;2.01;1.94;2.01;1.99;2.05;2.11;2.09;2.02;2.02;2.12;2.02;2.20;2.11;2.19;2.07;2.27
{| class="mw-collapsible mw-collapsed wikitable"
11;icon-close-DPI1.error.TART;24.51;25.03;25.17;24.54;23.86;23.70;24.02;23.61;24.10;24.53;23.92;23.75;23.73;23.78;23.42;25.40;24.21;24.55;23.96;24.96;24.41;24.96;24.16;24.20;23.65
! Example data
12;icon-open-DPI2.half.TART;2.60;2.60;2.60;2.53;2.51;2.53;2.59;2.43;2.66;2.60;2.47;2.61;2.64;2.43;2.49;2.63;2.61;2.60;2.52;3.06;2.65;2.74;2.69;2.44;2.43
|-
13;icon-open-DPI2.all.TART;3.45;3.22;3.38;3.47;3.10;3.31;3.47;3.13;3.37;3.14;3.28;3.20;3.40;3.15;3.14;3.23;3.41;3.16;3.26;3.55;3.29;3.49;3.44;3.11;3.22
| <pre>
14;icon-open-DPI2.error.TART;40.20;37.86;37.53;41.46;37.03;38.77;37.48;36.97;37.28;37.72;36.09;36.71;38.89;38.21;37.37;38.91;36.79;36.10;37.60;36.99;37.56;35.76;38.92;37.46;37.52
0;amazon.com/www.amazon.com/Kindle-Wireless-Reader-Wifi-Graphite/dp/B002Y27P3M/507846.html;66.34;54.15;53.08;55.79;49.12
15;icon-close-DPI2.half.TART;2.27;1.97;1.83;1.86;2.08;1.88;1.80;1.90;1.77;1.89;2.06;1.89;1.89;1.86;2.01;1.79;1.78;1.83;1.89;1.85;1.74;1.82;1.84;1.81;1.79
1;cgi.ebay.com/cgi.ebay.com/ALL-NEW-KINDLE-3-eBOOK-WIRELESS-READING-DEVICE-W-WIFI-/130496077314@pt=LH_DefaultDomain_0&hash=item1e622c1e02.html;50.85;46.57;39.51;36.71;36.47
16;icon-close-DPI2.all.TART;2.33;2.13;2.18;2.03;2.33;2.03;1.95;2.06;1.96;2.13;2.25;2.10;2.13;2.03;2.18;2.00;2.05;2.01;2.08;2.05;1.96;2.04;2.10;2.04;2.08
2;163.com/www.163.com/index.html;95.05;80.80;76.09;69.29;68.96
17;icon-close-DPI2.error.TART;22.99;23.02;24.32;23.58;23.05;23.34;22.92;23.22;22.90;23.33;23.33;23.05;22.80;23.45;24.05;22.39;22.14;22.97;22.85;22.13;22.89;22.98;23.69;22.99;23.08
3;mail.ru/mail.ru/index.html;66.21;52.04;56.33;55.11;45.61
18;iconFade-close-DPI2.half.TART;2.14;1.84;1.78;1.84;1.66;2.07;1.81;3.82;1.68;1.85;1.62;2.54;2.06;1.85;2.17;1.80;1.71;1.67;2.11;1.73;2.94;2.14;1.93;1.72;2.05
4;bbc.co.uk/www.bbc.co.uk/news/index.html;35.80;44.59;48.02;45.71;42.58
19;iconFade-close-DPI2.all.TART;2.17;1.76;1.80;1.89;1.70;1.93;1.80;3.38;1.78;1.90;1.70;2.50;1.94;1.81;2.29;1.82;1.79;1.76;2.23;1.80;2.85;2.06;1.84;1.83;2.09
5;store.apple.com/store.apple.com/us@mco=Nzc1MjMwNA.html;52.98;49.45;58.47;56.79;61.29
20;iconFade-close-DPI2.error.TART;4.67;4.11;3.69;4.51;4.46;3.88;4.54;3.68;4.56;3.82;4.32;4.87;4.42;3.72;3.72;4.54;4.93;4.46;4.64;3.39;4.09;3.28;3.58;4.11;3.80
6;imdb.com/www.imdb.com/title/tt1099212/index.html;46.51;55.12;46.22;50.60;47.63
21;iconFade-open-DPI2.half.TART;2.37;2.61;2.37;2.62;2.54;2.84;2.57;2.44;4.33;2.57;2.59;2.67;2.58;2.48;2.38;2.39;2.50;2.39;2.50;2.57;2.52;2.55;2.47;2.69;2.41
7;cnn.com/www.cnn.com/index.html;43.02;50.77;43.88;49.70;50.02
22;iconFade-open-DPI2.all.TART;2.45;2.64;2.39;2.60;2.57;2.60;2.61;2.59;3.14;2.55;2.54;2.66;2.57;2.48;2.47;2.46;2.55;2.45;2.51;2.61;2.54;2.58;2.50;2.54;2.40
8;sohu.com/www.sohu.com/index.html;74.03;62.89;63.30;67.71;89.35
23;iconFade-open-DPI2.error.TART;3.64;4.67;4.31;5.79;6.43;3.64;4.82;8.68;5.78;4.38;3.80;3.98;4.64;653.63;4.63;3.76;4.23;5.01;5.48;4.99;3.48;5.10;5.02;6.14;5.58
9;youku.com/www.youku.com/index.html;43.98;52.69;45.80;63.00;57.02
24;newtab-open-preload-no.half.TART;5.02;2.90;3.06;3.03;2.94;2.94;3.08;3.12;3.60;3.19;2.82;2.96;3.67;7.85;2.79;3.12;3.18;2.92;2.86;2.96;2.96;3.00;2.90;2.97;2.94
10;ifeng.com/ifeng.com/index.html;88.01;87.54;104.47;94.93;113.91
25;newtab-open-preload-no.all.TART;7.11;4.66;5.03;4.68;4.50;4.58;4.76;4.76;5.67;4.96;4.36;4.51;5.21;11.16;4.38;4.69;4.77;4.45;4.45;4.70;4.51;4.61;4.54;4.69;4.60
11;tudou.com/www.tudou.com/index.html;45.32;48.10;46.03;39.26;58.38
26;newtab-open-preload-no.error.TART;40.82;40.85;37.38;37.40;36.30;36.47;36.89;37.63;37.12;38.65;36.73;36.95;36.11;38.59;37.39;37.77;37.93;37.54;37.46;38.29;36.58;38.25;38.32;37.92;36.93
12;chemistry.about.com/chemistry.about.com/index.html;38.24;37.07;39.59;39.48;39.60
27;newtab-open-preload-yes.half.TART;3.14;2.96;2.97;8.37;2.98;3.00;2.96;3.05;3.12;3.48;3.07;3.23;3.05;2.88;2.92;3.06;2.90;3.01;3.19;2.90;3.18;3.11;3.04;3.16;3.21
13;beatonna.livejournal.com/beatonna.livejournal.com/index.html;35.59;50.75;36.17;48.49;52.61
28;newtab-open-preload-yes.all.TART;5.10;4.60;4.63;8.94;5.01;4.69;4.63;4.67;4.93;5.43;4.78;5.12;4.77;4.65;4.50;4.78;4.75;4.63;4.76;4.45;4.86;4.88;4.69;4.86;4.92
14;rakuten.co.jp/www.rakuten.co.jp/index.html;90.28;71.95;62.66;60.33;67.76
29;newtab-open-preload-yes.error.TART;35.90;37.24;38.57;40.60;36.04;38.12;38.78;36.73;36.91;36.69;38.12;36.69;37.79;35.80;36.11;38.01;36.59;38.85;37.14;37.30;38.02;38.95;37.64;37.86;36.43
15;uol.com.br/www.uol.com.br/index.html;42.89;48.05;53.77;40.02;42.41
</pre>
16;thepiratebay.org/thepiratebay.org/top/201.html;40.46;42.46;47.63;57.66;45.49
|}
17;page.renren.com/page.renren.com/index.html;47.61;66.78;47.91;62.78;47.19
 
18;chinaz.com/chinaz.com/index.html;50.34;58.17;118.43;55.47;63.80
==== Possible regression causes ====
19;globo.com/www.globo.com/index.html;41.34;38.52;42.82;53.14;45.20
* CART
20;spiegel.de/www.spiegel.de/index.html;33.60;34.34;36.25;36.18;47.04
** Regressions might be reported for patches that add new buttons to the menu panel by default, since this can add new rows to the panel and make it scrollable, which increases the amount of work to do during the transition. See {{bug|1230671}} for details.
21;dailymotion.com/www.dailymotion.com/us.html;37.68;36.13;39.52;37.15;42.79
 
22;goo.ne.jp/goo.ne.jp/index.html;50.74;47.30;63.04;58.41;58.96
=== sessionrestore/sessionrestore_no_auto_restore/sessionrestore_many_windows ===
23;stackoverflow.com/stackoverflow.com/questions/184618/what-is-the-best-comment-in-source-code-you-have-ever-encountered.html;44.66;44.40;43.39;47.38;57.65
* contact: :Yoric, :mikedeboer
24;ezinearticles.com/ezinearticles.com/index.html@Migraine-Ocular---The-Eye-Migraines&id=4684133.html;37.38;45.01;40.29;36.26;39.28
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/startup_test/sessionrestore talos/sessionrestore]
25;huffingtonpost.com/www.huffingtonpost.com/index.html;39.57;43.35;55.01;44.13;58.28
* bug: https://bugzilla.mozilla.org/show_bug.cgi?id=936630, https://bugzilla.mozilla.org/show_bug.cgi?id=1331937
26;media.photobucket.com/media.photobucket.com/image/funny%20gif/findstuff22/Best%20Images/Funny/funny-gif1.jpg@o=1.html;39.77;42.46;75.54;42.38;47.72
* type: Startup
27;imgur.com/imgur.com/gallery/index.html;34.72;37.37;46.74;40.93;37.08
* measuring: time spent reading and restoring the session.
28;reddit.com/www.reddit.com/index.html;42.47;39.89;51.54;51.51;41.68
* reporting: interval in ms (lower is better).
29;noimpactman.typepad.com/noimpactman.typepad.com/index.html;54.28;47.40;52.38;52.15;50.97
* data: we load the session restore index page 10 times to collect 1 set of 10 data points.
30;myspace.com/www.myspace.com/albumart.html;48.97;64.12;61.66;48.32;68.53
* summarization:
31;mashable.com/mashable.com/index.html;36.76;40.95;35.30;53.86;42.76
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] data point, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 9 data points; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l305 source: test.py]
32;dailymail.co.uk/www.dailymail.co.uk/ushome/index.html;42.06;40.64;44.24;37.32;61.35
** suite: identical to subtest
33;whois.domaintools.com/whois.domaintools.com/mozilla.com.html;34.73;35.23;39.25;48.24;35.72
 
34;indiatimes.com/www.indiatimes.com/index.html;52.67;55.51;46.29;52.69;58.82
Three tests measure the time spent reading and restoring the session from a valid sessionstore.js. Time starts ticking during event sessionRestoreInit and stops during event sessionRestored.
35;reuters.com/www.reuters.com/index.html;32.92;33.08;36.95;39.23;39.27
 
36;xinhuanet.com/xinhuanet.com/index.html;125.85;102.56;138.89;130.34;147.45
In ''sessionrestore'', this is tested with a configuration that requires the session to be restored. In ''sessionrestore_no_auto_restore'', this is tested with a configuration that requires the session to not be restored. Both of the above tests use a sessionstore.js file that contains one window and roughly 89 tabs. In ''sessionrestore_many_windows'', this is tested with a sessionstore.js that contains 3 windows and 130 tabs. The first window contains 50 tabs, 80 remaning tabs are divided equally between the second and the third window.
37;56.com/www.56.com/index.html;63.89;75.00;61.45;62.20;58.67
 
38;bild.de/www.bild.de/index.html;35.61;43.74;34.79;33.45;31.83
{| class="mw-collapsible mw-collapsed wikitable"
39;guardian.co.uk/www.guardian.co.uk/index.html;38.91;55.93;62.34;42.63;45.99
! Example data
40;naver.com/www.naver.com/index.html;78.10;89.07;127.67;75.18;109.32
|-
41;yelp.com/www.yelp.com/biz/alexanders-steakhouse-cupertino.html;42.54;46.92;39.19;49.82;50.43
| <pre>
42;wsj.com/online.wsj.com/home-page.html;46.43;55.51;44.16;81.79;48.78
[2362.0, 2147.0, 2171.0, 2134.0, 2116.0, 2145.0, 2141.0, 2141.0, 2136.0, 2080.0]
43;google.com/www.google.com/search@q=mozilla.html;35.62;36.71;44.47;45.00;40.22
44;xunlei.com/xunlei.com/index.html;67.57;60.69;83.83;85.53;85.08
45;aljazeera.net/aljazeera.net/portal.html;65.03;51.84;73.29;64.77;69.70
46;w3.org/www.w3.org/standards/webdesign/htmlcss.html;53.57;58.50;72.98;66.95;55.62
47;homeway.com.cn/www.hexun.com/index.html;105.59;117.32;108.95;116.10;70.32
48;youtube.com/www.youtube.com/music.html;40.53;41.48;59.67;40.81;40.07
49;people.com.cn/people.com.cn/index.html;96.49;103.64;115.12;66.05;117.84
</pre>
</pre>
|}
|}
Line 1,372: Line 1,377:
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.


=== tps ===
=== tresize ===
* contact: :blassey, :mconley
* contact: :jimm, :jmaher, :avih
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/tabswitch tabswitch]
* source: [[https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/startup_test/tresize/addon/content/tresize-test.html tresize-test.html]]
* bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1166132
* type: StartupTest
* type: Pageloader
* measuring: Time to do XUL resize, in ms (lower is better).
* measuring:  
* data: we run the tresize test page 20 times, resulting in 1 set of 20 data points.
**  The time between switching a tab and the first paint to the content area
* reporting:
* data: we load 50 web pages 5 times each, resulting in 50 sets of 5 data points.
* '''To run it locally''', you'd need [[#tp5n_pages_set|tp5n.zip]].
* summarization:
* summarization:
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] data point, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 4 data points; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l305 source: test.py]
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] '''5''' data points, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 15 data points; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l205 source: test.py]
** suite: [https://wiki.mozilla.org/Buildbot/Talos/Data#geometric_mean geometric mean] of the 50 subtest results.
** suite: same as subtest result
 
A purer form of paint measurement than tpaint. This test opens a single window positioned at 10,10 and sized to 300,300, then resizes the window outward |max| times measuring the amount of time it takes to repaint each resize. Dumps the resulting dataset and average to stdout or logfile.
 
In {{bug|1102479}} tresize was rewritten to work in e10s mode which involved a full rewrite of the test.
 
To run resize locally without talos, please [https://hg.mozilla.org/mozilla-central/raw-file/tip/testing/talos/talos/generate-tresize-xpi.html install the addon] to run the test locally.


{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
Line 1,390: Line 1,397:
|-
|-
| <pre>
| <pre>
0;amazon.com/www.amazon.com/Kindle-Wireless-Reader-Wifi-Graphite/dp/B002Y27P3M/507846.html;66.34;54.15;53.08;55.79;49.12
[23.2565333333333, 23.763383333333362, 22.58369999999999, 22.802766666666653, 22.304050000000025, 23.010383333333326, 22.865466666666677, 24.233716666666705, 24.110983333333365, 22.21390000000004, 23.910333333333316, 23.409816666666647, 19.873049999999992, 21.103966666666686, 20.389749999999978, 20.777349999999984, 20.326283333333365, 22.341616666666667, 20.29813333333336, 20.769600000000104]
1;cgi.ebay.com/cgi.ebay.com/ALL-NEW-KINDLE-3-eBOOK-WIRELESS-READING-DEVICE-W-WIFI-/130496077314@pt=LH_DefaultDomain_0&hash=item1e622c1e02.html;50.85;46.57;39.51;36.71;36.47
</pre>
2;163.com/www.163.com/index.html;95.05;80.80;76.09;69.29;68.96
|}
3;mail.ru/mail.ru/index.html;66.21;52.04;56.33;55.11;45.61
 
4;bbc.co.uk/www.bbc.co.uk/news/index.html;35.80;44.59;48.02;45.71;42.58
==== Possible regression causes ====
5;store.apple.com/store.apple.com/us@mco=Nzc1MjMwNA.html;52.98;49.45;58.47;56.79;61.29
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
6;imdb.com/www.imdb.com/title/tt1099212/index.html;46.51;55.12;46.22;50.60;47.63
 
7;cnn.com/www.cnn.com/index.html;43.02;50.77;43.88;49.70;50.02
 
8;sohu.com/www.sohu.com/index.html;74.03;62.89;63.30;67.71;89.35
=== ts_paint ===
9;youku.com/www.youku.com/index.html;43.98;52.69;45.80;63.00;57.02
 
10;ifeng.com/ifeng.com/index.html;88.01;87.54;104.47;94.93;113.91
* contact: :mak, :jimm, :jmaher
11;tudou.com/www.tudou.com/index.html;45.32;48.10;46.03;39.26;58.38
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/startup_test/tspaint_test.html tspaint_test.html]
12;chemistry.about.com/chemistry.about.com/index.html;38.24;37.07;39.59;39.48;39.60
* Perfomatic: "Ts, Paint"
13;beatonna.livejournal.com/beatonna.livejournal.com/index.html;35.59;50.75;36.17;48.49;52.61
* type: Startup
14;rakuten.co.jp/www.rakuten.co.jp/index.html;90.28;71.95;62.66;60.33;67.76
* data: 20 times we start the browser and time how long it takes to paint the startup test page, resulting in 1 set of 20 data points.
15;uol.com.br/www.uol.com.br/index.html;42.89;48.05;53.77;40.02;42.41
* summarization:
16;thepiratebay.org/thepiratebay.org/top/201.html;40.46;42.46;47.63;57.66;45.49
** subtest: identical to suite
17;page.renren.com/page.renren.com/index.html;47.61;66.78;47.91;62.78;47.19
** suite: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] data point, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 19 data points; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l135 source: test.py]
18;chinaz.com/chinaz.com/index.html;50.34;58.17;118.43;55.47;63.80
 
19;globo.com/www.globo.com/index.html;41.34;38.52;42.82;53.14;45.20
Starts the browser to display tspaint_test.html with the start time in the url, waits for [[Buildbot/Talos/Tests#Paint_Tests|MozAfterPaint and onLoad]] to fire, then records the end time and calculates the time to startup.
20;spiegel.de/www.spiegel.de/index.html;33.60;34.34;36.25;36.18;47.04
 
21;dailymotion.com/www.dailymotion.com/us.html;37.68;36.13;39.52;37.15;42.79
{| class="mw-collapsible mw-collapsed wikitable"
22;goo.ne.jp/goo.ne.jp/index.html;50.74;47.30;63.04;58.41;58.96
! Example data
23;stackoverflow.com/stackoverflow.com/questions/184618/what-is-the-best-comment-in-source-code-you-have-ever-encountered.html;44.66;44.40;43.39;47.38;57.65
|-
24;ezinearticles.com/ezinearticles.com/index.html@Migraine-Ocular---The-Eye-Migraines&id=4684133.html;37.38;45.01;40.29;36.26;39.28
| <pre>
25;huffingtonpost.com/www.huffingtonpost.com/index.html;39.57;43.35;55.01;44.13;58.28
[1666.0, 1195.0, 1139.0, 1198.0, 1248.0, 1224.0, 1213.0, 1194.0, 1229.0, 1196.0, 1191.0, 1230.0, 1247.0, 1169.0, 1217.0, 1184.0, 1196.0, 1192.0, 1224.0, 1192.0]
26;media.photobucket.com/media.photobucket.com/image/funny%20gif/findstuff22/Best%20Images/Funny/funny-gif1.jpg@o=1.html;39.77;42.46;75.54;42.38;47.72
</pre>
27;imgur.com/imgur.com/gallery/index.html;34.72;37.37;46.74;40.93;37.08
|}
28;reddit.com/www.reddit.com/index.html;42.47;39.89;51.54;51.51;41.68
 
29;noimpactman.typepad.com/noimpactman.typepad.com/index.html;54.28;47.40;52.38;52.15;50.97
==== Possible regression causes ====
30;myspace.com/www.myspace.com/albumart.html;48.97;64.12;61.66;48.32;68.53
* (and/or maybe tpaint?) will regress if a new <panel> element is added to the browser window (e.g. browser.xul) and it's frame gets created. Fix this by ensuring it's display:none by default.
31;mashable.com/mashable.com/index.html;36.76;40.95;35.30;53.86;42.76
 
32;dailymail.co.uk/www.dailymail.co.uk/ushome/index.html;42.06;40.64;44.24;37.32;61.35
=== tscrollx ===
33;whois.domaintools.com/whois.domaintools.com/mozilla.com.html;34.73;35.23;39.25;48.24;35.72
* contact: :jrmuizel, :jmaher, :avih
34;indiatimes.com/www.indiatimes.com/index.html;52.67;55.51;46.29;52.69;58.82
* source: [[https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/scroll scroll.manifest]]
35;reuters.com/www.reuters.com/index.html;32.92;33.08;36.95;39.23;39.27
* type: PageLoader
36;xinhuanet.com/xinhuanet.com/index.html;125.85;102.56;138.89;130.34;147.45
* measuring: Scroll performance
37;56.com/www.56.com/index.html;63.89;75.00;61.45;62.20;58.67
* reporting: Average frame interval (1/FPS). Lower is better.
38;bild.de/www.bild.de/index.html;35.61;43.74;34.79;33.45;31.83
* data: we load 6 pages 25 times each, collecting 6 sets of 25 data points
39;guardian.co.uk/www.guardian.co.uk/index.html;38.91;55.93;62.34;42.63;45.99
* summarization:
40;naver.com/www.naver.com/index.html;78.10;89.07;127.67;75.18;109.32
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] data point, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 24; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l623 source: test.py]
41;yelp.com/www.yelp.com/biz/alexanders-steakhouse-cupertino.html;42.54;46.92;39.19;49.82;50.43
** suite: [https://wiki.mozilla.org/Buildbot/Talos/Data#geometric_mean geometric mean] of the 6 subtest results.
42;wsj.com/online.wsj.com/home-page.html;46.43;55.51;44.16;81.79;48.78
 
43;google.com/www.google.com/search@q=mozilla.html;35.62;36.71;44.47;45.00;40.22
{| cellspacing="1" cellpadding="1" border="1"
44;xunlei.com/xunlei.com/index.html;67.57;60.69;83.83;85.53;85.08
|-
45;aljazeera.net/aljazeera.net/portal.html;65.03;51.84;73.29;64.77;69.70
| Talos test name
46;w3.org/www.w3.org/standards/webdesign/htmlcss.html;53.57;58.50;72.98;66.95;55.62
| Description
47;homeway.com.cn/www.hexun.com/index.html;105.59;117.32;108.95;116.10;70.32
|-
48;youtube.com/www.youtube.com/music.html;40.53;41.48;59.67;40.81;40.07
|tscrollx
49;people.com.cn/people.com.cn/index.html;96.49;103.64;115.12;66.05;117.84
|[https://groups.google.com/d/topic/mozilla.dev.platform/RICw5SJhNMo/discussion Replacing tscroll,tsvg with tscrollx,tsvgx]
</pre>
|}
|}


==== Possible regression causes ====
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
=== Microbench ===
* contact: :benwa
* source: [[https://dxr.mozilla.org/mozilla-central/source/testing/gtest/mozilla/MozGTestBench.cpp MozGTestBench.cpp]]
* type: Custom GTest micro-benchmarking
* data: Time taken for a GTest function to execute
* summarization: Not a Talos test. This suite is provides a way to add low level platform performance regression tests for things that are not suited to be tested by Talos. See the [[https://wiki.mozilla.org/Buildbot/Talos/Sheriffing#Microbench_Policy Microbench Sheriffing Policy]] for some notes on how to treat regressions.


==== Possible regression causes ====
This test scrolls several pages where each represent a different known "hard" case to scroll (* needinfo), and measures the average frames interval (1/FPS) on each. The ASAP test (tscrollx) iterates in unlimited frame-rate mode thus reflecting the maximum scroll throughput per page.  To turn on ASAP mode, we set these preferences:
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
preferences = {'layout.frame_rate': 0, 'docshell.event_starvation_delay_hint': 1}


=== tabpaint ===
{| class="mw-collapsible mw-collapsed wikitable"
* contact: :mconley
! Example data
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/tabpaint tabpaint]
|-
* bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1253382
| <pre>
* type: Pageloader
0;tiled.html;5.41;5.57;5.34;5.64;5.53;5.48;5.44;5.49;5.50;5.50;5.49;5.66;5.50;5.37;5.57;5.54;5.46;5.31;5.41;5.57;5.50;5.52;5.71;5.31;5.44
* measuring:
fixed.html;10.404609053497941;10.47;10.66;10.45;10.73;10.79;10.64;10.64;10.82;10.43;10.92;10.47;10.47;10.64;10.74;10.67;10.40;10.83;10.77;10.54;10.38;10.70;10.44;10.38;10.56
** The time it takes to paint the content of a newly opened tab when the tab is opened from the parent (ex: by hitting Ctrl-T)
downscale.html;5.493209876543211;5.27;5.50;5.50;5.51;5.46;5.58;5.58;5.51;5.49;5.49;5.47;9.09;5.56;5.61;5.50;5.47;5.59;5.47;5.49;5.60;5.61;5.58;5.40;5.43
** The time it takes to paint the content of a newly opened tab when the tab is opened from content (ex: by clicking on a target="_blank" link)
downscale.html;10.676522633744854;10.82;10.79;10.41;10.75;10.91;10.52;10.61;10.50;10.55;10.80;10.17;10.68;10.41;10.42;10.41;10.58;10.28;10.56;10.66;10.68;10.47;10.60;10.61;10.26
* '''NOT''' measuring:
4;iframe.svg;13.82;14.87;14.78;14.35;14.73;14.50;14.15;14.46;14.80;14.48;15.10;14.93;14.77;14.52;14.08;15.01;14.58;14.52;15.23;14.35;14.72;14.28;14.30;14.27;14.96
** The time it takes to animate the tabs. That's the responsibility of the TART test. tabpaint is strictly concerned with the painting of the web content.
5;reader.htm;10.72;10.62;10.23;10.48;10.42;10.64;10.40;10.40;10.14;10.60;10.51;10.36;10.57;10.41;10.52;10.75;10.19;10.72;10.44;9.75;10.49;10.07;10.54;10.46;10.44
* data: we load the tabpaint trigger page 20 times, each run produces two values (the time it takes to paint content when opened from the parent, and the time it takes to paint content when opened from content), resulting in 2 sets of 20 data points.
** Example:
{| class="mw-collapsible mw-collapsed wikitable"
! Example data
|-
| <pre>
0;tabpaint-from-parent;105;76;66;64;64;69;65;63;70;68;64;60;65;63;54;61;64;67;61;64
1; tabpaint-from-content;129;68;72;72;70;78;86;85;82;79;120;92;76;80;74;82;76;89;77;85
</pre>
</pre>
|}
|}
* summarization:
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] data point, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 19 data points
** suite: geometric_mean(subtests)


==== Possible regression causes ====
==== Possible regression causes ====
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
=== tsvgr_opacity ===
* contact: :jwatt, :jmaher, :avih
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/svg_opacity/svg_opacity.manifest]
* type: PageLoader
* data: we load the 2 svg opacity pages 25 times, resulting in 2 sets of 25 data points
* summarization:
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] '''5''' data points, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 20; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l623 source: test.py]
** suite: [https://wiki.mozilla.org/Buildbot/Talos/Data#geometric_mean geometric mean] of the 2 subtest results.


=== Other data ===
{| cellspacing="1" cellpadding="1" border="1"
 
|-
These are not part of the Talos code, but like Talos they are benchmarks that record data using the graphserver and are analyzed by the same scripts for regressions.
| Talos test name
 
| Description
==== Number of Constructors (num_ctors) ====
|-
|tsvgr_opacity
|[[#Row_Major_vs._Column_Major|Row Major]] and 25 cycles/page.
|}


This test runs at build time and measures the number of static initializers in the compiled code.  Reducing this number is helpful for [https://blog.mozilla.org/tglek/2010/05/27/startup-backward-constructors/ startup optimizations].
Renders many semi-transparent, partially overlapping SVG rectangles, and measures time to completion of this rendering.


* https://hg.mozilla.org/build/tools/file/348853aee492/buildfarm/utils/count_ctors.py
Note that this test also tends to reflect changes in network efficiency and navigation bar rendering issues:
** these are run for linux 32+64 opt and pgo builds.


===== Possible regression causes =====
* Most of the page load tests measure from before the location is changed, until onload + mozafterpaint, therefore any changes in chrome performance from the location change, or network performance (the pages load from a local web server) would affect page load times. SVG opacity is rather quick by itself, so any such chrome/network/etc performance changes would affect this test more than other page load tests (relatively, in percentages).
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.


=== tp6 ===
{| class="mw-collapsible mw-collapsed wikitable"
* contact: :rwood, :jmaher, :bsmedberg
! Example data
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/quantum_pageload tp6]
|-
* type: [https://wiki.mozilla.org/Buildbot/Talos/Tests#Page_Load_Tests PageLoader]
| <pre>
* reporting: intervals in ms (lower is better)
0;big-optimizable-group-opacity-2500.svg;170;171;205;249;249;244;192;252;192;431;182;250;189;249;151;168;209;194;247;250;193;250;255;247;247
* data: we load each of the tp6 pages 25 times
1;small-group-opacity-2500.svg;585;436;387;441;512;438;440;380;443;391;450;386;459;383;445;388;450;436;485;443;383;438;528;444;441
* summarization:
</pre>
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first five] data points, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 20; [http://searchfox.org/mozilla-central/rev/8a61c71153a79cda2e1ae7d477564347c607cc5f/testing/talos/talos/test.py#246 source: test.py]
|}
** suite: identical to subtest; each tp6_* test is reported individually (they're NOT aggregated)


The talos tp6 (also known as 'quantum-pageload') test suite loads various pages and reports their load times. Each test page/URL is reported individually. The result values (ms) is the 'time to first non-blank paint'.
==== Possible regression causes ====
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.


Instead of loading the page sets from a local file, a tool called [http://docs.mitmproxy.org/en/latest/introduction.html mitmproxy] is used to playback page archives via an https proxy. When the tp6 suite is run via ./mach talos-test, mozharness automatically downloads and installs the mitmproxy tool and pageset archives.


The mitmproxy tool is started, the proxy setup in Firefox, and the mitmproxy https security certificate is loaded into Firefox, all automatically. The pagesets are made available for playback via the proxy, at which point each of the tp6 test pages are ready for load and measurement.
=== tsvg_static ===
* contact: :jwatt, :jmaher, :dholbert, :neerja
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/svg_static/ svg_static]
* type: PageLoader
* data: we load the 5 svg pages 25 times, resulting in 5 sets of 25 data points
* summarization:
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] '''5''' data points, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 20; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l623 source: test.py]
** suite: [https://wiki.mozilla.org/Buildbot/Talos/Data#geometric_mean geometric mean] of the 5 subtest results.


The tp6 suite currently consists of these individual pageload tests:
{| cellspacing="1" cellpadding="1" border="1"
 
{| cellspacing="1" cellpadding="1" border="1"
|-
|-
| Talos test name
| Talos test name
| Description
| Description
|-
|-
| tp6_amazon
|tsvg_static
| Measures page load of an amazon search result for the keyword "laptop"
|An svg-only number that measures SVG rendering performance of some complex (but static) SVG content.
|-
| tp6_facebook
| Measures page load of a facebook timeline (post-login)
|-
| tp6_google
| Measures page load of a google search result for the keywords "barack obama"
|-
| tp6_youtube
| Measures page load of the youtube home page
|}
|}


{| class="mw-collapsible mw-collapsed wikitable"
{| class="mw-collapsible mw-collapsed wikitable"
! Example data for tp6_youtube
! Example data
|-
|-
| <pre>
| <pre>
https://www.youtube.co/;409;441;354;337;326;375;353;650;354;712;667;336;791;497;342;355;611;334;
0;gearflowers.svg;262;184;184;198;197;187;181;186;177;192;196;194;194;186;195;190;237;193;188;182;188;196;191;194;184
692;320;714;434;582;610;333
1;composite-scale.svg;69;52;48;49;57;51;52;87;52;49;49;51;58;53;64;57;49;65;67;58;53;59;56;68;50
2;composite-scale-opacity.svg;72;53;91;54;51;58;60;46;51;57;59;58;66;70;57;61;47;51;76;65;52;65;64;64;63
3;composite-scale-rotate.svg;70;76;89;62;62;78;57;77;79;82;74;56;61;79;73;64;75;74;81;82;76;58;77;61;62
4;composite-scale-rotate-opacity.svg;91;60;67;84;62;66;78;69;65;68;62;73;68;63;64;71;79;77;63;80;85;65;82;76;81
</pre>
</pre>
|}
|}


==== Possible regression causes ====
==== Possible regression causes ====
Some examples of things that cause regressions in this test are:
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
* Please update this when regressions are found


=== Stylo gtest microbenchmarks ===
=== tsvgx ===
* contact: :jwatt, :jmaher, :avih
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/svgx svgx]
* type: PageLoader
* data: we load the 7 svg pages 25 times, resulting in 7 sets of 25 data points
* summarization:
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] '''5''' data points, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 20; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l623 source: test.py]
** suite: [https://wiki.mozilla.org/Buildbot/Talos/Data#geometric_mean geometric mean] of the 7 subtest results.


* contact: :bholley, :SimonSapin
{| cellspacing="1" cellpadding="1" border="1"
* source: [https://dxr.mozilla.org/mozilla-central/source/layout/style/test/gtest]
|-
* type: [https://wiki.mozilla.org/Buildbot/Talos/Tests#Microbench Microbench]
| Talos test name
* reporting: intervals in ms (lower is better)
| Description
* data: each test is run and measured 5 times
|-
* summarization: take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the 5 data points; [https://dxr.mozilla.org/mozilla-central/source/testing/gtest/mozilla/MozGTestBench.cpp#43-46 source: MozGTestBench.cpp]
|tsvgx
|SVG-ASAP
|[https://groups.google.com/d/topic/mozilla.dev.platform/RICw5SJhNMo/discussion Replacing tscroll,tsvg with tscrollx,tsvgx]
|}


An svg-only number that measures SVG rendering performance, with animations or iterations of rendering. This is an ASAP test -- i.e. it iterates in unlimited frame-rate mode thus reflecting the maximum rendering throughput of each test. The reported value is the overall duration the sequence/animation took to complete.  To turn on ASAP mode, we set these preferences:
preferences = {'layout.frame_rate': 0, 'docshell.event_starvation_delay_hint': 1}


Servo_StyleSheet_FromUTF8Bytes_Bench parses a sample stylesheet 20 times with Stylo’s CSS parser that is written in Rust. It starts from an in-memory UTF-8 string, so that I/O or UTF-16-to-UTF-8 conversion is not measured.
{| class="mw-collapsible mw-collapsed wikitable"
 
! Example data
Gecko_nsCSSParser_ParseSheet_Bench does the same with Gecko’s previous CSS parser that is written in C++, for comparison.
|-
 
| <pre>
Servo_DeclarationBlock_SetPropertyById_Bench parses the string "10px" with Stylo’s CSS parser and sets it as the value of a property in a declaration block, a million times. This is similar to animations that are based on JavaScript code modifying Element.style instead of using CSS @keyframes.
0;hixie-001.xml;562;555;508;521;522;520;499;510;492;514;502;504;500;521;510;506;511;505;495;517;520;512;503;504;502
 
1;hixie-002.xml;510;613;536;530;536;522;498;505;500;504;498;529;498;509;493;512;501;506;504;499;496;505;508;511;503
Servo_DeclarationBlock_SetPropertyById_WithInitialSpace_Bench is the same, but with the string " 10px" with an initial space. That initial space is less typical of JS animations, but is almost always there in stylesheets or full declarations like "width: 10px". This microbenchmark was used to test the effect of some specific code changes. Regressions here may be acceptable if Servo_StyleSheet_FromUTF8Bytes_Bench is not affected.
2;hixie-003.xml;339;248;242;261;250;241;240;248;258;244;235;240;247;248;239;247;241;245;242;245;251;239;241;240;237
3;hixie-004.xml;709;540;538;536;540;536;552;539;535;535;543;533;536;535;545;537;537;537;537;539;538;535;536;538;536
4;hixie-005.xml;3096;3086;3003;3809;3213;3323;3143;3313;3192;3203;3225;3048;3069;3101;3189;3251;3172;3122;3266;3183;3159;3076;3014;3237;3100
5;hixie-006.xml;5586;5668;5565;5666;5668;5728;5886;5534;5484;5607;5678;5577;5745;5753;5532;5585;5506;5516;5648;5778;5894;5994;5794;5852;5810
6;hixie-007.xml;1823;1743;1739;1743;1744;1787;1802;1788;1782;1766;1787;1750;1748;1788;1766;1779;1767;1794;1758;1768;1781;1773;1765;1798;1805
</pre>
|}


==== Possible regression causes ====
==== Possible regression causes ====
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
* Did you change the dimensions of the content area? Even a little? The tsvgx test seems to be sensitive to changes like this. See {{bug|1375479}}, for example. Usually, these sorts of "regressions" aren't real regressions - they just mean that we need to re-baseline our expectations from the test.


=== TestStandardURL ===
=== V8, version 7===
 
* contact: :h4writer, :jmaher
* contact: :valentin
* source: [[https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/v8_7 v8.manifest]]
* source:
* type: PageLoader
* type:
* measuring: ???
* reporting:
* reporting: weighted score (higher is better)
* data:
* data: there are 10 subtests in kraken, each subtest is an internal benchmark and generates a single replicate.
* summarization:
* summarization:
** subtest: For all of the 10 data points, we take filter with [https://wiki.mozilla.org/Buildbot/Talos/Data#v8_subtest v8_subtest] to apply the reference value for the subtest to generate a single number. [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l489 source: test.py]
** suite: [https://wiki.mozilla.org/Buildbot/Talos/Data#v8_metric v8 metric] is basically the geometric mean of the 10 subtests and multiples the output by 100.


TestStandardURL.Perf - Runs 10K iterations on the most important getters and setters of nsStandardURL
this is the V8 (version 7) javascript benchmark taken verbatim and slightly modified to fit into our pageloader extension and talos harness.


TestStandardURL.NormalizePerf - Runs 20K x 5 iterations on nsStandardURL::NormalizeIPv4 on input that are valid IPv4 strings
The previous version of this test is V8 version 5 which was run on selective branches and operating systems.


TestStandardURL.NormalizePerfFails - Runs 20K x 5 iterations on nsStandardURL::NormalizeIPv4 on input that is not valid IPv4
{| class="mw-collapsible mw-collapsed wikitable"
 
! Example data
==== Possible regression causes ====
|-
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
| <pre>
 
0;Richards;152.99877600979192
=== PerfStrip Tests ===
1;DeltaBlue;210.7481559536354
2;Encrypt;291.29041654529567
3;Decrypt;5358.288770053476
4;RayTrace;1833.3333333333333
5;Earley;678.42605156038
6;Boyer;10265.30612244898
7;RegExp;28333.333333333332
8;Splay;528.2620179609086
9;NavierStokes;5055.555555555556
</pre>
|}


* contact: :milan
==== Possible regression causes ====
* source: https://dxr.mozilla.org/mozilla-central/source/xpcom/tests/gtest/TestStrings.cpp
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
* type: Microbench
* reporting: execution time in ms (lower is better) for 100k function calls
* data: each test run and measured 5 times
* summarization:


PerfStripWhitespace - call StripWhitespace() on 5 different test cases 20k times (each)


PerfStripCharsWhitespace - call StripChars("\f\t\r\n") on 5 different test cases 20k times (each)


PerfStripCRLF - call StripCRLF() on 5 different test cases 20k times (each)
=== xperf ===
* contact: :aklotz, :jmaher
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/xtalos xperf instrumentation]
* type: Pageloader (tp5n) / Startup
* measuring: IO counters from windows (currently, only startup IO is in scope)
* reporting: Summary of read/write counters for disk, network (lower is better)


PerfStripCharsCRLF() - call StripChars("\r\n") on 5 different test cases 20k times (each)
Talos will turn orange for 'x' jobs on windows 7 if your changeset accesses files which are not predefined in the [[https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/xtalos/xperf_whitelist.json whitelist]] during startup; specifically, before the "[https://hg.mozilla.org/mozilla-central/file/0eebc33d8593/toolkit/components/startup/nsAppStartup.cpp#l631 sessionstore-windows-restored]" Firefox event.  If your job turns orange, you will see a list of files in Treeherder (or in the log file) which have been accessed unexpectedly (similar to this):
* TEST-UNEXPECTED-FAIL : xperf: File '{profile}\secmod.db' was accessed and we were not expecting it. DiskReadCount: 6, DiskWriteCount: 0, DiskReadBytes: 16904, DiskWriteBytes: 0
* TEST-UNEXPECTED-FAIL : xperf: File '{profile}\cert8.db' was accessed and we were not expecting it. DiskReadCount: 4, DiskWriteCount: 0, DiskReadBytes: 33288, DiskWriteBytes: 0
* TEST-UNEXPECTED-FAIL : xperf: File 'c:\$logfile' was accessed and we were not expecting it. DiskReadCount: 0, DiskWriteCount: 2, DiskReadBytes: 0, DiskWriteBytes: 32768 TEST-UNEXPECTED-FAIL : xperf: File '{profile}\secmod.db' was accessed and we were not expecting it. DiskReadCount: 6, DiskWriteCount: 0, DiskReadBytes: 16904, DiskWriteBytes: 0
* TEST-UNEXPECTED-FAIL : xperf: File '{profile}\cert8.db' was accessed and we were not expecting it. DiskReadCount: 4, DiskWriteCount: 0, DiskReadBytes: 33288, DiskWriteBytes: 0
* TEST-UNEXPECTED-FAIL : xperf: File 'c:\$logfile' was accessed and we were not expecting it. DiskReadCount: 0, DiskWriteCount: 2, DiskReadBytes: 0, DiskWriteBytes: 32768


==== Possible regression causes ====
In the case that these files are expected to be accessed during startup by your changeset, then we can add them to the [https://bugzilla.mozilla.org/enter_bug.cgi?product=Testing&component=Talos whitelist].
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.


=== rasterflood_svg ===
Xperf runs tp5 while collecting xperf metrics for disk IO and network IO. The providers we listen for are:
* contact: :dvander
* [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/xperf.config#l10 'PROC_THREAD', 'LOADER', 'HARD_FAULTS', 'FILENAME', 'FILE_IO', 'FILE_IO_INIT']
* source: [[https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/gfx/benchmarks/rasterflood_svg.html rasterflood_svg.html]]
* type: [https://wiki.mozilla.org/Buildbot/Talos#Page_Load_Tests PageLoader]
* data: we load the rasterflood_svg.html page ten times, measuring pageload each time, generating 10 data points.
* summarization:
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] data point, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 9; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l986 source: test.py]


This page animates some complex SVG patterns in a requestAnimationFrame callback. However, it also churns the CPU during each callback, spinning an empty loop for 14ms each frame. The intent is that, if we consider the rasterization costs to be 0, then the animation should run close to 60fps. Otherwise it will lag. Since rasterization costs are not 0, the lower we can get them, the faster the test will run. The test runs in ASAP mode to maximize framerate. The result is how quickly the browser is able to render 600 frames of the animation.
The values we collect during stackwalk are:
 
* [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/xperf.config#l11 'FileRead', 'FileWrite', 'FileFlush']
Improvements (or regressions) to general painting performance or SVG are likely to affect this benchmark.
 
==== Possible regression causes ====
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
 
=== rasterflood_gradient ===
* contact: :dvander
* source: [[https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/gfx/benchmarks/rasterflood_gradient.html rasterflood_gradient.html]]
* type: [https://wiki.mozilla.org/Buildbot/Talos#Page_Load_Tests PageLoader]
* data: we load the rasterflood_gradient.html page ten times, computing a score each time, generating 10 data points.
* summarization:
** subtest: [https://wiki.mozilla.org/Buildbot/Talos/Data#ignore_first ignore first] data point, then take the [https://wiki.mozilla.org/Buildbot/Talos/Data#median median] of the remaining 9; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l986 source: test.py]
 
This page animates some complex gradient patterns in a requestAnimationFrame callback. However, it also churns the CPU during each callback, spinning an empty loop for 14ms each frame. The intent is that, if we consider the rasterization costs to be 0, then the animation should run close to 60fps. Otherwise it will lag. Since rasterization costs are not 0, the lower we can get them, the faster the test will run. The test runs in ASAP mode to maximize framerate.
 
The test runs for 10 seconds, and the resulting score is how many frames we were able to render during that time. Higher is better. Improvements (or regressions) to general painting performance or gradient rendering will affect this benchmark.


==== Possible regression causes ====
==== Possible regression causes ====
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
Mconley
Confirmed users, Bureaucrats and Sysops emeriti
1,006

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1186204"