12
edits
Changes
Updated test descriptions--added test name where needed and missing tests, re-ordered alphabetically
== a11y ==
'''Test Name'''
a11yr
* contact: :surkov
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/a11y a11y.manifest]
* reporting: test time in ms (lower is better)
This test ensures basic a11y tables and permutations do not cause performance regressions.
</pre>
|}
== about-newtab ==
'''Test Name'''
about_newtab_with_snipppets
{{todo|''add test details''}}
== about-preferences ==
'''Test Name'''
about_preferences_basic
* contact: :jaws
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/about-preferences/about_preferences_basic.manifest]
== ARES-6 ==
'''Test Name'''
ARES6
* contact: :jandem
* source: [https://searchfox.org/mozilla-central/source/third_party/webkit/PerformanceTests/ARES-6 ARES-6]
== Basic compositor video ==
'''Test Name'''
basic_compositor_video
* contact: :davidb
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/video video]
* '''Lower is better'''
* unit: ms/frame
{| class="mw-collapsible mw-collapsed wikitable"
|}
== perf-reftest cpstartup ==* contact: :bholley* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/perf-reftest perf-reftest]* type: [[#Page_Load_Tests|PageLoader]]* reporting: intervals in ms (lower is better)* data: each test loads 25 times* summarization:** subtest: [[../Data#ignore_first|ignore first]] 5 data points, then take the [[../Data#median|median]] of the remaining 20 data points; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l734 source: test.py]** suite: identical to subtest '''Important note:''' This test now requires an 'opt' build. If the perf-reftest is ran on a non-opt build, it will time out (more specifically on innertext-1.html, and possibly others in the future). Style system performance test suite. The perf-reftest suite is a unique talos suite where each subtest loads two different test pages: a 'base' page (i.e. bloom_basic) and a 'reference' page (i.e. bloom_basic_ref), and then compares each of the page load times against eachother to determine the variance. Talos runs each of the two pages as if they are stand-alone tests, and then calculates and reports the variance; the test output 'replicates' reported from bloom_basic are actually the comparisons between the 'base' and 'reference' pages for each page load cycle. The suite contains multiple subtests, each of which contains a base page and a reference page. If you wish to see the individual 'base' and 'reference' page results instead of just the reported difference, the 'base_replicates' and 'ref_replicates' can be found in the PERFHERDER_DATA log file output, and in the 'local.json' talos output file when running talos locally. In production, both of the page replicates are also archived in the perfherder-data.json file. The perfherder-data.json file is archived after each run in production, and can be found on the Treeherder Job Details tab when the perf-reftest job symbol is selected. This test suite was ported over from the style-perf-tests (https://github.com/heycam/style-perf-tests). {| class="mw-collapsible mw-collapsed wikitable"! Example data|-| <pre>"replicates": [1.185, 1.69, 1.22, 0.36, 11.26, 3.835, 3.315, 1.355, 3.185, 2.485, 2.2, 1.01, 0.9, 1.22, 1.9,0.285, 1.52, 0.31, 2.58, 0.725, 2.31, 2.67, 3.295, 1.57, 0.3], "value": 1.7349999999999999, "unit": "ms", "base_replicates": [166.94000000000003, 165.16, 165.64000000000001, 165.04000000000002, 167.355, 165.175,165.325, 165.11, 164.175, 164.78, 165.555, 165.885, 166.83499999999998, 165.76500000000001, 164.375, 166.825,167.13, 166.425, 169.22500000000002, 164.955, 165.335, 164.45000000000002, 164.85500000000002, 165.005, 166.035]}], "ref_replicates": [165.755, 166.85000000000002, 166.85999999999999, 165.4, 178.615, 169.01, 168.64, 166.465,167.36, 167.265, 167.75500000000002, 166.895, 167.735, 166.985, 166.275, 166.54000000000002, 165.61, 166.115,166.64499999999998, 165.68, 167.64499999999998, 167.12, 168.15, 166.575, 166.335], </pre>|} == perf-reftest-singletons ==* contact: :heycam* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/perf-reftest-singletons perf-reftest-singletons]* type: [[#Page_Load_Tests|PageLoader]]* reporting: intervals in ms (lower is better)* data: each test loads 25 times* summarization:** subtest: [[../Data#ignore_first|ignore first]] 5 data points, then take the [[../Data#median|median]] of the remaining 20 data points; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l734 source: test.py]** suite: identical to subtest
* contact: :mconley
* measuring: Time from opening a new tab (which creates a new content process) to having that new content process be ready to load URLs.
== DAMP ==
'''Test Name'''
damp
* contact: :ochameau
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/devtools damp]
== displaylist_mutate ==
'''Test Name'''
displaylist_mutate
* contact: :mattwoodrow
* source: [[https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/layout/benchmarks/displaylist_mutate.html displaylist_mutate.html]]
=== Dromaeo CSS ===
'''Test Name'''
dromaeo_css
* contact: :bz
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/dromaeo css.manifest]
=== Dromaeo DOM (Linux64 only) ===
'''Test Name'''
dromaeo_dom
* contact: :bz
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/dromaeo dom.manifest]
== glterrain ==
'''Test Name'''
glterrain
* contact: :jgilbert
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/webgl/benchmarks/terrain glterrain]
* type: [[TestEngineering/Performance/Talos#Page_Load_Tests|PageLoader]]
* data: we load the perftest.html page (which generates 4 metrics to track) 25 times, resulting in 4 sets of 25 data points
* summarization:Measures average frames interval while animating a simple WebGL scene
** subtest: [[../Data#ignore_first|ignore first]] data point, then take the [[../Data#median|median]] of the remaining 24; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l381 source: test.py]
** suite: [[../Data#geometric_mean|geometric mean]] of the 4 subtest results.
This tests animates a simple WebGL scene (static textured landscape, one moving light source, rotating viewport) and measure the frames throughput (expressed as average interval) over 100 frames. It runs in ASAP mode (vsync off) and measures the same scene 4 times - for all combination of antialiasing and alpha. It reports the results as 4 values - one for each combination. Lower results are better.
== glvideo ==
'''Test Name'''
glvideo
* contact: :jgilbert
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/webgl/benchmarks/video glvideo]
* type: [[TestEngineering/Performance/Talos#Page_Load_Tests|PageLoader]]
* data: 5 cycles of the entire benchmark, each subtest will have 5 data points (see below)
* summarization:WebGL video texture update with 1080p video. Measures mean tick time across 100 ticks.
** subtest: [[../Data#ignore_first|ignore first]] data point, then take the [[../Data#median|median]] of the remaining 4; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l522 source: test.py]
** suite: [[../Data#geometric_mean|geometric mean]] of the 4 subtest results.
* '''Lower is better'''
* unit: ms/100 ticks
{| class="mw-collapsible mw-collapsed wikitable"
== jetstream ==
'''Test Name'''
JetStream
* contact: :jandem
* source: [[https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/jetstream/jetstream.manifest jetstream.manifest]] and jetstream.zip from tooltool
== kraken ==
'''Test Name'''
kraken
* contact: :sdetar
* source: [[https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/kraken kraken.manifest]]
|}
== motionmark motionmark_animometer == '''Test Name''' motionmark_animometer
* contact: :davidb
* source: [[https://searchfox.org/mozilla-central/source/third_party/webkit/PerformanceTests/MotionMark source]] [[https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/motionmark manifests]]
** subtest: FPS from the subtest, each subtest is run for 15 seconds, repeat this 5 times and report the median value
** suite: we take a geometric mean of all the subtests (9 for animometer, 11 for html suite)
== motionmark_html ==
'''Test Name'''
motionmark_html_suite
{{todo|''add test details''}}
== motionmark_webgl ==
'''Test Name'''
motionmark_webgl
* contact: :jgilbert
* source: [[https://searchfox.org/mozilla-central/source/third_party/webkit/PerformanceTests/MotionMark source]] [[https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/motionmark/webgl.manifest manifest]]
== pdfpaint ==
'''Test Name'''
pdfpaint
* contact: :bdahl
* source:
* data: load a PDF 20 times
|-
| Talos test name<pre>| Description"replicates": [1.185, 1.69, 1.22, 0.36, 11.26, 3.835, 3.315, 1.355, 3.185, 2.485, 2.2, 1.01, 0.9, 1.22, 1.9,|-0.285, 1.52, 0.31, 2.58, 0.725, 2.31, 2.67, 3.295, 1.57, 0.3], "value": 1.7349999999999999, "unit": "ms",|pdfpaint| "base_replicates": [166.94000000000003, 165.16, 165.64000000000001, 165.04000000000002, 167.355, 165.175,165.325, 165.11, 164.175, 164.78, 165.555, 165.885, 166.83499999999998, 165.76500000000001, 164.375, 166.825,167.13, 166.425, 169.22500000000002, 164.955, 165.335, 164.45000000000002, 164.85500000000002, 165.005, 166.035]}], "ref_replicates": [165.755, 166.85000000000002, 166.85999999999999, 165.4, 178.615, 169.01, 168.64, 166.465,167.36, 167.265, 167.75500000000002, 166.895, 167.735, 166.985, 166.275, 166.54000000000002, 165.61, 166.115,166.64499999999998, 165.68, 167.64499999999998, 167.12, 168.15, 166.575, 166.335], </pre>
|}
== rasterflood_svg perf-reftest-singletons == '''Test Name''' perf_reftest_singletons * contact: :rhuntheycam* source: [https://searchfoxdxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/gfx/benchmarks/rasterflood_svg.html rasterflood_svg.htmlperf-reftest-singletons perf-reftest-singletons]* type: [[TestEngineering/Performance/Talos#Page_Load_Tests|PageLoader]]* reporting: intervals in ms (lower is better)* data: we load the rasterflood_svg.html page ten each test loads 25 times, measuring pageload each time, generating 10 data points.
* summarization:
** subtest: [[../Data#ignore_first|ignore first]] 5 data pointpoints, then take the [[../Data#median|median]] of the remaining 920 data points; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l986 l734 source: test.py]** suite: identical to subtest Individual style system performance tests. The perf-reftest-singletons suite runs the perf-reftest 'base' pages (i.e. bloom_basic) test individually, and reports the values for that single test page alone, NOT the comparison of two different pages. There are multiple subtests in this suite, each just containing the base page on its own.
This page animates some complex SVG patterns in a requestAnimationFrame callback. However, it also churns the CPU during each callback, spinning an empty loop for 14ms each frame. The intent is that, if we consider the rasterization costs to be 0, then test suite was ported over from the animation should run close to 60fps. Otherwise it will lagstyle-perf-tests (https://github. Since rasterization costs are not 0, the lower we can get them, the faster the test will run. The test runs in ASAP mode to maximize framerate. The result is how quickly the browser is able to render 600 frames of the animationcom/heycam/style-perf-tests).
== rasterflood_gradient ==
The test runs for 10 seconds, and the resulting score is how many frames we were able to render during that time. Higher is better. Improvements (or regressions) to general painting performance or gradient rendering will affect this benchmark.
== rasterflood_svg ==
'''Test Name'''
rasterflood_svg
* contact: :rhunt
* source: [https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/gfx/benchmarks/rasterflood_svg.html rasterflood_svg.html]
* type: [[TestEngineering/Performance/Talos#Page_Load_Tests|PageLoader]]
* data: we load the rasterflood_svg.html page ten times, measuring pageload each time, generating 10 data points.
* summarization:
** subtest: [[../Data#ignore_first|ignore first]] data point, then take the [[../Data#median|median]] of the remaining 9; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l986 source: test.py]
This page animates some complex SVG patterns in a requestAnimationFrame callback. However, it also churns the CPU during each callback, spinning an empty loop for 14ms each frame. The intent is that, if we consider the rasterization costs to be 0, then the animation should run close to 60fps. Otherwise it will lag. Since rasterization costs are not 0, the lower we can get them, the faster the test will run. The test runs in ASAP mode to maximize framerate. The result is how quickly the browser is able to render 600 frames of the animation.
Improvements (or regressions) to general painting performance or SVG are likely to affect this benchmark.
== sessionrestore ==
'''Test Name'''
sessionrestore
* contact: :mikedeboer, :mconley, :felipe
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/startup_test/sessionrestore talos/sessionrestore]
</pre>
|}
== sessionrestore_many_windows ==
'''Test Name'''
sessionrestore_many_windows
See [[#sessionrestore]].
== sessionrestore_no_auto_restore ==
'''Test Name'''
sessionrestore_no_auto_restore
See [[#sessionrestore]].
== sessionrestore_many_windows startup_about_home_paint ==See [[#sessionrestore]].'''Test Name'''
* contact: :mconley
* source: [[https://hg.mozilla.org/mozilla-central/file/tip/testing/talos/talos/startup_test/startup_about_home_paint/addon/]]
</pre>
|}
== startup_about_home_paint_cached ==
'''Test Name'''
startup_about_home_paint_cached
{{todo|''add test details''}}
== startup_about_home_paint_realworld_webextensions ==
'''Test Name'''
startup_about_home_paint_realworld_webextensions
* contact: :mconley
* source: [[https://hg.mozilla.org/mozilla-central/file/tip/testing/talos/talos/startup_test/startup_about_home_paint/addon/]]
</pre>
|}
== stylebench ==
'''Test Name'''
stylebench
* contact: :emilio
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/stylebench/stylebench.manifest stylebench.manifest]
* type: [[TestEngineering/Performance/Talos#Page_Load_Tests|PageLoader]]
* measuring: speed of dynamic style recalculation
* reporting: runs/minute score
== tabpaint ==
'''Test Name'''
tabpaint
* contact: :mconley
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/tabpaint tabpaint]
** subtest: [[../Data#ignore_first|ignore first]] data point, then take the [[../Data#median|median]] of the remaining 19 data points
** suite: geometric_mean(subtests)
== tabswitch ==
'''Test Name'''
tabswitch
* contact: :mconley
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/tabswitch tabswitch]
* bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1166132
* type: Pageloader
* measuring:
** The time between switching a tab and the first paint to the content area
* reporting:
* data: we load 50 web pages 5 times each, resulting in 50 sets of 5 data points.
* '''To run it locally''', you'd need [[#Page_sets|tp5n.zip]].
* summarization:
** subtest: [[../Data#ignore_first|ignore first]] data point, then take the [[../Data#median|median]] of the remaining 4 data points; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l305 source: test.py]
** suite: [[../Data#geometric_mean|geometric mean]] of the 50 subtest results.
{| class="mw-collapsible mw-collapsed wikitable"
! Example data
|-
| <pre>
0;amazon.com/www.amazon.com/Kindle-Wireless-Reader-Wifi-Graphite/dp/B002Y27P3M/507846.html;66.34;54.15;53.08;55.79;49.12
1;cgi.ebay.com/cgi.ebay.com/ALL-NEW-KINDLE-3-eBOOK-WIRELESS-READING-DEVICE-W-WIFI-/130496077314@pt=LH_DefaultDomain_0&hash=item1e622c1e02.html;50.85;46.57;39.51;36.71;36.47
2;163.com/www.163.com/index.html;95.05;80.80;76.09;69.29;68.96
3;mail.ru/mail.ru/index.html;66.21;52.04;56.33;55.11;45.61
4;bbc.co.uk/www.bbc.co.uk/news/index.html;35.80;44.59;48.02;45.71;42.58
5;store.apple.com/store.apple.com/us@mco=Nzc1MjMwNA.html;52.98;49.45;58.47;56.79;61.29
6;imdb.com/www.imdb.com/title/tt1099212/index.html;46.51;55.12;46.22;50.60;47.63
7;cnn.com/www.cnn.com/index.html;43.02;50.77;43.88;49.70;50.02
8;sohu.com/www.sohu.com/index.html;74.03;62.89;63.30;67.71;89.35
9;youku.com/www.youku.com/index.html;43.98;52.69;45.80;63.00;57.02
10;ifeng.com/ifeng.com/index.html;88.01;87.54;104.47;94.93;113.91
11;tudou.com/www.tudou.com/index.html;45.32;48.10;46.03;39.26;58.38
12;chemistry.about.com/chemistry.about.com/index.html;38.24;37.07;39.59;39.48;39.60
13;beatonna.livejournal.com/beatonna.livejournal.com/index.html;35.59;50.75;36.17;48.49;52.61
14;rakuten.co.jp/www.rakuten.co.jp/index.html;90.28;71.95;62.66;60.33;67.76
15;uol.com.br/www.uol.com.br/index.html;42.89;48.05;53.77;40.02;42.41
16;thepiratebay.org/thepiratebay.org/top/201.html;40.46;42.46;47.63;57.66;45.49
17;page.renren.com/page.renren.com/index.html;47.61;66.78;47.91;62.78;47.19
18;chinaz.com/chinaz.com/index.html;50.34;58.17;118.43;55.47;63.80
19;globo.com/www.globo.com/index.html;41.34;38.52;42.82;53.14;45.20
20;spiegel.de/www.spiegel.de/index.html;33.60;34.34;36.25;36.18;47.04
21;dailymotion.com/www.dailymotion.com/us.html;37.68;36.13;39.52;37.15;42.79
22;goo.ne.jp/goo.ne.jp/index.html;50.74;47.30;63.04;58.41;58.96
23;stackoverflow.com/stackoverflow.com/questions/184618/what-is-the-best-comment-in-source-code-you-have-ever-encountered.html;44.66;44.40;43.39;47.38;57.65
24;ezinearticles.com/ezinearticles.com/index.html@Migraine-Ocular---The-Eye-Migraines&id=4684133.html;37.38;45.01;40.29;36.26;39.28
25;huffingtonpost.com/www.huffingtonpost.com/index.html;39.57;43.35;55.01;44.13;58.28
26;media.photobucket.com/media.photobucket.com/image/funny%20gif/findstuff22/Best%20Images/Funny/funny-gif1.jpg@o=1.html;39.77;42.46;75.54;42.38;47.72
27;imgur.com/imgur.com/gallery/index.html;34.72;37.37;46.74;40.93;37.08
28;reddit.com/www.reddit.com/index.html;42.47;39.89;51.54;51.51;41.68
29;noimpactman.typepad.com/noimpactman.typepad.com/index.html;54.28;47.40;52.38;52.15;50.97
30;myspace.com/www.myspace.com/albumart.html;48.97;64.12;61.66;48.32;68.53
31;mashable.com/mashable.com/index.html;36.76;40.95;35.30;53.86;42.76
32;dailymail.co.uk/www.dailymail.co.uk/ushome/index.html;42.06;40.64;44.24;37.32;61.35
33;whois.domaintools.com/whois.domaintools.com/mozilla.com.html;34.73;35.23;39.25;48.24;35.72
34;indiatimes.com/www.indiatimes.com/index.html;52.67;55.51;46.29;52.69;58.82
35;reuters.com/www.reuters.com/index.html;32.92;33.08;36.95;39.23;39.27
36;xinhuanet.com/xinhuanet.com/index.html;125.85;102.56;138.89;130.34;147.45
37;56.com/www.56.com/index.html;63.89;75.00;61.45;62.20;58.67
38;bild.de/www.bild.de/index.html;35.61;43.74;34.79;33.45;31.83
39;guardian.co.uk/www.guardian.co.uk/index.html;38.91;55.93;62.34;42.63;45.99
40;naver.com/www.naver.com/index.html;78.10;89.07;127.67;75.18;109.32
41;yelp.com/www.yelp.com/biz/alexanders-steakhouse-cupertino.html;42.54;46.92;39.19;49.82;50.43
42;wsj.com/online.wsj.com/home-page.html;46.43;55.51;44.16;81.79;48.78
43;google.com/www.google.com/search@q=mozilla.html;35.62;36.71;44.47;45.00;40.22
44;xunlei.com/xunlei.com/index.html;67.57;60.69;83.83;85.53;85.08
45;aljazeera.net/aljazeera.net/portal.html;65.03;51.84;73.29;64.77;69.70
46;w3.org/www.w3.org/standards/webdesign/htmlcss.html;53.57;58.50;72.98;66.95;55.62
47;homeway.com.cn/www.hexun.com/index.html;105.59;117.32;108.95;116.10;70.32
48;youtube.com/www.youtube.com/music.html;40.53;41.48;59.67;40.81;40.07
49;people.com.cn/people.com.cn/index.html;96.49;103.64;115.12;66.05;117.84
</pre>
|}
== tart ==
'''Test Name'''
tart
* contact: :mconley
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/tart tart]
</pre>
|}
== tart_flex ==
'''Test Name'''
tart_flex
{{todo|''add test details''}}
== tp5 ==
Note that the tp5 test no longer exists (only talos-tp5o) though many tests still make use of this pageset. Here, we provide an overview of the tp5 pageset and some information about how data using the tp5 pageset might be used in various suites.
'''Test Name'''
tp5o
* contact: :davehunt
* data: we load each of the 51 tp5o pages 25 times, resulting in 51 sets of 25 data points.
* '''To run it locally''', you'd need [[#Page_sets|tp5n.zip]].
* summarization:tp5 with limited pageset (48 pages as others have too much noise)
** subtest: [[../Data#ignore_first|ignore first]] '''5''' data points, then take the [[../Data#median|median]] of the remaining 20; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l449 source: test.py]
** suite: [[../Data#geometric_mean|geometric mean]] of the 51 subtest results.
Tests the time it takes Firefox to load the [[#Page_sets|tp5 web page test set]]. The web set was culled from the Alexa top 500 April 8th, 2011 and consists of 100 pages in tp5n and 51 in tp5o. Some suites use a subset of these, i.e. 48/51 tests to reduce noise - check with the owner of the test suite which uses the pageset to check if this difference exists there.
== tp5o_scroll ==
'''Test Name'''
tp5o_scroll
* contact: :kats
* source: [[#Page_sets|tp5n.zip]]
* data: we load each of the 51 tp5o pages 12 times, resulting in 51 sets of 12 data points.
* '''To run it locally''', you'd need [[#Page_sets|tp5n.zip]].
* summarization:Measures average frames interval while scrolling the pages of the tp5o set
** subtest: [[../Data#ignore_first|ignore first]] data point, then take the [[../Data#median|median]] of the remaining 11; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l470 source: test.py]
** suite: [[../Data#geometric_mean|geometric mean]] of the 51 subtest results.
This test is identical to tscrollx, but it scrolls the 50 pages of the tp5o set (rather than 6 synthetic pages which tscrollx scrolls). There are two variants for each test page. The "regular" variant waits 500ms after the page load event fires, then iterates 100 scroll steps of 10px each (or until the bottom of the page is reached - whichever comes first), then reports the average frame interval. The "CSSOM" variant is similar, but uses APZ's smooth scrolling mechanism to do compositor scrolling instead of main-thread scrolling. So it just requests the final scroll destination and the compositor handles the scrolling and reports frame intervals.
* Slowdown in rasterization of content
* Slowdown in composite times
== tp5o_webext ==
'''Test Name'''
tp5o_webext
{{todo|''add test details''}}
== tpaint ==
{{warning|This test no longer exists}}
* contact: :davidb
* source: [[https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/startup_test/tpaint.html tpaint-window.html]]
=== Possible regression causes ===
* None listed yet. If you fix a regression for this test and have some tips to share, this is a good place for them.
== tresize ==
== ts_paint ==
'''Test Name'''
ts_paint
* contact: :davidb
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/startup_test/tspaint_test.html tspaint_test.html]
=== Possible regression causes ===
* (and/or maybe tpaint?) will regress if a new <panel> element is added to the browser window (e.g. browser.xul) and it's frame gets created. Fix this by ensuring it's display:none by default.
== ts_paint_flex ==
'''Test Name'''
ts_paint_flex
{{todo|''add test details''}}
== ts_paint_heavy ==
'''Test Name'''
ts_paint_heavy
[[#ts_paint|ts_paint]] test run against a heavy user profile.
{{todo|''add test details''}}
== ts_paint_webext ==
'''Test Name'''
ts_paint_webext
{{todo|''add test details''}}
== tscrollx ==
'''Test Name'''
tscrollx
* contact: :jrmuizel
* source: [[https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/scroll scroll.manifest]]
* reporting: Average frame interval (1 ÷ fps). Lower is better.
* data: we load 6 pages 25 times each, collecting 6 sets of 25 data points
* summarization:[https://groups.google.com/d/topic/mozilla.dev.platform/RICw5SJhNMo/discussion Replacing tscroll,tsvg with tscrollx,tsvgx]
** subtest: [[../Data#ignore_first|ignore first]] data point, then take the [[../Data#median|median]] of the remaining 24; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l623 source: test.py]
** suite: [[../Data#geometric_mean|geometric mean]] of the 6 subtest results.
This test scrolls several pages where each represent a different known "hard" case to scroll (* needinfo), and measures the average frames interval (1/FPS) on each. The ASAP test (tscrollx) iterates in unlimited frame-rate mode thus reflecting the maximum scroll throughput per page. To turn on ASAP mode, we set these preferences:
</pre>
|}
== tsvg_static ==
'''Test Name'''
tsvg_static
* contact: :jwatt, :dholbert, :neerja
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/svg_static/ svg_static]
* type: PageLoader
* data: we load the 5 svg pages 25 times, resulting in 5 sets of 25 data points
* summarization: An svg-only number that measures SVG rendering performance of some complex (but static) SVG content.
** subtest: [[../Data#ignore_first|ignore first]] '''5''' data points, then take the [[../Data#median|median]] of the remaining 20; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l623 source: test.py]
** suite: [[../Data#geometric_mean|geometric mean]] of the 5 subtest results.
{| class="mw-collapsible mw-collapsed wikitable"
! Example data
|-
| <pre>
0;gearflowers.svg;262;184;184;198;197;187;181;186;177;192;196;194;194;186;195;190;237;193;188;182;188;196;191;194;184
1;composite-scale.svg;69;52;48;49;57;51;52;87;52;49;49;51;58;53;64;57;49;65;67;58;53;59;56;68;50
2;composite-scale-opacity.svg;72;53;91;54;51;58;60;46;51;57;59;58;66;70;57;61;47;51;76;65;52;65;64;64;63
3;composite-scale-rotate.svg;70;76;89;62;62;78;57;77;79;82;74;56;61;79;73;64;75;74;81;82;76;58;77;61;62
4;composite-scale-rotate-opacity.svg;91;60;67;84;62;66;78;69;65;68;62;73;68;63;64;71;79;77;63;80;85;65;82;76;81
</pre>
|}
== tsvgm ==
'''Test Name'''
tsvgm
An svg-only number that measures SVG rendering performance
for dynamic content only.
{{todo|''add test details''}}
== tsvgr_opacity ==
'''Test Name'''
tsvgr_opacity
* contact: :jwatt, :dholbert
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/svg_opacity/svg_opacity.manifest]
* type: PageLoader
* data: we load the 2 svg opacity pages 25 times, resulting in 2 sets of 25 data points
* summarization:[[#Row_Major_vs._Column_Major|Row Major]] and 25 cycles/page.
** subtest: [[../Data#ignore_first|ignore first]] '''5''' data points, then take the [[../Data#median|median]] of the remaining 20; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l623 source: test.py]
** suite: [[../Data#geometric_mean|geometric mean]] of the 2 subtest results.
Renders many semi-transparent, partially overlapping SVG rectangles, and measures time to completion of this rendering.
|}
== tsvg_static tsvgx ==* contact: :jwatt, :dholbert, :neerja* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/svg_static/ svg_static]* type: PageLoader* data: we load the 5 svg pages 25 times, resulting in 5 sets of 25 data points* summarization:** subtest: [[../Data#ignore_first|ignore first]] '''5''' data points, then take the [[../Data#median|median]] of the remaining 20; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l623 source: test.py]** suite: [[../Data#geometric_mean|geometric mean]] of the 5 subtest results.
* contact: :jwatt, :dholbert
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/svgx svgx]
* type: PageLoader
* data: we load the 7 svg pages 25 times, resulting in 7 sets of 25 data points
* summarization:[https://groups.google.com/d/topic/mozilla.dev.platform/RICw5SJhNMo/discussion Replacing tscroll,tsvg with tscrollx,tsvgx]
** subtest: [[../Data#ignore_first|ignore first]] '''5''' data points, then take the [[../Data#median|median]] of the remaining 20; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l623 source: test.py]
** suite: [[../Data#geometric_mean|geometric mean]] of the 7 subtest results.
An svg-only number that measures SVG rendering performance, with animations or iterations of rendering. This is an ASAP test -- i.e. it iterates in unlimited frame-rate mode thus reflecting the maximum rendering throughput of each test. The reported value is the overall duration the sequence/animation took to complete. To turn on ASAP mode, we set these preferences:
=== Possible regression causes ===
* Did you change the dimensions of the content area? Even a little? The tsvgx test seems to be sensitive to changes like this. See {{bug|1375479}}, for example. Usually, these sorts of "regressions" aren't real regressions - they just mean that we need to re-baseline our expectations from the test.
== twinopen ==
'''Test Name'''
twinopen ext+twinopen:twinopen.html
* contact: :bdahl, :jimm
* source: [[https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/twinopen twinopen]]
* type: Startup
* data: we open a new browser window 20 times, resulting in 1 set of 20 data points.
* summarization: Time from calling OpenBrowserWindow until the chrome of the new window has [https://developer.mozilla.org/en-US/docs/Web/Events#MozAfterPaint painted].
** subtest: [[../Data#ignore_first|ignore first]] '''5''' data points, then take the [[../Data#median|median]] of the remaining 15; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l190 source: test.py]
** suite: identical to subtest
Tests the amount of time it takes the open a new window from a currently open browser. This test does not include startup time. Multiple test windows are opened in succession, results reported are the average amount of time required to create and display a window in the running instance of the browser. (Measures ctrl-n performance.)
{| class="mw-collapsible mw-collapsed wikitable"
! Example data
|-
| <pre>
[209.219, 222.180, 225.299, 225.970, 228.090, 229.450, 230.625, 236.315, 239.804, 242.795, 244.5, 244.770, 250.524, 251.785, 253.074, 255.349, 264.729, 266.014, 269.399, 326.190]
</pre>
|}
== xperf (tp5n) ==
