canmove, Confirmed users, Bureaucrats and Sysops emeriti
4,747
edits
TestEngineering/Performance/Raptor/Youtube playback performance: Difference between revisions
TestEngineering/Performance/Raptor/Youtube playback performance (view source)
Revision as of 09:45, 23 May 2019
, 23 May 2019Added general information, and for subtests and updating the test site
(Initial import) |
(Added general information, and for subtests and updating the test site) |
||
| Line 1: | Line 1: | ||
__TOC__ | __TOC__ | ||
=== | |||
= YouTube playback performance test = | |||
The benchmark is measuring the playback performance of video streaming by assembling dropped frames during video playback. The test suite is a copy of the [https://ytlr-cert.appspot.com/2019/main.html?test_type=playbackperf-test Playback Performance Tests] from Google, and run in our own infrastructure. | |||
== Running locally == | |||
For information in how to setup your system to run the tests locally consult the | |||
[https://wiki.mozilla.org/Performance_sheriffing/Raptor#Running_Locally_2 general Raptor documentation] first. | |||
Once done the benchmark test can be run via the following command: | |||
$ ./mach raptor-test --test=raptor-youtube-playback | |||
=== Customizing the test execution === | |||
There are some variables you can tweak in the [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/raptor-youtube-playback.ini raptor-youtube-playback] manifest file to customize the benchmark test. | |||
==== Subset of tests ==== | |||
To only run a subset of tests the benchmark test page supports the '''tests''' argument, which allows to add any number of tests based on their index. Currently it needs to be a comma separated list, but will be soon extended for ranges too. | |||
The following line will only cause the videos of index 6, 47, and 90 to run: | |||
test_url = http://[...]main.html?test_type=playbackperf-test&raptor=true&command=run&exclude=1,2&tests=6,47,90 | |||
This is helpful if specific video codecs, resolutions, or playback speeds are causing performance issues. | |||
== Profiling == | |||
TODO | TODO | ||
== Updating the test suite == | |||
In inregular intervals Google will update its test suite. As we also want to have those changes mirrored for our instance the [https://github.com/mozilla/perf-youtube-playback/ perf-youtube-playback] repository has to be used. Changing its content on the `raptor` branch will automatically be picked-up by yttest.prod.mozaws.net where the tests are hosted on. | |||
Details in how to update the code can be found at https://github.com/mozilla/perf-youtube-playback/wiki. | |||