Changes

For all Raptor page-load tests, the pages are played back from [[https://wiki.mozilla.org/TestEngineering/Performance/Raptor/Mitmproxy Mitmproxy]] recordings. If you need the HTML page source (outside of the Mitmproxy recording) for debugging, the raw HTML can be found in our [https://github.com/mozilla/perf-automation/tree/master/pagesets perf-automation github repo]. All the pages in a test suite an be run documentation is generated by calling the top-level test name, i.e.:  ./mach raptor --test raptor-tp6-1 Individual test pages can be ran by calling the subtest, i.e.:  ./mach raptor --test raptor-tp6-google-firefox Some of the page recordings contain [[https://wiki.mozilla.org/Performance_sheriffing/Raptor/Mitmproxy#Adding_Hero_Elements hero elements]]. When hero elements are measured, the value is the time until the hero element appears on the page (in MS). All pageload tests can be found at [https://searchfox.org/mozillafirefox-central/source/testing/raptor/raptor/tests/tp6/ raptor-tp6 tests ] Below are the details for page-load suites: ===== raptor-tp6-1 to 10 =====* contact: :rwood, :davehunt, :bebe* type: page-load* browsers: Firefox desktop, Chromium, Google Chrome desktop (Windows and OSX only at the moment)* measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, loadtime* measuring on Chrome: first-contentful-paint, loadtime* page-cycles: 25* reporting: Each pagecycle measures all the values (in MS)docs. The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).* test INI's: [https://searchfoxmozilla.org/mozillacode-central/sourcequality/testinglint/raptorlinters/raptor/tests/tp6/desktop raptor-tp6-1 to 10 perfdocs.html PerfDocs]. ===== raptor-tp6-cold-1 to 4 =====* contact: :rwood, :davehunt, :bebe* type: page-load, cold* browsers: Firefox desktop, Chromium, Google Chrome desktop (Windows and OSX only at the moment)* measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, loadtime* measuring on Chrome: first-contentful-paint, loadtime* page-cycles: 25* reporting: Each pagecycle measures all the values (available in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).* test INI's: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop raptor-tp6-cold-1 to 4 ]. ===== raptor-tp6m-1 to 10 =====* contact: :rwood, :davehunt, :bebe* type: page-load* browsers: Firefox Android Geckoview Example App* measuring: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime* page-cycles: 15* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).* test INI: [https://searchfox.org/mozillafirefox-central/source/testing/raptor/raptor/tests/tp6/mobile raptor-tp6m-1 to 10]docs. ===== raptor-tp6m-cold-1 to 27 =====* contact: :rwood, :davehunt, :bebe* type: page-load, cold* browsers: Firefox Android Geckoview Example App* measuring: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime* page-cycles: 15* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile raptor-tp6m-cold-1 to 27]. ===== raptor-tp6m-cold-1 to 9-fennec68 =====* contact: :rwood, :davehunt, :bebe* type: page-load* browsers: Firefox Android Fennec ESR 68 App* measuring: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime* page-cycles: 15* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).* test INI: [https://searchfox.org/mozilla-central/source/testing/raptorperfdocs/raptor/tests/tp6/mobile raptor-tp6m-1 to 9-fennec68]. ===== raptor-tp6m-cold-1 to 27-fennec68 =====* contact: :rwood, :davehunt, :bebe* type: page-load, cold* browsers: html Firefox Android Fennec ESR 68 App* measuring: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime* page-cycles: 15* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile raptor-tp6m-cold-1 to 14-fennec68Source Docs].