Changes

Jump to: navigation, search

TestEngineering/Performance/Raptor

10,122 bytes removed, 12:11, 18 September 2019
Page-Load Tests
Some of the page recordings contain [[https://wiki.mozilla.org/Performance_sheriffing/Raptor/Mitmproxy#Adding_Hero_Elements hero elements]]. When hero elements are measured, the value is the time until the hero element appears on the page (in MS).
Below are the details for each page-load suite, and the test pages contained within each.suites:
==== raptor-tp6-1 to 10 ====* contact: :rwood, :jmaherdavehunt, :bebe
* type: page-load
* browsers: Firefox desktop, Google Chrome desktop (Windows and OSX only at the moment)* measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, hero element, time-to-first-interactive, loadtime* measuring on Chrome: first-contentful-paint, hero element, loadtime* page-cycles: 25* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/raptor-tp6-1.ini raptor-tp6-1.ini]. ''' Test pages in tp6-1 (* = firefox or chrome):''' [raptor-tp6-amazon-*]* URL: https://www.amazon.com/s/url=search-alias%3Daps&field-keywords=laptop* Hero: string description element for first laptop in search results [raptor-tp6-facebook-*]* URL: https://www.facebook.com (logged into a user account)* Hero: on the Facebook 'Home' icon [raptor-tp6-google-*]* URL: https://www.google.com/search?hl=en&q=barack+obama&cad=h* Hero: bigger photo of Obama in search results towards top right [raptor-tp6-youtube-*]* URL: https://www.youtube.com* Hero: YouTube logo on the top left ==== raptor-tp6-2 ====* contact: :rwood, :jmaher* type: page-load* browsers: Firefox desktop, Google Chrome desktop (Windows and OSX only at the moment)* measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, hero element, time-to-first-interactive, loadtime* measuring on Chrome: first-contentful-paint, hero element, loadtime* page-cycles: 25* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/raptor-tp6-2.ini raptor-tp6-2.ini]. ''' Test pages in tp6-2 (* = firefox or chrome):''' [raptor-tp6-docs-*]* URL: https://docs.google.com/document/d/1US-07msg12slQtI_xchzYxcKlTs6Fp7WqIc6W5GK5M8/edit?usp=sharing* Hero: blue 'Sign In' button on the top right [raptor-tp6-sheets-*]* URL: https://docs.google.com/spreadsheets/d/1jT9qfZFAeqNoOK97gruc34Zb7y_Q-O_drZ8kSXT-4D4/edit?usp=sharing* Hero: blue 'Sign In' button on the top right [raptor-tp6-slides-*]* URL: https://docs.google.com/presentation/d/1Ici0ceWwpFvmIb3EmKeWSq_vAQdmmdFcWqaiLqUkJng/edit?usp=sharing* Hero: blue 'Sign In' button on the top right ==== raptor-tp6-3 ====* contact: :rwood, :jmaher, :bebe * type: page-load* browsers: Firefox desktopChromium, Google Chrome desktop (Windows and OSX only at the moment)* measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime
* measuring on Chrome: first-contentful-paint, loadtime
* page-cycles: 25
* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).
* test INI's: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/raptor-tp6-3.ini /desktop raptor-tp6-3.ini1 to 10 ].
''' Test pages in tp6-3 (* = firefox or chrome):'''
[raptor-tp6-imdb-*]* URL: https://www.imdb.com/title/tt0084967/?ref_=nv_sr_2 [raptor-tp6-imgur-*]* URL: https://imgur.com/gallery/m5tYJL6 [raptor-tp6-wikia-*]* URL: http://fandom.wikia.com/articles/fallout-76-will-live-and-die-on-the-creativity-of-its-playerbase ==== raptor-tp6tp6m-4 ====* contact: :rwood, :jmaher, :bebe * type: page-load* browsers: Firefox desktop, Google Chrome desktop (Windows and OSX only at the moment)* measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime* measuring on Chrome: first-contentful-paint, loadtime* page-cycles: 25* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due 1 to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/raptor-tp6-4.ini raptor-tp6-4.ini]. ''' Test pages in tp6-4 (* = firefox or chrome):''' [raptor-tp6-bing-*]* URL: https://www.bing.com/search?q=barack+obama [raptor-tp6-yandex-*]* URL: https://yandex.ru/search/?text=barack%20obama&lr=10115 ==== raptor-tp6-5 ====* contact: :rwood, :jmaher, :bebe * type: page-load* browsers: Firefox desktop, Google Chrome desktop (Windows and OSX only at the moment)* measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime* measuring on Chrome: first-contentful-paint, loadtime* page-cycles: 25* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/raptor-tp6-5.ini raptor-tp6-5.ini]. ''' Test pages in tp6-5 (* = firefox or chrome):''' [raptor-tp6-apple-*]* URL: https://www.apple.com/macbook-pro/ [raptor-tp6-microsoft-*]* URL: https://www.microsoft.com/en-us/windows/get-windows-10 ==== raptor-tp6-6 ====* contact: :rwood, :jmaher, :bebe * type: page-load* browsers: Firefox desktop, Google Chrome desktop (Windows and OSX only at the moment)* measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime* measuring on Chrome: first-contentful-paint, loadtime* page-cycles: 25* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/raptor-tp6-6.ini raptor-tp6-6.ini]. ''' Test pages in tp6-6 (* = firefox or chrome):''' [raptor-tp6-reddit-*]* URL: https://www.reddit.com/r/technology/comments/9sqwyh/we_posed_as_100_senators_to_run_ads_on_facebook/ [raptor-tp6-yahoo-news-*]* URL: https://www.yahoo.com/lifestyle/police-respond-noise-complaint-end-playing-video-games-respectful-tenants-002329963.html ==== raptor-tp6-7 ====* contact: :rwood, :jmaher, :bebe * type: page-load* browsers: Firefox desktop, Google Chrome desktop (Windows and OSX only at the moment)* measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime* measuring on Chrome: first-contentful-paint, loadtime* page-cycles: 25* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/raptor-tp6-7.ini raptor-tp6-7.ini]. ''' Test pages in tp6-7 (* = firefox or chrome):''' [raptor-tp6-instagram-*]* URL: https://www.instagram.com/ [raptor-tp6-twitter-*]* URL: https://twitter.com/BarackObama [raptor-tp6-yahoo-mail-*]* URL: https://mail.yahoo.com/ ==== raptor-tp6-8 ====* contact: :rwood, :jmaherdavehunt, :bebe * type: page-load* browsers: Firefox desktop, Google Chrome desktop (Windows and OSX only at the moment)* measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime* measuring on Chrome: first-contentful-paint, loadtime* page-cycles: 25* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/raptor-tp6-8.ini raptor-tp6-8.ini]. ''' Test pages in tp6-8 (* = firefox or chrome):''' [raptor-tp6-ebay-*]* URL: https://www.ebay.com/ [raptor-tp6-wikipedia-*]* URL: https://en.wikipedia.org/wiki/Barack_Obama ==== raptor-tp6-9 ====* contact: :rwood, :jmaher, :bebe * type: page-load* browsers: Firefox desktop, Google Chrome desktop (Windows and OSX only at the moment)* measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime* measuring on Chrome: first-contentful-paint, loadtime* page-cycles: 25* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/raptor-tp6-9.ini raptor-tp6-9.ini]. ''' Test pages in tp6-9 (* = firefox or chrome):''' [raptor-tp6-google-mail-*]* URL: https://mail.google.com/ [raptor-tp6-pinterest-*]* URL: https://pinterest.com/ ==== raptor-tp6-10 ====* contact: :rwood, :jmaher, :bebe * type: page-load* browsers: Firefox desktop, Google Chrome desktop (Windows and OSX only at the moment)* measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime* measuring on Chrome: first-contentful-paint, loadtime* page-cycles: 25* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/raptor-tp6-10.ini raptor-tp6-10.ini]. ''' Test pages in tp6-10 (* = firefox or chrome):''' [raptor-tp6-paypal-*]* URL: https://www.paypal.com/myaccount/summary/ ==== raptor-tp6m-1 ====* contact: :rwood, :davehunt
* type: page-load
* browsers: Firefox Android Geckoview Example App
* page-cycles: 15
* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).
* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile raptor-tp6m-1.ini raptor-tp6m-1.ini]. ''' Test pages in tp6m-1 (* = firefox or chrome):''' [raptor-tp6m-amazon-*to 10]* URL: https://www.amazon.com* Hero: None [raptor-tp6m-facebook-*]* URL: https://m.facebook.com (logged into a user account)* Hero: None [raptor-tp6m-google-*]* URL: https://www.google.com* Hero: None [raptor-tp6m-youtube-*]* URL: https://www.youtube.com* Hero: None
=== Benchmark Tests ===
Confirm
132
edits

Navigation menu