MozillaWiki

TestEngineering/Performance/Raptor: Difference between revisions

Page Discussion

TestEngineering/Performance/Raptor (view source)

Revision as of 12:11, 18 September 2019

10,122 bytes removed ,  18 September 2019
→‎Page-Load Tests
No edit summary
Line 428: Line 428:
Some of the page recordings contain [[https://wiki.mozilla.org/Performance_sheriffing/Raptor/Mitmproxy#Adding_Hero_Elements hero elements]]. When hero elements are measured, the value is the time until the hero element appears on the page (in MS).
Some of the page recordings contain [[https://wiki.mozilla.org/Performance_sheriffing/Raptor/Mitmproxy#Adding_Hero_Elements hero elements]]. When hero elements are measured, the value is the time until the hero element appears on the page (in MS).


Below are the details for each page-load suite, and the test pages contained within each.
Below are the details for page-load suites:


==== raptor-tp6-1 ====
==== raptor-tp6-1 to 10 ====
* contact: :rwood, :jmaher
* contact: :rwood, :davehunt, :bebe
* type: page-load
* type: page-load
* browsers: Firefox desktop, Google Chrome desktop (Windows and OSX only at the moment)
* browsers: Firefox desktop, Chromium, Google Chrome desktop (Windows and OSX only at the moment)
* measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, hero element, time-to-first-interactive, loadtime
* measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, loadtime
* measuring on Chrome: first-contentful-paint, hero element, loadtime
* page-cycles: 25
* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).
* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/raptor-tp6-1.ini raptor-tp6-1.ini].
 
''' Test pages in tp6-1 (* = firefox or chrome):'''
 
[raptor-tp6-amazon-*]
* URL: https://www.amazon.com/s/url=search-alias%3Daps&field-keywords=laptop
* Hero: string description element for first laptop in search results
 
[raptor-tp6-facebook-*]
* URL: https://www.facebook.com (logged into a user account)
* Hero: on the Facebook 'Home' icon
 
[raptor-tp6-google-*]
* URL: https://www.google.com/search?hl=en&q=barack+obama&cad=h
* Hero: bigger photo of Obama in search results towards top right
 
[raptor-tp6-youtube-*]
* URL: https://www.youtube.com
* Hero: YouTube logo on the top left
 
==== raptor-tp6-2 ====
* contact: :rwood, :jmaher
* type: page-load
* browsers: Firefox desktop, Google Chrome desktop (Windows and OSX only at the moment)
* measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, hero element, time-to-first-interactive, loadtime
* measuring on Chrome: first-contentful-paint, hero element, loadtime
* page-cycles: 25
* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).
* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/raptor-tp6-2.ini raptor-tp6-2.ini].
 
''' Test pages in tp6-2 (* = firefox or chrome):'''
 
[raptor-tp6-docs-*]
* URL: https://docs.google.com/document/d/1US-07msg12slQtI_xchzYxcKlTs6Fp7WqIc6W5GK5M8/edit?usp=sharing
* Hero: blue 'Sign In' button on the top right
 
[raptor-tp6-sheets-*]
* URL: https://docs.google.com/spreadsheets/d/1jT9qfZFAeqNoOK97gruc34Zb7y_Q-O_drZ8kSXT-4D4/edit?usp=sharing
* Hero: blue 'Sign In' button on the top right
 
[raptor-tp6-slides-*]
* URL: https://docs.google.com/presentation/d/1Ici0ceWwpFvmIb3EmKeWSq_vAQdmmdFcWqaiLqUkJng/edit?usp=sharing
* Hero: blue 'Sign In' button on the top right
 
==== raptor-tp6-3 ====
* contact: :rwood, :jmaher, :bebe
* type: page-load
* browsers: Firefox desktop, Google Chrome desktop (Windows and OSX only at the moment)
* measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime
* measuring on Chrome: first-contentful-paint, loadtime
* measuring on Chrome: first-contentful-paint, loadtime
* page-cycles: 25
* page-cycles: 25
* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).
* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).
* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/raptor-tp6-3.ini raptor-tp6-3.ini].
* test INI's: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop raptor-tp6-1 to 10 ].


''' Test pages in tp6-3 (* = firefox or chrome):'''


[raptor-tp6-imdb-*]
==== raptor-tp6m-1 to 10 ====
* URL: https://www.imdb.com/title/tt0084967/?ref_=nv_sr_2
* contact: :rwood, :davehunt, :bebe
 
[raptor-tp6-imgur-*]
* URL: https://imgur.com/gallery/m5tYJL6
 
[raptor-tp6-wikia-*]
* URL: http://fandom.wikia.com/articles/fallout-76-will-live-and-die-on-the-creativity-of-its-playerbase
 
==== raptor-tp6-4 ====
* contact: :rwood, :jmaher, :bebe
* type: page-load
* browsers: Firefox desktop, Google Chrome desktop (Windows and OSX only at the moment)
* measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime
* measuring on Chrome: first-contentful-paint, loadtime
* page-cycles: 25
* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).
* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/raptor-tp6-4.ini raptor-tp6-4.ini].
 
''' Test pages in tp6-4 (* = firefox or chrome):'''
 
[raptor-tp6-bing-*]
* URL: https://www.bing.com/search?q=barack+obama
 
[raptor-tp6-yandex-*]
* URL: https://yandex.ru/search/?text=barack%20obama&lr=10115
 
==== raptor-tp6-5 ====
* contact: :rwood, :jmaher, :bebe
* type: page-load
* browsers: Firefox desktop, Google Chrome desktop (Windows and OSX only at the moment)
* measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime
* measuring on Chrome: first-contentful-paint, loadtime
* page-cycles: 25
* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).
* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/raptor-tp6-5.ini raptor-tp6-5.ini].
 
''' Test pages in tp6-5 (* = firefox or chrome):'''
 
[raptor-tp6-apple-*]
* URL: https://www.apple.com/macbook-pro/
 
[raptor-tp6-microsoft-*]
* URL: https://www.microsoft.com/en-us/windows/get-windows-10
 
==== raptor-tp6-6 ====
* contact: :rwood, :jmaher, :bebe
* type: page-load
* browsers: Firefox desktop, Google Chrome desktop (Windows and OSX only at the moment)
* measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime
* measuring on Chrome: first-contentful-paint, loadtime
* page-cycles: 25
* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).
* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/raptor-tp6-6.ini raptor-tp6-6.ini].
 
''' Test pages in tp6-6 (* = firefox or chrome):'''
 
[raptor-tp6-reddit-*]
* URL: https://www.reddit.com/r/technology/comments/9sqwyh/we_posed_as_100_senators_to_run_ads_on_facebook/
 
[raptor-tp6-yahoo-news-*]
* URL: https://www.yahoo.com/lifestyle/police-respond-noise-complaint-end-playing-video-games-respectful-tenants-002329963.html
 
==== raptor-tp6-7 ====
* contact: :rwood, :jmaher, :bebe
* type: page-load
* browsers: Firefox desktop, Google Chrome desktop (Windows and OSX only at the moment)
* measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime
* measuring on Chrome: first-contentful-paint, loadtime
* page-cycles: 25
* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).
* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/raptor-tp6-7.ini raptor-tp6-7.ini].
 
''' Test pages in tp6-7 (* = firefox or chrome):'''
 
[raptor-tp6-instagram-*]
* URL: https://www.instagram.com/
 
[raptor-tp6-twitter-*]
* URL: https://twitter.com/BarackObama
 
[raptor-tp6-yahoo-mail-*]
* URL: https://mail.yahoo.com/
 
==== raptor-tp6-8 ====
* contact: :rwood, :jmaher, :bebe  
* type: page-load
* browsers: Firefox desktop, Google Chrome desktop (Windows and OSX only at the moment)
* measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime
* measuring on Chrome: first-contentful-paint, loadtime
* page-cycles: 25
* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).
* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/raptor-tp6-8.ini raptor-tp6-8.ini].
 
''' Test pages in tp6-8 (* = firefox or chrome):'''
 
[raptor-tp6-ebay-*]
* URL: https://www.ebay.com/
 
[raptor-tp6-wikipedia-*]
* URL: https://en.wikipedia.org/wiki/Barack_Obama
 
==== raptor-tp6-9 ====
* contact: :rwood, :jmaher, :bebe
* type: page-load
* browsers: Firefox desktop, Google Chrome desktop (Windows and OSX only at the moment)
* measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime
* measuring on Chrome: first-contentful-paint, loadtime
* page-cycles: 25
* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).
* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/raptor-tp6-9.ini raptor-tp6-9.ini].
 
''' Test pages in tp6-9 (* = firefox or chrome):'''
 
[raptor-tp6-google-mail-*]
* URL: https://mail.google.com/
 
[raptor-tp6-pinterest-*]
* URL: https://pinterest.com/
 
==== raptor-tp6-10 ====
* contact: :rwood, :jmaher, :bebe
* type: page-load
* browsers: Firefox desktop, Google Chrome desktop (Windows and OSX only at the moment)
* measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime
* measuring on Chrome: first-contentful-paint, loadtime
* page-cycles: 25
* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).
* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/raptor-tp6-10.ini raptor-tp6-10.ini].
 
''' Test pages in tp6-10 (* = firefox or chrome):'''
 
[raptor-tp6-paypal-*]
* URL: https://www.paypal.com/myaccount/summary/
 
==== raptor-tp6m-1 ====
* contact: :rwood, :davehunt
* type: page-load
* type: page-load
* browsers: Firefox Android Geckoview Example App
* browsers: Firefox Android Geckoview Example App
Line 636: Line 448:
* page-cycles: 15
* page-cycles: 15
* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).
* reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).
* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/raptor-tp6m-1.ini raptor-tp6m-1.ini].
* test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile raptor-tp6m-1 to 10].
 
''' Test pages in tp6m-1 (* = firefox or chrome):'''
 
[raptor-tp6m-amazon-*]
* URL: https://www.amazon.com
* Hero: None
 
[raptor-tp6m-facebook-*]
* URL: https://m.facebook.com (logged into a user account)
* Hero: None
 
[raptor-tp6m-google-*]
* URL: https://www.google.com
* Hero: None
 
[raptor-tp6m-youtube-*]
* URL: https://www.youtube.com
* Hero: None


=== Benchmark Tests ===
=== Benchmark Tests ===
Bebef 1987
Confirmed users
132

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1218010"