Confirmed users
2,204
edits
TestEngineering/Performance/Results: Difference between revisions
Jump to navigation
Jump to search
TestEngineering/Performance/Results (view source)
Revision as of 14:32, 9 August 2019
, 9 August 2019no edit summary
No edit summary |
No edit summary |
||
| Line 3: | Line 3: | ||
There are a variety of places where performance results are available, some of these can be queried directly, and others are dashboards built on top of those sources. | There are a variety of places where performance results are available, some of these can be queried directly, and others are dashboards built on top of those sources. | ||
= ActiveData Query Tool = | = What should I use? = | ||
* Does my patch affect performance? | |||
* Did my push affect performance? | |||
* How does Firefox compare against other browsers? | |||
* Are we meeting our goals? | |||
* What performance tests are we running? | |||
* What platforms are we running performance tests against? | |||
= Data Sources = | |||
== ActiveData Query Tool == | |||
ActiveData is a collection of billions of records covering unit tests, jobs, performance data, and source code revision history. The intended use is for automated clients or for building dashboards, however the query tool can be used for exploration. | ActiveData is a collection of billions of records covering unit tests, jobs, performance data, and source code revision history. The intended use is for automated clients or for building dashboards, however the query tool can be used for exploration. | ||
| Line 14: | Line 23: | ||
* Contact: Kyle Lahnakoski [ekyle] | * Contact: Kyle Lahnakoski [ekyle] | ||
== Examples == | === Examples === | ||
* [https://activedata.allizom.org/tools/query.html#query_id=rqnxuVpf Performance frameworks] | * [https://activedata.allizom.org/tools/query.html#query_id=rqnxuVpf Performance frameworks] | ||
* [https://activedata.allizom.org/tools/query.html#query_id=rqnxuVpf Performance suites] | * [https://activedata.allizom.org/tools/query.html#query_id=rqnxuVpf Performance suites] | ||
* [https://activedata.allizom.org/tools/query.html#query_id=1dtPBeZR Performance suites running on Android hardware] | * [https://activedata.allizom.org/tools/query.html#query_id=1dtPBeZR Performance suites running on Android hardware] | ||
= ActiveData Recipes = | == ActiveData Recipes == | ||
Due to the size and complexity of ActiveData, using it to answer everyday questions is not a straightforward task and the built-in query tool is often insufficient. ActiveData Recipes was created to provide an intuitive way to not only use ActiveData, but to save your queries for future use and share them with others. | Due to the size and complexity of ActiveData, using it to answer everyday questions is not a straightforward task and the built-in query tool is often insufficient. ActiveData Recipes was created to provide an intuitive way to not only use ActiveData, but to save your queries for future use and share them with others. | ||
| Line 30: | Line 39: | ||
* Contact: Andrew Halberstadt [ahal] | * Contact: Andrew Halberstadt [ahal] | ||
= Datadog = | == Datadog == | ||
{{todo|add description}} | {{todo|add description}} | ||
== WebPageTest == | === WebPageTest === | ||
* Location: https://app.datadoghq.com/dashboard/lists/manual/18187 | * Location: https://app.datadoghq.com/dashboard/lists/manual/18187 | ||
* Access: File a bug to request access by replacing '''[user]''' and '''[email]''' in [https://bugzilla.mozilla.org/enter_bug.cgi?assigned_to=nobody%40mozilla.org&bug_ignored=0&bug_severity=normal&bug_status=NEW&bug_type=task&cf_fx_iteration=---&cf_fx_points=---&cf_status_firefox68=---&cf_status_firefox69=---&cf_status_firefox70=---&cf_status_firefox_esr60=---&cf_status_firefox_esr68=---&cf_tracking_firefox68=---&cf_tracking_firefox69=---&cf_tracking_firefox70=---&cf_tracking_firefox_esr60=---&cf_tracking_firefox_esr68=---&cf_tracking_firefox_relnote=---&comment=Please%20add%20%5Bemail%5D%20to%20the%20cloudservices_datadog%20LDAP%20group%20and%20be%20sent%20an%20invitation%20to%20access%20Datadog%3F&component=Operations&contenttypemethod=list&contenttypeselection=text%2Fplain&defined_groups=1&flag_type-37=X&flag_type-607=X&flag_type-708=X&flag_type-721=X&flag_type-737=X&flag_type-748=X&flag_type-787=X&flag_type-800=X&flag_type-803=X&flag_type-846=X&flag_type-864=X&flag_type-929=X&flag_type-936=X&flag_type-941=X&form_name=enter_bug&maketemplate=Remember%20values%20as%20bookmarkable%20template&op_sys=Unspecified&priority=--&product=Cloud%20Services&rep_platform=Unspecified&short_desc=Grant%20%5Buser%5D%20access%20to%20Datadog&target_milestone=---&version=unspecified this template]. | * Access: File a bug to request access by replacing '''[user]''' and '''[email]''' in [https://bugzilla.mozilla.org/enter_bug.cgi?assigned_to=nobody%40mozilla.org&bug_ignored=0&bug_severity=normal&bug_status=NEW&bug_type=task&cf_fx_iteration=---&cf_fx_points=---&cf_status_firefox68=---&cf_status_firefox69=---&cf_status_firefox70=---&cf_status_firefox_esr60=---&cf_status_firefox_esr68=---&cf_tracking_firefox68=---&cf_tracking_firefox69=---&cf_tracking_firefox70=---&cf_tracking_firefox_esr60=---&cf_tracking_firefox_esr68=---&cf_tracking_firefox_relnote=---&comment=Please%20add%20%5Bemail%5D%20to%20the%20cloudservices_datadog%20LDAP%20group%20and%20be%20sent%20an%20invitation%20to%20access%20Datadog%3F&component=Operations&contenttypemethod=list&contenttypeselection=text%2Fplain&defined_groups=1&flag_type-37=X&flag_type-607=X&flag_type-708=X&flag_type-721=X&flag_type-737=X&flag_type-748=X&flag_type-787=X&flag_type-800=X&flag_type-803=X&flag_type-846=X&flag_type-864=X&flag_type-929=X&flag_type-936=X&flag_type-941=X&form_name=enter_bug&maketemplate=Remember%20values%20as%20bookmarkable%20template&op_sys=Unspecified&priority=--&product=Cloud%20Services&rep_platform=Unspecified&short_desc=Grant%20%5Buser%5D%20access%20to%20Datadog&target_milestone=---&version=unspecified this template]. | ||
| Line 42: | Line 51: | ||
* Contact: Stephen Donner [stephend] | * Contact: Stephen Donner [stephend] | ||
= Firefox Health = | == Firefox Health == | ||
The health dashboard tracks metrics and statistics important for tracking performance improvements. | The health dashboard tracks metrics and statistics important for tracking performance improvements. | ||
| Line 53: | Line 62: | ||
* Contact: Kyle Lahnakoski [ekyle] | * Contact: Kyle Lahnakoski [ekyle] | ||
= Firefox Performance Dashboard = | == Firefox Performance Dashboard == | ||
* Location: https://arewefastyet.com/ | * Location: https://arewefastyet.com/ | ||
* Access: Public | * Access: Public | ||
| Line 62: | Line 71: | ||
* Contact: Armen Zambrano Gasparnian [armenzg] | * Contact: Armen Zambrano Gasparnian [armenzg] | ||
= Metrics = | == Metrics == | ||
{{todo|https://metrics.mozilla.com/protected/shiny/cdowhygelund/firefox-perf-telemetry-dashboard/}} | {{todo|https://metrics.mozilla.com/protected/shiny/cdowhygelund/firefox-perf-telemetry-dashboard/}} | ||
= Nimbledroid = | == Nimbledroid == | ||
{{todo|add description}} | {{todo|add description}} | ||
| Line 75: | Line 84: | ||
* Contact: No-Jun Park [npark] | * Contact: No-Jun Park [npark] | ||
= Perfherder = | == Perfherder == | ||
Perfherder is an interactive dashboard intended to allow monitoring and analysis of automated performance tests run against Mozilla products. Perfherder is part of the [[EngineeringProductivity/Projects/Treeherder|Treeherder]] project. | Perfherder is an interactive dashboard intended to allow monitoring and analysis of automated performance tests run against Mozilla products. Perfherder is part of the [[EngineeringProductivity/Projects/Treeherder|Treeherder]] project. | ||
| Line 86: | Line 95: | ||
* Contact: Sarah Clements [sclements] | * Contact: Sarah Clements [sclements] | ||
== Alerts == | === Alerts === | ||
{{todo|add description}} | {{todo|add description}} | ||
| Line 92: | Line 101: | ||
* Documentation: {{todo|}} | * Documentation: {{todo|}} | ||
== Compare == | === Compare === | ||
{{todo|add description}} | {{todo|add description}} | ||
| Line 98: | Line 107: | ||
* Documentation: {{todo|}} | * Documentation: {{todo|}} | ||
== Graphs == | === Graphs === | ||
{{todo|add description}} | {{todo|add description}} | ||
| Line 104: | Line 113: | ||
* Documentation: {{todo|}} | * Documentation: {{todo|}} | ||
= Redash = | == Redash == | ||
{{todo|add description}} | {{todo|add description}} | ||
| Line 115: | Line 124: | ||
* Contact: {{todo|}} | * Contact: {{todo|}} | ||
= Ad-hoc results = | == Ad-hoc results == | ||
{{todo|add description}} | {{todo|add description}} | ||
* [https://drive.google.com/drive/folders/11a90GH_xwcqwkcAmzCZAkwmQeuSoXA3n Performance/Projects/Desktop/Testing] | * [https://drive.google.com/drive/folders/11a90GH_xwcqwkcAmzCZAkwmQeuSoXA3n Performance/Projects/Desktop/Testing] | ||