TestEngineering/Performance: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Redirected page to Performance/Tools)
 
(72 intermediate revisions by 10 users not shown)
Line 1: Line 1:
== What we do ==
#REDIRECT [[Performance/Tools]]


== What we don't do ==
{{DISPLAYTITLE:Firefox Performance Test Engineering 🔥🦊⏱️}}[[File:Fxperftest.png|thumb|right]]


== Onboarding ==
= New contributors =


== Objectives ==
If you are a new contributor, or would like to start contributing you can find a guide to help you [[/NewContributors|here]].
* Increase regression detection '''coverage''' to page load on Android products including WebView comparisons
* Increase quality standards by measuring and alerting on power usage for Android and ARM64
* Build dashboards to improve '''visibility''' of how we are tracking against our release criteria for Android
* Reducing noise in test results and adding annotations to provide '''clarity''' of signal for regressions
* Develop measurements and mechanisms for reporting on our tools, policies and documentation for how to improve '''clarity''' and '''efficiency''' of risk assessments


== Team ==
= Where to find us =
* Dave Hunt [:davehunt]
* [https://chat.mozilla.org/#/room/#perftest:mozilla.org #perftest]
* Rob Wood [:rwood]
* Kyle Lahnakoski [:ekyle]
* Stephen Donner [:stephend]
* Ionut Goldan [:igoldan]
* Florian Strugariu [:bebe]
* Greg Mierzwinski [:sparky]


== Projects ==
= Team purpose =
To support the infrastructure and creation of automated tests for evaluating the performance of Firefox products. This provides value by exposing gaps in coverage, revealing areas where we can make performance gains, identifying performance regressions in a timely manner, and by providing key performance metrics that assist in determining how Firefox measures against release criteria.


Trello: https://trello.com/b/gc3AlyhB/firefox-performance-test-engineering
= What we do =
* Identification of gaps in performance test infrastructure and monitoring.
* Designing and building performance test infrastructure and monitoring solutions.
* Supporting Firefox engineers on writing performance tests.
* Supporting Firefox engineers on investigating regressions identified by tests.
* Collaboration with release operations on infrastructure requirements.
* Standing up performance tests in continuous integration environments.
* Monitoring performance test results and identifying potential regressions.
* Supporting performance sheriffs with tools to assist in identifying regressions.
* Developing test plans for performance testing.
* Running adhoc manual or partially automated performance testing.


=== Raptor ===
= What we don't do =
Description
* Maintenance of infrastructure hardware.
* Maintain the continuous integration pipeline.
* Writing/maintaining all performance tests.


* Owner: Rob Wood [:rwood]
= Meetings =
* Link to source code
{{/Meetings}}
* Link to good first bugs
* User documentation
* Developer documentation


=== Talos ===
= Onboarding =
Welcome to the team! You are encouraged to improve the [[/Onboarding|onboarding page]]. If you need to ask questions that are not already covered, please update the page so that the next person has a better onboarding experience.


=== WebPageTest ===
= Workflow =
* Owner: Stephen Donner [:stephend]
* [[/Triage Process/]]
* Source: https://github.com/mozilla/wpt-api
* [[/Review Process/]]
* (User and Dev) Docs: https://mozilla-wpt-api-docs.readthedocs.io/en/master/


=== Perfherder ===
= Projects =
* [[/Fenix/]]
* [[/Raptor/]]
* [[/Raptor/Browsertime/]]
* [[/Talos/]]


=== Firefox Performance Dashboard ===
= Results =
See our {{wip|[[/Results|results page]]}}.


=== Firefox Health Dashboard ===
= Resources =
* [[/Glossary/]]
* [https://docs.google.com/document/d/1SswqYIAm4h8vlwfMc0pfGEJwXpFECyubGDezRZHHPFE/edit Strategies for investigating intermittents]
* [https://docs.google.com/document/d/1HV2_z8hwhI2w8EbURtkYjpikVG5g9QeKEPo9h5msuRs/edit Following up perf bugs]
* [https://docs.google.com/document/d/103SRVVcE2SZNYP3kFXGeiVQrusH2Wj2yv8SWaDvB9SM/edit Excessive Android device queue response plan]
* [[/FAQ#How_can_I_do_a_bisection.3F|Bisection Workflow]]
* [[/Sheriffing/CompareView|CompareView]]

Latest revision as of 17:57, 24 July 2023

Redirect to:

Fxperftest.png

New contributors

If you are a new contributor, or would like to start contributing you can find a guide to help you here.

Where to find us

Team purpose

To support the infrastructure and creation of automated tests for evaluating the performance of Firefox products. This provides value by exposing gaps in coverage, revealing areas where we can make performance gains, identifying performance regressions in a timely manner, and by providing key performance metrics that assist in determining how Firefox measures against release criteria.

What we do

  • Identification of gaps in performance test infrastructure and monitoring.
  • Designing and building performance test infrastructure and monitoring solutions.
  • Supporting Firefox engineers on writing performance tests.
  • Supporting Firefox engineers on investigating regressions identified by tests.
  • Collaboration with release operations on infrastructure requirements.
  • Standing up performance tests in continuous integration environments.
  • Monitoring performance test results and identifying potential regressions.
  • Supporting performance sheriffs with tools to assist in identifying regressions.
  • Developing test plans for performance testing.
  • Running adhoc manual or partially automated performance testing.

What we don't do

  • Maintenance of infrastructure hardware.
  • Maintain the continuous integration pipeline.
  • Writing/maintaining all performance tests.

Meetings

Onboarding

Welcome to the team! You are encouraged to improve the onboarding page. If you need to ask questions that are not already covered, please update the page so that the next person has a better onboarding experience.

Workflow

Projects

Results

See our 🚧 results page 🚧.

Resources