Firefox Performance Test Engineering 🔥🦊⏱

From MozillaWiki
Jump to: navigation, search


What we do

What we don't do

Onboarding

Objectives

  • Increase regression detection coverage to page load on Android products including WebView comparisons
  • Increase quality standards by measuring and alerting on power usage for Android and ARM64
  • Build dashboards to improve visibility of how we are tracking against our release criteria for Android
  • Reducing noise in test results and adding annotations to provide clarity of signal for regressions
  • Develop measurements and mechanisms for reporting on our tools, policies and documentation for how to improve clarity and efficiency of risk assessments

Team

  • Dave Hunt [:davehunt]
  • Rob Wood [:rwood]
  • Stephen Donner [:stephend]
  • Greg Mierzwinski [:sparky]
  • Ionuț Goldan [:igoldan]
  • Florian Strugariu [:bebe]
  • Marian Raiciof [:marauder]
  • Alex Ionescu [:alexandrui]

Projects

Trello: https://trello.com/b/gc3AlyhB/firefox-performance-test-engineering

Raptor

Raptor is a Python testing framework used to run browser benchmark and browser page-load performance tests. Raptor is cross-browser compatible and is currently running in Mozilla taskcluster production on Firefox Desktop, Firefox Android, and on Google Chromium.

Talos

Talos is a Python performance testing framework that is usable on Windows, Mac and Linux. Talos is our versatile performance testing framework we use at Mozilla. It was created to serve as a test runner for the existing performance tests that Mozilla was running back in 2007 as well as providing an extensible framework for new tests as they were created.

WebPageTest

Perfherder

Perfherder is an interactive dashboard intended to allow monitoring and analysis of automated performance tests run against Mozilla products (currently Firefox and Firefox for Android). Perfherder is part of the Treeherder project.

Firefox Are We Fast Yet Dashboard

Shows a variety of benchmarks, run on a variety of platforms. Meant to be a detailed view of performance.

Firefox Health Dashboard

The health dashboard tracks metrics and statistics important for tracking performance improvements. Meant to be a high level view.

Resources