MozillaWiki

TestEngineering: Difference between revisions

Page Discussion

TestEngineering (view source)

Revision as of 06:11, 13 December 2024

21,277 bytes removed ,  13 December 2024
Formatting
(update loads and antenna projects)
(Formatting)
 
(47 intermediate revisions by 7 users not shown)
Line 1: Line 1:
== Mission Statement ==
The Test Engineering team supports the larger Firefox organization by enabling identification of risk to our products and eliminating quality related blind spots through tooling, testing, automation, benchmarks, and reporting.


= Overview =
More info is at: http://firefox-test-engineering.readthedocs.io/en/latest/
Firefox Test Engineering oversees the test automation, infrastructure, and manual testing for all of Firefox's services, sites, and apps that are external to the core browsers (and in some cases within the browsers as well).


= Mission =
== Focus Areas ==
Our mission is to provide testing and tools to positively impact the quality of Mozilla sites, apps, and services.
* Interop
* Performance
* GeckoView
* Fenix
* Amazon
* CI Automation
* Ecosystem
* Telemetry
* Premium Services
* Energy
* Accessibility
* Reporting
* Monitoring
* Experiments


= The Team =
== Enduring Objectives ==
* Manager: Stuart Philp
* Increase visibility into product risks
* Technical Lead: Krupa Raj
** Streamline reporting
* Technical Lead: [https://mozillians.org/u/davehunt/ Dave Hunt]
** Reduce blind spots
* Technical Lead: Richard Pappalardo
* Improve efficiency and clarity of risk assessments
* Staff Engineer: Tarek Ziade
** Tight feedback loops
* Senior Test Engineer: [[User:Kthiessen|Karl Thiessen]]
** Be opinionated (i.e binary or traffic light recommendations)
* Senior Test Engineer: Peter deHaan
** Reduce guesswork, roundtrips
* Senior Test Engineer: Chris Hartjes
** Standardize between projects and types of reports
* Senior Test Engineer: [https://mozillians.org/en-US/u/stephend/ Stephen Donner]
* Maintain or improve coverage and quality standards
* Senior Test Engineer: Kevin Brosnan
** Identify and fill gaps
* Senior Test Engineer: Aaron Train
** Document and measure existing work
* Senior Test Engineer: Matt Brandt
** Establish/Improve release criteria consistency
* Senior Test Engineer: Rebecca Billings
** Reduce effort on maintenance
* Senior Test Engineer: John Dorlus
* Reduce cross-functional friction
* Senior Test Engineer: No-Jun Park
** Advocacy
* Junior Test Engineer: Benny Forehand Jr.
** Reporting
** Tools
** Benchmarks
** Relationship building


=Technical Groups=
== People ==
Firefox Test Engineering is divided into three technical groups. These groups align on the skills, tools, and focus needed to provide the coverage and quality for the projects they oversee. Each technical group has a technical lead, who oversees all of the project's testing needs and best practices.
Director: Stuart Philp
 
<br>Architect: Tarek Ziade
==User Interfaces==
<br>Interop. Manager: David Burns
* https://wiki.mozilla.org/TestEngineering/UI
<br>Performance. Manager: Dave Hunt
* Lead: Dave Hunt
<br>Ecosystem. Manager: Krupa Raj
 
<br>CI Automation. Manager: Joel Maher
==Services==
<br>Mobile. Manager: No-Jun Park
* https://wiki.mozilla.org/TestEngineering/Services
* Lead: Richard Pappalardo
 
==Customization==
* https://wiki.mozilla.org/TestEngineering/Customization
* Lead: Krupa Raj
 
=Full Project List=
{| class="wikitable sortable"
|-
! Project !! Primary !! Secondary !! IRC Channel !! Technical Group !! Test Suite !! Unit !! Functional/UI !! Load !! Performance !! Accessibility !! Security !! Localization !! Swagger (route def)
|-
| PageShot (Test Pilot) || Peter || Softvision || #pageshot || Customization
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:pink;text-align:center;" | none
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| NoMore404s (Test Pilot) || Karl || Softvision || #testpilot || Customization
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:pink;text-align:center;" | none
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| MinVid (Test Pilot) || Peter || Softvision || #testpilot || Customization
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:pink;text-align:center;" | none
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| Fathom (Test Pilot) || Peter || Softvision || #testpilot || Customization
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:pink;text-align:center;" | none
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| Heatmap (Test Pilot) || Peter || Softvision || #testpilot || Customization
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:pink;text-align:center;" | none
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| Universal Search (Test Pilot) || Stephen || Peter || #universal-search || Customization
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:pink;text-align:center;" | none
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| Activity Stream (Test Pilot) || Peter || John || #activity-stream || Customization
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:pink;text-align:center;" | none
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| WebExtensions || Krupa || Softvision || #webextensions || Customization
| style="background:lightgreen;text-align:center;" | [https://testrail.stage.mozaws.net/index.php?/suites/view/37]
| style="background:lightgray;text-align:center;" | n/a
| style="background:pink;text-align:center;" | none
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| Test Pilot (web site) || Peter || John || #testpilot || Customization
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:pink;text-align:center;" | none
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| Socorro || Matt || Richard || #socorro || Customization
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgreen;text-align:center;" | [https://github.com/mozilla/socorro/tree/master/e2e-tests]
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| Telemetry || John || Softvision || #telemetry || Customization
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:pink;text-align:center;" | none
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| addons.mozilla.org || Krupa || Softvision || #amo || Customization
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgreen;text-align:center;" | [https://github.com/mozilla/Addon-Tests] [https://github.com/mozilla/addons-server/tree/master/tests/ui] [https://github.com/mozilla/addons-frontend/tree/master/tests/ui]
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| TabCenter (Test Pilot) || Peter || Softvision || #testpilot || Customization
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:pink;text-align:center;" | none
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| Blok (Test Pilot) || Rebecca || Softvision || #testpilot || Customization
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:pink;text-align:center;" | none
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| Treeherder || Rebecca || Dave || #treeherder || User Interface
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgreen;text-align:center;" | [https://github.com/mozilla/treeherder-tests]
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| developer.mozilla.org || Matt || Dave || #mdndev || User Interface
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgreen;text-align:center;" | [https://github.com/mozilla/kuma/tree/master/tests/ui] [https://github.com/mozilla/kuma/tree/master/tests/functional]
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| mozilla.org || Rebecca || Dave || #bedrock || User Interface
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgreen;text-align:center;" | [https://github.com/mozilla/bedrock/tree/master/tests/functional]
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| FoxPuppet || Benny || Dave/John || #fx-test || User Interface
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| Activity Stream (iOS) || Aaron || No-Jun || #mobile || User Interface
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:pink;text-align:center;" | none
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| Activity Stream (Android) || Kevin || No-Jun || #mobile || User Interface
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:pink;text-align:center;" | none
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| Prox (iOS) || Aaron || No-Jun || #mobile || User Interface
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:pink;text-align:center;" | none
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| Focus (iOS) || Aaron || No-Jun || #mobile || User Interface
| style="background:lightgreen;text-align:center;" | [https://testrail.stage.mozaws.net/index.php?/suites/view/321]
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgreen;text-align:center;" | [https://github.com/mozilla-mobile/focus/tree/master/XCUITest]
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| ABSearch || Karl || Chris || #absearch || Services
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:pink;text-align:center;" | none
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| [https://github.com/mozilla/antenna Antenna] || Matt || Richard || #breakpad || Services
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightyellow;text-align:center;"  | TBD
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| [http://autopush.readthedocs.io Autopush] || Richard || Rebecca || #push || Services
| style="background:lightgreen;text-align:center;" |  [https://testrail.stage.mozaws.net/index.php?/projects/overview/31]
| style="background:lightgreen;text-align:center;" | [https://github.com/mozilla-services/autopush]
| style="background:lightyellow;text-align:center;"  | TBD
| style="background:lightgreen;text-align:center;" | [https://github.com/mozilla-services/ap-loadtester]
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| Balrog || Chris || Karl || #balrog || Services
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:pink;text-align:center;" | none
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| Bouncer || Matt || Dave || #stubby || Services
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgreen;text-align:center;" | [https://github.com/mozilla-services/go-bouncer/tree/master/tests/e2e]
| style="background:lightyellow;text-align:center;" | TBD
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| Firefox Accounts || Karl || Peter || #fxa || Services
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgreen;text-align:center;" | [https://github.com/mozilla/fxa-content-server#testing] [https://github.com/mozilla/fxapom]
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| [https://kinto.readthedocs.io Kinto] || Chris || Karl || #storage || Services
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgrey;text-align:center;" | n/a
| style="background:lightyellow;text-align:center;" | TBD
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| [https://github.com/loads Loads Tool] || Richard || Chris || #fx-test || Services
| style="background:darkgray;text-align:center;" | N/A
| style="background:lightgray;text-align:center;" | n/a
| style="background:darkgray;text-align:center;" | N/A
| style="background:darkgray;text-align:center;" | N/A
| style="background:darkgray;text-align:center;" | N/A
| style="background:darkgray;text-align:center;" | N/A
| style="background:lightgray;text-align:center;" | n/a
| style="background:darkgray;text-align:center;" | N/A
| style="background:lightgray;text-align:center;" | n/a
|-
| Shavar (Tracking Protection) || Rebecca || Richard || #shavar || Services
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgreen;text-align:center;" |[https://github.com/mozilla-services/shavar/]
| style="background:lightyellow;text-align:center;" | TBD
| style="background:lightgreen;text-align:center;" | [https://github.com/rpappalax/shavar-loadtests]
| style="background:lightyellow;text-align:center;" | TBD
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgreen;text-align:center;" | n/a
|-
| SHIELD || Chris || Karl || #normandy || Services
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:pink;text-align:center;" | none
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| [https://wiki.mozilla.org/Firefox/Stub_Attribution Stub Attribution] || Stephen || Matt || #stubby || Services
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgreen;text-align:center;" | [https://github.com/mozilla-services/stub_attribution] [https://github.com/mozilla/bedrock/tree/master/bedrock/firefox/tests]
| style="background:lightyellow;text-align:center;" | TBD
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightyellow;text-align:center;" | TBD
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|-
| [https://wiki.mozilla.org/Services/Sync Sync] || Karl || Chris || #sync || Services
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:pink;text-align:center;" | none
| style="background:lightyellow;text-align:center;" | TBD
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|}
JeanPaulLucien
Confirmed users
514

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1156927...1252953"