TestEngineering: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(add testsuite testrail link column)
(Formatting)
 
(61 intermediate revisions by 9 users not shown)
Line 1: Line 1:
== Mission Statement ==
The Test Engineering team supports the larger Firefox organization by enabling identification of risk to our products and eliminating quality related blind spots through tooling, testing, automation, benchmarks, and reporting.


= Overview =
More info is at: http://firefox-test-engineering.readthedocs.io/en/latest/
Firefox Test Engineering oversees the test automation, infrastructure, and manual testing for all of Firefox's services, sites, and apps that are external to the core browsers (and in some cases within the browsers as well).


= Mission =
== Focus Areas ==
Our mission is to provide testing and tools to positively impact the quality of Mozilla sites, apps, and services.
* Interop
* Performance
* GeckoView
* Fenix
* Amazon
* CI Automation
* Ecosystem
* Telemetry
* Premium Services
* Energy
* Accessibility
* Reporting
* Monitoring
* Experiments


= The Team =
== Enduring Objectives ==
* Manager: Stuart Philp
* Increase visibility into product risks
* Technical Lead: Krupa Raj
** Streamline reporting
* Technical Lead: [https://mozillians.org/u/davehunt/ Dave Hunt]
** Reduce blind spots
* Technical Lead: Richard Pappalardo
* Improve efficiency and clarity of risk assessments
* Senior Test Engineer: [[User:Kthiessen|Karl Thiessen]]
** Tight feedback loops
* Senior Test Engineer: Peter deHaan
** Be opinionated (i.e binary or traffic light recommendations)
* Senior Test Engineer: Chris Hartjes
** Reduce guesswork, roundtrips
* Senior Test Engineer: [https://mozillians.org/en-US/u/stephend/ Stephen Donner]
** Standardize between projects and types of reports
* Senior Test Engineer: Kevin Brosnan
* Maintain or improve coverage and quality standards
* Senior Test Engineer: Aaron Train
** Identify and fill gaps
* Senior Test Engineer: Matt Brandt
** Document and measure existing work
* Senior Test Engineer: Rebecca Billings
** Establish/Improve release criteria consistency
* Senior Test Engineer: John Dorlus
** Reduce effort on maintenance
* Senior Test Engineer: No-Jun Park
* Reduce cross-functional friction
* Junior Test Engineer: Benny Forehand Jr.
** Advocacy
** Reporting
** Tools
** Benchmarks
** Relationship building


=Technical Groups=
== People ==
Firefox Test Engineering is divided into three technical groups. These groups align on the skills, tools, and focus needed to provide the coverage and quality for the projects they oversee. Each technical group has a technical lead, who oversees all of the project's testing needs and best practices.
Director: Stuart Philp
 
<br>Architect: Tarek Ziade
==User Interfaces==
<br>Interop. Manager: David Burns
* https://wiki.mozilla.org/TestEngineering/UI
<br>Performance. Manager: Dave Hunt
* Lead: Dave Hunt
<br>Ecosystem. Manager: Krupa Raj
 
<br>CI Automation. Manager: Joel Maher
==Services==
<br>Mobile. Manager: No-Jun Park
* https://wiki.mozilla.org/TestEngineering/Services
* Lead: Richard Pappalardo
 
==Customization==
* https://wiki.mozilla.org/TestEngineering/Customization
* Lead: Krupa Raj
 
=Full Project List=
{| class="wikitable sortable"
|-
! Project !! Primary !! Secondary !! Channel !! Technical Group !! Test Suite !! Unit !! Functional/UI !! Load !! Performance !! Accessibility !! Security !! Localization !! Swagger (route def)
|-
| PageShot (Test Pilot) || Peter || Softvision || #pageshot || Customization || N/A || N/A
| style="background:pink;text-align:center;" | none
|| N/A || N/A || N/A || N/A || N/A || N/A
|-
| NoMore404s (Test Pilot) || Karl || Softvision || #testpilot || Customization || N/A || N/A
| style="background:pink;text-align:center;" | none
|| N/A || N/A || N/A || N/A || N/A || N/A
|-
| MinVid (Test Pilot) || Peter || Softvision || #testpilot || Customization || N/A || N/A
| style="background:pink;text-align:center;" | none
|| N/A || N/A || N/A || N/A || N/A || N/A
|-
| Fathom (Test Pilot) || Peter || Softvision || #testpilot || Customization || N/A || N/A
| style="background:pink;text-align:center;" | none
|| N/A || N/A || N/A || N/A || N/A || N/A
|-
| Heatmap (Test Pilot) || Peter || Softvision || #testpilot || Customization || N/A || N/A
| style="background:pink;text-align:center;" | none
|| N/A || N/A || N/A || N/A || N/A || N/A
|-
| Universal Search (Test Pilot) || Stephen || Peter || #universal-search || Customization || N/A || N/A
| style="background:pink;text-align:center;" | none
|| N/A || N/A || N/A || N/A || N/A || N/A
|-
| Activity Stream (Test Pilot) || Peter || John || #activity-stream || Customization || N/A || N/A
| style="background:pink;text-align:center;" | none
|| N/A || N/A || N/A || N/A || N/A || N/A
|-
| WebExtensions || Krupa || Softvision || #webextensions || Customization
| style="background:lightgreen;text-align:center;" | [https://testrail.stage.mozaws.net/index.php?/suites/view/37]
|| N/A
| style="background:pink;text-align:center;" | none
|| N/A || N/A || N/A || N/A || N/A || N/A
|-
| Test Pilot (web site) || Peter || John || #testpilot || Customization || N/A || N/A
| style="background:pink;text-align:center;" | none
|| N/A || N/A || N/A || N/A || N/A || N/A
|-
| Socorro || Matt || Stephen || #socorro || Customization || N/A || N/A
| style="background:lightgreen;text-align:center;" | [https://github.com/mozilla/socorro/tree/master/e2e-tests]
|| N/A || N/A || N/A || N/A || N/A || N/A
|-
| Telemetry || John || Softvision || #telemetry || Customization || N/A || N/A
| style="background:pink;text-align:center;" | none
|| N/A || N/A || N/A || N/A || N/A || N/A
|-
| addons.mozilla.org || Krupa || Softvision || #amo || Customization || N/A || N/A
| style="background:lightgreen;text-align:center;" | [https://github.com/mozilla/Addon-Tests] [https://github.com/mozilla/addons-server/tree/master/tests/ui] [https://github.com/mozilla/addons-frontend/tree/master/tests/ui]
|| N/A || N/A || N/A || N/A || N/A || N/A
|-
| TabCenter (Test Pilot) || Peter || Softvision || #testpilot || Customization || N/A || N/A
| style="background:pink;text-align:center;" | none
|| N/A || N/A || N/A || N/A || N/A || N/A
|-
| Blok (Test Pilot) || Rebecca || Softvision || #testpilot || Customization || N/A || N/A
| style="background:pink;text-align:center;" | none
|| N/A || N/A || N/A || N/A || N/A || N/A
|-
| Treeherder || Rebecca || Dave || #treeherder || User Interface || N/A || N/A
| style="background:lightgreen;text-align:center;" | [https://github.com/mozilla/treeherder-tests]
|| N/A || N/A || N/A || N/A || N/A || N/A
|-
| developer.mozilla.org || Matt || Dave || #mdndev || User Interface || N/A || N/A
| style="background:lightgreen;text-align:center;" | [https://github.com/mozilla/kuma/tree/master/tests/ui] [https://github.com/mozilla/kuma/tree/master/tests/functional]
|| N/A || N/A || N/A || N/A || N/A || N/A
|-
| mozilla.org || Rebecca || Dave || #bedrock || User Interface || N/A || N/A
| style="background:lightgreen;text-align:center;" | [https://github.com/mozilla/bedrock/tree/master/tests/functional]
|| N/A || N/A || N/A || N/A || N/A || N/A
|-
| FoxPuppet || Benny || Dave/John || #fx-test || User Interface || N/A || N/A
| style="background:lightgray;text-align:center;" | n/a
|| N/A || N/A || N/A || N/A || N/A || N/A
|-
| Activity Stream (iOS) || Aaron || No-Jun || #mobile || User Interface || N/A || N/A
| style="background:pink;text-align:center;" | none
|| N/A || N/A || N/A || N/A || N/A || N/A
|-
| Activity Stream (Android) || Kevin || No-Jun || #mobile || User Interface || N/A || N/A
| style="background:pink;text-align:center;" | none
|| N/A || N/A || N/A || N/A || N/A || N/A
|-
| Prox (iOS) || Aaron || No-Jun || #mobile || User Interface || N/A || N/A
| style="background:pink;text-align:center;" | none
|| N/A || N/A || N/A || N/A || N/A || N/A
|-
| Focus (iOS) || Aaron || No-Jun || #mobile || User Interface || N/A || N/A
| style="background:pink;text-align:center;" | none
|| N/A || N/A || N/A || N/A || N/A || N/A
|-
| ABSearch || Karl || Chris || #absearch || Services || N/A || N/A
| style="background:pink;text-align:center;" | none
|| N/A
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|| N/A
| style="background:lightgray;text-align:center;" | n/a
|| N/A
|-
| Balrog || Chris || Karl || #balrog || Services || N/A || N/A
| style="background:pink;text-align:center;" | none
|| N/A
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|| N/A
| style="background:lightgray;text-align:center;" | n/a
|| N/A
|-
| Bouncer || Matt || Dave || #stubby || Services || N/A || N/A
| style="background:lightgreen;text-align:center;" | [https://github.com/mozilla-services/go-bouncer/tree/master/tests/e2e]
| style="background:lightgreen;text-align:center;" | TBD
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|| N/A
| style="background:lightgray;text-align:center;" | n/a
|| N/A
|-
| Firefox Accounts || Karl || Peter || #fxa || Services || N/A || N/A
| style="background:lightgreen;text-align:center;" | [https://github.com/mozilla/fxa-content-server#testing] [https://github.com/mozilla/fxapom]
|| N/A
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|| N/A
| style="background:lightgray;text-align:center;" | n/a
|| N/A
|-
| [https://kinto.readthedocs.io Kinto] || Chris || Karl || #storage || Services || N/A || N/A
| style="background:lightgrey;text-align:center;" | n/a
| style="background:lightgreen;text-align:center;" | TBD
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|| N/A
| style="background:lightgray;text-align:center;" | n/a
|| N/A
|-
| [https://github.com/loads Loads Tool] || Richard || Chris || #fx-test || Services || N/A || N/A
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|| N/A
|-
| [http://autopush.readthedocs.io Push Notifications] || Richard || Rebecca || #push || Services || N/A
| style="background:lightgreen;text-align:center;" | [https://github.com/mozilla-services/autopush]
| style="background:lightgreen;text-align:center;"  | TBD
| style="background:lightgreen;text-align:center;" | [https://github.com/mozilla-services/ap-loadtester]
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|| N/A
| style="background:lightgray;text-align:center;" | n/a
|| N/A
|-
| Shavar (Tracking Protection) || Rebecca || Richard || #shavar || Services || N/A
| style="background:lightgreen;text-align:center;" |[https://github.com/mozilla-services/shavar/]
| style="background:lightgreen;text-align:center;" | TBD
| style="background:lightgreen;text-align:center;" | [https://github.com/rpappalax/shavar-loadtests]
| style="background:lightgreen;text-align:center;" | TBD
| style="background:lightgray;text-align:center;" | n/a
|| N/A
| style="background:lightgray;text-align:center;" | n/a
|| N/A
|-
| SHIELD || Chris || Karl || #normandy || Services || N/A || N/A
| style="background:pink;text-align:center;" | none
|| N/A
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|| N/A
| style="background:lightgray;text-align:center;" | n/a
|| N/A
|-
| [https://wiki.mozilla.org/Firefox/Stub_Attribution Stub Attribution] || Stephen || Matt || #stubby || Services || N/A
| style="background:lightgreen;text-align:center;" | [https://github.com/mozilla-services/stub_attribution] [https://github.com/mozilla/bedrock/tree/master/bedrock/firefox/tests]
| style="background:lightgreen;text-align:center;" | TBD
|| N/A
| style="background:lightgreen;text-align:center;" | TBD
| style="background:lightgray;text-align:center;" | n/a
|| N/A
| style="background:lightgray;text-align:center;" | n/a
|| N/A
|-
| [https://wiki.mozilla.org/Services/Sync Sync] || Karl || Chris || #sync || Services || N/A || N/A
| style="background:pink;text-align:center;" | none
| style="background:lightgreen;text-align:center;" | TBD
| style="background:lightgray;text-align:center;" | n/a
| style="background:lightgray;text-align:center;" | n/a
|| N/A
| style="background:lightgray;text-align:center;" | n/a
|| N/A
|}

Latest revision as of 06:11, 13 December 2024

Mission Statement

The Test Engineering team supports the larger Firefox organization by enabling identification of risk to our products and eliminating quality related blind spots through tooling, testing, automation, benchmarks, and reporting.

More info is at: http://firefox-test-engineering.readthedocs.io/en/latest/

Focus Areas

  • Interop
  • Performance
  • GeckoView
  • Fenix
  • Amazon
  • CI Automation
  • Ecosystem
  • Telemetry
  • Premium Services
  • Energy
  • Accessibility
  • Reporting
  • Monitoring
  • Experiments

Enduring Objectives

  • Increase visibility into product risks
    • Streamline reporting
    • Reduce blind spots
  • Improve efficiency and clarity of risk assessments
    • Tight feedback loops
    • Be opinionated (i.e binary or traffic light recommendations)
    • Reduce guesswork, roundtrips
    • Standardize between projects and types of reports
  • Maintain or improve coverage and quality standards
    • Identify and fill gaps
    • Document and measure existing work
    • Establish/Improve release criteria consistency
    • Reduce effort on maintenance
  • Reduce cross-functional friction
    • Advocacy
    • Reporting
    • Tools
    • Benchmarks
    • Relationship building

People

Director: Stuart Philp
Architect: Tarek Ziade
Interop. Manager: David Burns
Performance. Manager: Dave Hunt
Ecosystem. Manager: Krupa Raj
CI Automation. Manager: Joel Maher
Mobile. Manager: No-Jun Park

Retrieved from "https://wiki.mozilla.org/index.php?title=TestEngineering&oldid=1252953"