Auto-tools/Projects/TestHarnesses
Contents
Team
Problem
Mozilla has thousands of tests which run 10+ harnesses. As Firefox evolves in features and supported platforms, the test harnesses need to accommodate the needs and challenges faced to keep our new and existing tests running reliably.
Goals & Considerations
Our primary goal is to ensure our harnesses share as much code as possible and remove old code which depended on older features (of python, firefox, etc.) or was not written in a scalable way. Other goals are to simplify our manifests used while allowing the manifest solution to have more flexibility in scheduling tests.
Non-Goals
We are not interested in rewriting our harnesses from scratch or making the perfect harness. We are not looking to switch from Python and Javascript.
Dependencies / Who will use this
Harnesses are primarily used by automation and developers. We need to support harnesses from up to 9 revisions ago (almost a year) when we look at ESR and older B2G branches.
Design and Approach
Overview of test jobs and their harnesses]
Milestones and Dates
As of March 2015, our current priorities are:
- fine grain test selection
- json manifests android, webgl, imptests
- mochitest chrome on b2g
- ssltunnel to android
- error reporting for asan xpcshell
- windows process ctrl+c
- run from tests.zip
Implementation
Getting Involved
Getting Started
The best way is to find a bug and ask the mentor. In general, it is best to have a local build of Firefox
Expectations
For the majority of our work, it lives in mozilla-central (hg, not git) and we follow these guidelines.
Bugs
ID | Summary | Priority | Status |
---|---|---|---|
1112322 | Improve error reporting on xpcshell ASAN failures | -- | NEW |
1124182 | [Tracking] Fine grained test selection | -- | NEW |
2 Total; 2 Open (100%); 0 Resolved (0%); 0 Verified (0%);