Auto-tools/Projects/TestHarnesses

From MozillaWiki
Jump to: navigation, search

Team

Problem

Mozilla has thousands of tests which run 10+ harnesses. As Firefox evolves in features and supported platforms, the test harnesses need to accommodate the needs and challenges faced to keep our new and existing tests running reliably.

Goals & Considerations

Our primary goal is to ensure our harnesses share as much code as possible and remove old code which depended on older features (of python, firefox, etc.) or was not written in a scalable way. Other goals are to simplify our manifests used while allowing the manifest solution to have more flexibility in scheduling tests.

Non-Goals

We are not interested in rewriting our harnesses from scratch or making the perfect harness. We are not looking to switch from Python and Javascript.

Dependencies / Who will use this

Harnesses are primarily used by automation and developers. We need to support harnesses from up to 9 revisions ago (almost a year) when we look at ESR and older B2G branches.

Design and Approach

Overview of test jobs and their harnesses]

Milestones and Dates

As of March 2015, our current priorities are:

Implementation

Getting Involved

Getting Started

The best way is to find a bug and ask the mentor. In general, it is best to have a local build of Firefox

Expectations

For the majority of our work, it lives in mozilla-central (hg, not git) and we follow these guidelines.

Bugs

Full Query
ID Summary Priority Status
1112322 Improve error reporting on xpcshell ASAN failures -- NEW
1124182 [Tracking] Fine grained test selection -- NEW

2 Total; 2 Open (100%); 0 Resolved (0%); 0 Verified (0%);