Auto-tools/Projects/Bisect in the cloud

Team

David Burns (AutomatedTester)

Problem

As we look to run specific tests to a revision we there are possible times that we miss catching a regression. To do this we need to have a mechanism that allows us to bisect builds really quickly on a number of different platforms.

This will allow developers and sheriffs to find the bad revision really quickly to either fix or back out of the tree.

Goals & Considerations

Overall Requirements

• The ability to input a bad and good revision, the test that can reproduce this issue and which tree to run against
• Be able to work again different types of regression like tests or build
• Bisect on Pushes and Revisions
• Output results to be consumed by interfaces like TBPL
• Ability to handle custom test vs running existing test suite
• Have the ability to spawn multiple builds based on group requesting. E.g. Sheriffs have tree closed and need to bisect quickly.

Considerations

• Can we extend MozRegression to be core to this so we dont have to do builds again
• Can we use the Self-Serve API for some tasks to make things quicker
• We need be able to reuse RelEng infrastructure to run tasks.

Non-Goals

Anything of note that is specifically not going to be accomplished in this project. The "what not".

Design and Approach

High-level design ideas and concepts. The "how" in a general sense.

Implementation

Technical notes, plans, and designs detailing how the project will be realized. The specifics of "how".