Auto-tools/Projects/Bisect in the cloud

Team

David Burns (AutomatedTester)

Problem

As we look to run specific tests to a revision we there are possible times that we miss catching a regression. To do this we need to have a mechanism that allows us to bisect builds really quickly on a number of different platforms.

This will allow developers and sheriffs to find the bad revision really quickly to either fix or back out of the tree.

Goals & Considerations

Overall Requirements

• The ability to input a bad and good revision, the test that can reproduce this issue and which tree to run against
• Be able to work again different types of regression like tests or build
• Bisect on Pushes and Revisions
• Output results to be consumed by interfaces like TBPL
• Ability to handle custom test vs running existing test suite
• Have the ability to spawn multiple builds based on group requesting. E.g. Sheriffs have tree closed and need to bisect quickly.

Considerations

• How can we use mach
• Can we extend MozRegression to be core to this so we dont have to do builds again
• Can we use the Self-Serve API for some tasks to make things quicker
• We need be able to reuse RelEng infrastructure to run tasks.

Non-Goals

This is not building out infrastructure, nor the tools to run tests. This will just call a scheduler and download the builds(if they are available) to be distributed to the relevant machines for running

Design and Approach

Ideas to approach

• Re-use MozRegression when range is within 30 days
  • If not schedule a build that we can then use
• Push different items into a queue that we can then pop so tasks are managed properly
• have web interface, using playdoh, to manage tasks

Implementation

Technical notes, plans, and designs detailing how the project will be realized. The specifics of "how".