canmove, Confirmed users, Bureaucrats and Sysops emeriti
4,747
edits
QA/Automation/Projects/Mozmill Automation/On Demand Test Framework: Difference between revisions
QA/Automation/Projects/Mozmill Automation/On Demand Test Framework (view source)
Revision as of 10:40, 13 January 2014
, 13 January 2014Whimboo moved page Auto-tools/Automation Development/Projects/Mozmill Automation/On Demand Test Framework to QA/Automation/Projects/Mozmill Automation/On Demand Test Framework: Team reorganization
m (Whimboo moved page Auto-tools/Automation Development/Projects/Mozmill Automation/On Demand Test Framework to QA/Automation/Projects/Mozmill Automation/On Demand Test Framework: Team reorganization) |
|||
| (7 intermediate revisions by 3 users not shown) | |||
| Line 18: | Line 18: | ||
|- valign="top" | |- valign="top" | ||
| '''Pivotal:''' | | '''Pivotal:''' | ||
| | | https://www.pivotaltracker.com/projects/404111 | ||
|- valign="top" | |- valign="top" | ||
| '''Bugzilla:''' | | '''Bugzilla:''' | ||
| | | https://bugzilla.mozilla.org/show_bug.cgi?id=698654 | ||
|- valign="top" | |||
| '''Other Docs:''' | |||
| [[/Docs|Index]] | |||
|} | |} | ||
| Line 34: | Line 37: | ||
| Implementation of a remote mechanism to trigger Mozmill functional and update tests for Firefox releases | | Implementation of a remote mechanism to trigger Mozmill functional and update tests for Firefox releases | ||
|} | |} | ||
==Requirements== | |||
* We want to be able to kick off test runs with as little interaction and micromanaging as we can. | |||
* A test run equates to a script with one or more parameters, which will be run on one or more environments. These parameters correspond mostly to the test matrix, and will vary per run type. | |||
* The test run is capable of, within a given environment, doing everything from app deployment to testing to app cleanup. Or we can split these up if more convenient. | |||
* We'll want the system to spread testing among a number of available machines per environment, and ultimately create the machines as needed. | |||
* Ultimately, we'd like the ability to slice the test run on a parameter value to run in parallel. If we can get it, understanding how to slice the test run per-test would be even better. | |||
* We'd like to clearly monitor run progress, and watchdog for hung machines. | |||
* We'd like to see understandable test results, preferably consolidated across a whole run (at least per-env, if not cross-env). | |||
==Project Milestones== | ==Project Milestones== | ||
| Line 92: | Line 105: | ||
==Resources== | ==Resources== | ||
* [[/CI Documentation|Continuous Integration Details]] | |||
* [[/Public|Public Posts and Discussions]] | * [[/Public|Public Posts and Discussions]] | ||
* [[/Presentations|Presentations]] | * [[/Presentations|Presentations]] | ||
* [[/FAQ|FAQ]] | * [[/FAQ|FAQ]] | ||
* [[/Contribution|How to Contribute]] | * [[/Contribution|How to Contribute]] | ||