QA/Automation/Projects/Mozmill Automation/Mozmill CI: Difference between revisions

Marking pages as obsolete
(Marking pages as obsolete)
 
(8 intermediate revisions by 3 users not shown)
Line 1: Line 1:
__NOTOC__
__NOTOC__
'''OBSOLETE - The system is no longer running. See https://bugzilla.mozilla.org/show_bug.cgi?id=1386628'''
=Overview=
=Overview=
The Mozmill Automation team currently performs Mozmill tests triggered by a daily cron job. This approach has some weaknesses:
The Automation Development team is working on a Mozmill CI solution which triggers Mozmill tests immediately when a new build of Firefox has been made available.
* If no build lands, we still attempt to perform Mozmill tests
* If a build lands off schedule, we attempt to test it at the wrong time or miss it completely
* There's no way to use this style of trigger to pre-qualify a build (or even promptly post-qualify it) so results have limited impact
* Because we update build-over-build, using the same installation, anomalies in testing (or bugs in a given build) can cause the process to get "off-track" and poison subsequent Mozmill tests.
 
Instead, we'd like to move towards performing Mozmill tests immediately when a build has been made available. We'd also like to independently download and verify each day's build, rather than relying on using the same install day over day.


The ultimate goal is to build a system within QA, then hand off the process to Release Engineering for ongoing maintenance.
The ultimate goal is to build a system within QA, then hand off the process to Release Engineering for ongoing maintenance.
Line 19: Line 16:
  |- valign="top"
  |- valign="top"
  | '''Leads:'''
  | '''Leads:'''
  | [mailto:hskupin@mozilla.com Henrik Skupin], [mailto:dhunt@mozilla.com Dave Hunt]
  | [mailto:hskupin@mozilla.com Henrik Skupin]
  |- valign="top"
  |- valign="top"
  | '''Contributors:'''
  | '''Contributors:'''
  | Geo Mealer
  | [mailto:andreea.matei@softvisioninc.eu Andreea Matei], [mailto:cosmin.malutan@softvision.ro Cosmin Malutan]
|- valign="top"
| '''Tracker:'''
| https://www.pivotaltracker.com/projects/298905?label=Triggered%20Daily%20Tests
|- valign="top"
| '''Tracking Bug:'''
| https://bugzilla.mozilla.org/show_bug.cgi?id=709052
  |- valign="top"
  |- valign="top"
  | '''Repository:'''  
  | '''Repository:'''  
  | http://hg.mozilla.org/qa/mozmill-automation/
  | https://github.com/mozilla/mozmill-ci
  |- valign="top"
  |- valign="top"
  | '''Etherpad:'''
  | '''Etherpad:'''
  | http://etherpad.mozilla.com:9000/qas-triggered-tests
  | https://etherpad.mozilla.org/mozmill-ci
|}
|}


Line 40: Line 31:
A complete set of documents can be found on the pages linked below:  
A complete set of documents can be found on the pages linked below:  


* [[/Duties|Duties]]
* [[/Requirements|Requirements]]
* [[/Requirements|Requirements]]
* [[/Design Documents|Design Documents]]
* [[/Design Documents|Design Documents]]
Line 51: Line 43:
| style="background:#EFEFEF; width:66%" | '''Details'''
| style="background:#EFEFEF; width:66%" | '''Details'''
|- valign="top"
|- valign="top"
| Dec 12th, 2011
| Sep 30th, 2014
| -
| [https://github.com/mozilla/mozmill-ci/issues/270 Issue #270]
| Documentation ready for CI implementation
| Get full coverage of all locales for Firefox beta and release testing
|- valign="top"
| Dec 31st, 2011
| -
| CI system running for daily tests
|}
|}
canmove, Confirmed users, Bureaucrats and Sysops emeriti
4,747

edits