User:Rhelmer:Nightly Release Automation schedule

From MozillaWiki
Jump to: navigation, search

We are moving nightlies to the system used for release automation.

The initial goal is to replace multi-tinderbox with Buildbot, and use the bootstrap Build step. Tinderbox will continue to handle build, push, updates, but these will be gradually moved to bootstrap steps and out of Tinderbox.

Mozilla 1.8 branch rollout

We've started by installing 3 nightly/depend builders on staging-1.8-master, and used the existing release automation slaves (which started out as clones of the Mozilla 1.8 nightly/depend builders):

  • staging-prometheus-vm (linux)
  • staging-pacifica-vm (win32)
  • bm-xserve03 (macosx)

These use Tinderbox client to report individually to Mozilla 1.8 Staging Tinderbox tree.

The next step is to roll this out to the production machines, and start reporting to the Mozilla 1.8 Tinderbox tree. These are also clones of the Mozilla 1.8 nightly/depend builders:

  • production-prometheus-vm (linux)
  • production-pacifica-vm (win32)
  • bm-xserve05 (macosx)

We need to be able to fall back to the old system if there are problems, and we feel that it's desirable to run both systems in parallel for a week to make sure that there are no differences before we stop the current system.

Here is the rollout plan:

  1. file bug, patch production-1.8-master master.cfg to use equivalent nightly scheduling to staging-1.8-master master.cfg
  2. confirm that production builds start showing up on Mozilla 1.8 Tinderbox tree.
  3. switch test-only tinderboxes (bl-bldxp01, bl-bldlnx01) to start watching production boxes
  4. notify QA to switch Talos to the new nightly builders
  5. confirm that test numbers and graph remain stable. TBD - possibly switch back and forth for a few days
  6. Mac runs it's own performance tests, TBD - if numbers remain stable set up test-only Mac tinderbox on existing box OR merge graphs when switching over