Release Engineering Buildduty Meeting

Date: 2013-12-10
Time: 10:00am EST
Room: ReleaseEngineering Vidyo room
Meeting notes: https://wiki.mozilla.org/ReleaseEngineering/Buildduty/Meetings:2013-12-10

Status of buildduty period

bhearsum to file bug on slaveapi finer grained access + giving sheriffs full access for now
Callek to followup with :ashish/IT about 927941/942969
rail to look into redis restart script
coop to investigate move to existing webops redis cluster

(coop) meeting timing
- move to 1pm EST to accommodate jlund
(jh) we need to be more consistent with slave health management
- eg. w64-ix-slave30 up since 11/6/2013, accessible via SSH, just needs a reboot. Hasn't done a build in 28 days. Try build queue has two builds waiting for 38 minutes needlessly.
- (coop) if you (re)open a slave bug, please take the time to open the corresponding recovery/diagnostics bug
(bhearsum) bug 939948 - add nagios bug queue checks for buildduty and loan requests components
(armenzg) creating an EC2 instance did not fail clearly
- I was using an old deploypass password
- the only way to know was to login as root to the machine and check the puppetize.log
- any ideas on how to solve this?
(coop) goals for 2014
- what subset in Q1?
  - slave reboots in slave health
  - integrate slave loan process into slaveapi
    - automate secret cleanup in puppet (not necessarily tied to slaveapi at outset)
  - AWS instance creation via slaveapi
  - VS 2012 - https://bugzilla.mozilla.org/show_bug.cgi?id=946859
  - finish clobberer
- rest of 2014?
  - slave loan automation
  - builder utilization
    - talk to datazilla team
  - different instance types in AWS (ongoing)
  - Windows builds in AWS
  - new releng project branch in TBPL + dedicated master + hg
interesting point of order: hg.m.o failed connections relating possibly to zeus filed today: Bug https://bugzil.la/948426