ReleaseEngineering/Buildduty/SVMeetings/Nov7-Nov11

Meeting Details

• Room Link: https://v.mozilla.com/flex.html?roomdirect.html&key=RSAkZHawz3ED

• Meetings every Wednesday
• Meeting notes archive:
  • https://wiki.mozilla.org/ReleaseEngineering/Buildduty/SVMeetings

• Upcoming vacation/PTO:
  • Alin: November 28-29
  • Andrei:
  • Florin:
  • Coop:
  • Kmoir:
• Holidays:
  • Canada:
  • Romania:
    • November 30
    • December 1 (National Day)

Possible bugs

• Write a tool to update bugzilla bugs for outstanding machine loans and ask if the loaner is still needed on say, a weekly basis: https://bugzilla.mozilla.org/show_bug.cgi?id=1171165
• Long Inter-Job Timing on t-yosemite-r7 https://bugzilla.mozilla.org/show_bug.cgi?id=1258480
• Create new script to setup a buildbot master for development: https://bugzil.la/1275135

From coop:

• Triage
• Prep for Hawaii

From kmoir:

• Revert changes to https://bugzilla.mozilla.org/show_bug.cgi?id=1302578 g2.2xlarge price spike and add more capacity to the pool in other regions https://bugzilla.mozilla.org/show_bug.cgi?id=1302549. (Done and I think ondemand instances were also added by rail/catlee)
• How are preparations for the mercurial upgrades going for build/test machines? (to support bug 1298976)
  • [alin]: still need to be done for Windows 10 testers
• Other possible bugs:
  • Schedule win10vm test jobs on try https://bugzilla.mozilla.org/show_bug.cgi?id=1304065
  • watch_pending should send an alert when we're outbid for spot instances https://bugzilla.mozilla.org/show_bug.cgi?id=1302567
  • Create alerts for t-w732 and g-w732 pending https://bugzilla.mozilla.org/show_bug.cgi?id=1304158
• Is there further automation that you can implement to make regular tasks easier i.e. the machine loan email as a template

SV: 2016/11/07 - 2016/11/11

• Bug 1315766 - Create alerts for ratio of pool size to pending jobs
  • For the moment the pending jobs are been alerted based on the pools which have a unique slavepool id
  • Alerting only hardware pools will not give as an overview of what is happens,and we also have some pools with both hardware and aws machines
  • We must have alerts also for the pools with aws
• Bug 1285483 - t-yosemite-r7 machines have high coreaudiod and launchd CPU usage (no updates)
• Bug 1302567 - watch_pending should send an alert when we're outbid for spot instances
  • Papertrail logs are not enough for determining whether we have a real problem
  • Will need to find a better solution for this alert
• Investigating bug 1315266 - buildbot and puppet masters shouldn't listen for NTP connections