ReleaseEngineering/Buildduty/SVMeetings/September26-September30

Meeting Details

• Room Link: https://v.mozilla.com/flex.html?roomdirect.html&key=RSAkZHawz3ED

• Meetings every Wednesday
• Meeting notes archive:
  • https://wiki.mozilla.org/ReleaseEngineering/Buildduty/SVMeetings

Upcoming vacation/PTO:

• Alin: Oct 21
• Andrei: September 29
• Florin: November 1-10
• Coop:
• Ihsiao:
• Ashiue:
• Kmoir:

Holidays:

• Canada:
  • Thanksgiving October 10
• Romania:
  • November 30
  • December 1 (National Day)
• Taiwan:
  • National Day, 10/10

Possible bugs

• Write a tool to update bugzilla bugs for outstanding machine loans and ask if the loaner is still needed on say, a weekly basis: https://bugzilla.mozilla.org/show_bug.cgi?id=1171165
• Long Inter-Job Timing on t-yosemite-r7 https://bugzilla.mozilla.org/show_bug.cgi?id=1258480
• Create new script to setup a buildbot master for development: https://bugzil.la/1275135

From coop:

• Deploy Mercurial 3.9 to automation - bug 1298976
• AMI canaries? https://bugzilla.mozilla.org/show_bug.cgi?id=1146341

From kmoir:

• Revert changes to https://bugzilla.mozilla.org/show_bug.cgi?id=1302578 g2.2xlarge price spike and add more capacity to the pool in other regions https://bugzilla.mozilla.org/show_bug.cgi?id=1302549. (Done and I think ondemand instances were also added by rail/catlee)
• How are preparations for the mercurial upgrades going for build/test machines? (to support bug 1298976)
• Other possible bugs:
  • Schedule win10vm test jobs on try https://bugzilla.mozilla.org/show_bug.cgi?id=1304065
  • watch_pending should send an alert when we're outbid for spot instances https://bugzilla.mozilla.org/show_bug.cgi?id=1302567
  • Create alerts for t-w732 and g-w732 pending https://bugzilla.mozilla.org/show_bug.cgi?id=1304158

Taipei: 2016/09/26 - 2016/09/30

• Check failures on trees and mark failures/file new bugs
• Backout changesets if any failures or bustage
• Merge training from Tomcat
• Triage perf alerts
• Investigate perf regression issues
• Loan request: bug 1304472

SV: 2016/09/26 - 2016/09/30

• https://bugzilla.mozilla.org/show_bug.cgi?id=1299421
  • Switch to running restart_masters.sh on the buildduty-tools machine
  • Created the enviornment and the script
  • Tested on buildbot-master69
• https://bugzilla.mozilla.org/show_bug.cgi?id=1304065
  • Schedule win10vm test jobs on try
  • Created patch for watch_pending.cfg to spin up the new w10 instances when there are jobs to run
  • Rwood also created a patch and asked for a review from Catlee
• https://bugzilla.mozilla.org/show_bug.cgi?id=1304158
  • Create alerts for t-w732 and g-w732 pending
• Did a bit of cleaning on loaners
  • Pinged people to see if they still need the loaned machines
  • Updated inventory and vpn_releng_loan entries
  • Re-imaged several machines, updated bugs
• Working on:
  • Bug 1305564 - Alert for stale AMI or failing AMI generation
  • Bug 1304861 - Windows loaners have an old version of Mercurial
  • Bug 1303006 - ensure win7-ix jobs are not run on try for unittests which are running on AWS vm