ReleaseEngineering/Buildduty/SVMeetings/Oct10-Oct14

Meeting Details

• Room Link: https://v.mozilla.com/flex.html?roomdirect.html&key=RSAkZHawz3ED

• Meetings every Wednesday
• Meeting notes archive:
  • https://wiki.mozilla.org/ReleaseEngineering/Buildduty/SVMeetings

Upcoming vacation/PTO:

• Alin: Oct 21
• Andrei:
• Florin: November 1-10
• Coop:
• Ihsiao:
• Ashiue: Oct 24
• Kmoir:

Holidays:

• Canada:
  • Thanksgiving October 10
• Romania:
  • November 30
  • December 1 (National Day)
• Taiwan:
  • Retrocession Day, 10/25

Possible bugs

• Write a tool to update bugzilla bugs for outstanding machine loans and ask if the loaner is still needed on say, a weekly basis: https://bugzilla.mozilla.org/show_bug.cgi?id=1171165
• Long Inter-Job Timing on t-yosemite-r7 https://bugzilla.mozilla.org/show_bug.cgi?id=1258480
• Create new script to setup a buildbot master for development: https://bugzil.la/1275135

From coop:

• Incident response (from hg.m.o post-mortem)
  • https://wiki.mozilla.org/ReleaseEngineering/Buildduty/Dealing_With_Outages
• Server alerts: finishing up work on fwunit and ssl cert changes
• https://bugzilla.mozilla.org/show_bug.cgi?id=1307757
• Prep for Hawaii

From kmoir:

• Revert changes to https://bugzilla.mozilla.org/show_bug.cgi?id=1302578 g2.2xlarge price spike and add more capacity to the pool in other regions https://bugzilla.mozilla.org/show_bug.cgi?id=1302549. (Done and I think ondemand instances were also added by rail/catlee)
• How are preparations for the mercurial upgrades going for build/test machines? (to support bug 1298976)
• Other possible bugs:
  • Schedule win10vm test jobs on try https://bugzilla.mozilla.org/show_bug.cgi?id=1304065
  • watch_pending should send an alert when we're outbid for spot instances https://bugzilla.mozilla.org/show_bug.cgi?id=1302567
  • Create alerts for t-w732 and g-w732 pending https://bugzilla.mozilla.org/show_bug.cgi?id=1304158

Taipei: 2016/10/10 - 2016/10/14

• Check failures on trees and mark failures/file new bugs
• Backout changesets if any failures or bustage: bug 1288997, bug 1277803, bug 1297535
• Will have meetings with Tomcat every week start from Oct. 25
• Triage perf alerts
• Perfherder bug : not all data showing up on the perf graph (bug 1309294)

SV: 2016/10/10 - 2016/10/14

• Bug 1307757 - Deploy python 2.7.12 to releng servers
  • Managed to create the rpm for x86_64, deployed it on releng-puppet2
  • Created a loaner similar to buildduty-tools machines, wrote the required puppet manifests in order to create a virtual environment with some predefined python packages installed ⇒ worked OK
  • Updated the manifests to install python 2.7.12, will need to debug the puppet errors received
• Bug 1298437 - access for Wander and Joel to the 20 yosemite-r7 loaners
  • Wander should already have access via ssh
  • A good way to ensure that access for Joel? (ni on the bug)
    • Check that his ssh pubkey is in authorized_keys for cltbld

• Bug 1304065 - Schedule win10vm test jobs on try
  • https://bugzilla.mozilla.org/show_bug.cgi?id=1304065
  • are we making changes to other branches or just on try

• Bug 1093196 - reschedule HSTS and HPKP automatic updates to run daily, and be visible on treeherder
  • https://bugzilla.mozilla.org/show_bug.cgi?id=1093196
  • Need to make these jobs visible in Treeherder
  • According to Phil the periodic update job doesn’t start out with a revision