ReleaseEngineering/Buildduty/SVMeetings/Nov9-Nov13

From MozillaWiki
Jump to: navigation, search

Upcoming vacation/PTO:

  • alin - dec 24, dec 28-31
  • nov 17 - Vlad and Alin will be in Bucharest for VISA

Meetings every Tuesday and Thursday

https://wiki.mozilla.org/ReleaseEngineering/Buildduty/SVMeetings/Nov2-Nov6


09.11.2015 - 10-11.2015

New bugs to work on disable media-tests on linux asan, debug (on development and release-stabilization branches) https://bugzilla.mozilla.org/show_bug.cgi?id=1223072 Verify decommissioned pandas have stopped taking jobs - https://bugzilla.mozilla.org/show_bug.cgi?id=1223256

[alin] https://bugzilla.mozilla.org/show_bug.cgi?id=1223072 - disable media-tests on linux asan, debug created a test master, working on the patch 

     2. re-imaged several bld-lion-r5 slaves today, but noticed that for this particular pool of machines we have a high ratio of failed jobs (caused by timeouts, failures in running some scripts, failures in connecting to an upload location, etc)

Q: is there a reason to try to fix this/should we take care of other things and ignore this? 

     3. today we noticed alerts like “aws-manager2.srv.releng.scl3.mozilla.com:procs age - golden AMI is CRITICAL: ELAPSED CRITICAL: 5 crit, 0 warn out of 5 processes with args ec2-golden” in #buildduty

in aws console we can see that the AMIs have been created looked over documentation to see if we can find some steps to deal with such issue, but had little luck some suggestions on this would be useful.. https://wiki.mozilla.org/ReleaseEngineering/How_To/Work_with_Golden_AMIs#How_to_force_AMI_generation on aws-manager2 in /var/log/messages we can see a lot of error messages : Nov 10 05:42:44 aws-manager2.srv.releng.scl3.mozilla.com aws_watch_pending.py: <Response><Errors><Error>InsufficientFreeAddressesInSubnet<Message>There are not enough free addresses in subnet 'subnet-56082710' to satisfy the requested number of instances.</Message></Error></Errors><RequestID>aaa0e78a-02fc-48e5-a242-b4f3a9b52110</RequestID></Response> Nov 10 05:58:01 aws-manager2.srv.releng.scl3.mozilla.com aws_watch_pending.py: lockfile: Sorry, giving up on "/builds/aws_manager/aws_watch_pending.py.lock" discussed with grenade and he confirmed that the old processes for golden_ami can be killed. After we killed the processes the alert become to OK state.


11.11.2015 - 12-11.2015 https://bugzilla.mozilla.org/show_bug.cgi?id=1214362 the path of my master : /builds/buildbot/vlad.ciobancai/mochitest-test also androidarm_4_3.py and androidarm.py have been updated, the changes have been added as a patch to the bug (bug1214362_androidarm.py.patch) this is the error that I’m receiving https://pastebin.mozilla.org/8851971 10.10.5? will try to enable early next week regardless of single failure (will be disabled/hidden)

https://bugzilla.mozilla.org/show_bug.cgi?id=1223072 - updated patch https://bugzilla.mozilla.org/show_bug.cgi?id=1223399

Retrieved from "https://wiki.mozilla.org/index.php?title=ReleaseEngineering/Buildduty/SVMeetings/Nov9-Nov13&oldid=1105742"