From MozillaWiki
Jump to: navigation, search

Upcoming vacation/PTO:

  • alin - dec 24, dec 28-31

Meetings every Tuesday and Thursday

30.10.2015 - 03-11.2015

New bugs

There are two blocker bugs alerting in buildduty, please see if they need help or can be assigned to someone

Could you go through the buildduty report and see if any of these bugs could be closed

Add alert for buildbot lag You could needinfo catlee to see where the current graph is that measure this. This is similar to the bug that checks pending counts. We want alerts when certain buildbot masters hit certain limits.

Enable DOM push Mochitests on Android Fennec adding new tests on mobile, let me know if you have questions [Vlad] I will start looking over it

[alin] created and uploaded the python script to check the age of the backlog waiting for :catlee’s review

     2.    cleaned the buildduty report a little bit:

marked as resolved several problem tracking bugs:

Windows slaves: we have been monitoring them lately, things look good

one talos slave: got re-imaged as a win732 slave did a needinfo on bug 1185116 to see if the slave is still needed + some others from yesterday

[vlad] Attached the date and time settings for the slave Kim if you can provide us more details

03.11.2015 - 05-11.2015


noticed that when receiving alerts for a high number of pending jobs, we still use the results returned by the old version of e.g: builds is WARNING: WARNING Pending Builds: 2083 on cruncher, I checked the logs from /var/log/nrpe.log → they contain entries like the one above looking at /usr/lib64/nagios/plugins/custom/check_pending_builds will show the old version of the script Q: is there a certain period of time needed to do the sync with the version from the repo ( Or do we need to change this?

2. - create nagios checks for buildbot backlog age and master lag yesterday I updated the script that checks the age of the backlog, waiting for Chris’s review looked over the script to create an alert using graphite data, played with graphite to understand it better, tested the script locally using several values for the thresholds we would need to create a nagios alert to check the lag on the masters too. Is there a repo that contains the config files we need to modify? talk to arr for more info on how to create nagios alerts

3. Loaning a dev-linux64-ec2 vm → gets stuck when running papertrail does not contain any data on the issue connected to aws manager, switched to buildduty user and then managed to ssh to the instance: uptime puppetize.log contains the following:

Contacting puppet server puppet

5 Nov 09:26:06 ntpdate[20126]: rate limit response from server.
5 Nov 09:26:07 ntpdate[20126]: adjust time server offset -0.135954 sec

Got incorrect certificates (!?) Looks like it is failing here