Build:OutageReports:20070415-01

Outage Template

On April 15th at 07:30, bm-xserve09 had experienced a service outage for ~ 40 hours.

What was affected:

Trunk mac builds for Thunderbird and XULRunner.

What was the cause of the outage:

In the console (datestamp unknown):

bm-serve09:~ cltbld$ cd /builds/tinderbox/
bm-serve09:~ cltbld$ ./tinderbox multi start
multi build already running with PID 342

At the end of the Thunderbird log:

cvs -q <expanded SeaMonkeyAll>
cvs [checkout aborted]: received termination signal
Kill by signal 15.
make[1]: *** [real_checkout] Terminated
make: *** [checkout] Terminated

What will be done to prevent this in the future:

Multi tinderbox was stopped (PID 342 was not found, ps confirmed nothing running prior to this), then restarted.