Firefox

Build 1

Issues

Intermittent/silly things

  • linux64 repack 2/10 hang downloading mar file for the first locale
Downloading http://stage.mozilla.org/pub/mozilla.org/firefox/candidates/40.0-candidates/build1/linux-x86_64/en-US/firefox-40.0.tar.bz2 to firefox.tar.bz2
Downloading http://stage.mozilla.org/pub/mozilla.org/firefox/candidates/38.0.5-candidates/build4/update/linux-x86_64/bn-BD/firefox-38.0.5.complete.mar to firefox-38.0.5.bn-BD.complete.mar
Downloading http://stage.mozilla.org/pub/mozilla.org/firefox/candidates/35.0.1-candidates/build1/update/linux-x86_64/bn-BD/firefox-35.0.1.complete.mar to firefox-35.0.1.bn-BD.complete.mar
Downloading http://stage.mozilla.org/pub/mozilla.org/firefox/candidates/37.0.2-candidates/build1/update/linux-x86_64/bn-BD/firefox-37.0.2.complete.mar to firefox-37.0.2.bn-BD.complete.mar

command timed out: 2400 seconds without output ... program finished with exit code 247
  • win32 repack 10/10 ran out of space with two locales to do. bug 1190644 to increase the requirement. It auto-retried but that was missed, and the standalone_repack job was used
  • win32 repack 6/10 also failed, got 9.0G after asking for 7G. Auto-retried.

Release channel updates builder failure

firefox_release_updates failed with duplicate past-update lines, fixed with a config tweak

This probably also caused the bustage in beta_final_verification and beta_update_verify (some but not all, strangely). Should we block on both updates builders completing ?

beta update verify failures in chunks 3 and 4 (all platforms

Both of these are showing the "complete" block missing from update response for 39.0 release (they failures look slightly different, because chunk 3 is doing a full check, and chunk 4 is doing a quick check).

When Ben investigated this, Balrog was returning a response that included a complete and a partial, but the partial was pointing at Bouncer instead of directly at FTP. This is because the 39.0 -> 40.0 partial is shared with the release channel, so they Balrog submitter didn't submit it for Beta as well. We should fix that, and also bug 1170919.

Mac release update verify failures

5 of the mac release update_verify failed with either an empty response, or completes that specified download.m.o instead of stage.m.o (despite being on the release-localtest channel). Not reproducible now, so must have been caching issues. We may need a delay at the end of the updates builder again. Also should look to see if caching is doing what we expect.