Releases/Firefox 38.0.5/BuildNotes

From MozillaWiki
Jump to: navigation, search


Build 1

Failed before it started because the production-0.8 branch of buildbotcustom was busted for non-release reasons. bhearsum backed out the bustage and started build2.

Build 2

Issues starting because the release-runner reconfig conflicted with the hourly reconfig, so that the two schedulers on bm81 as well as bm71 and bm74 were not reconfiged by release runner. eg

[FAIL] lockfile (reconfig.lock) found in

When the sendchange was sent we got this in the build_scheduler:

 2015-05-21 14:01:41-0700 [Broker,46296,] FIREFOX_38_0_1_RELEASE revision does not match script_repo_revision FIREFOX_38_0_5_RELEASE

bug 1167467 for these issues. Recovered by reconfiging the 4 affected masters, and re-doing the sendchange.

Cancelled in tagging, because the branding and update config was not set up properly. We need to backout 27bacb9dff64 for build3. Done in 79f9cd31b4b1.

Build 3

Deal with missing beta channel updates

KaiRo noticed that 38.0.5 didn't become available on beta-cdntest. This is because we only enable the beta channel for "final" releases (, which means "two digit version number".

Our assumption was broken by 38.0.5, so this never happened. I _think_ the best way to fix this up is to tweak the release config by hand, retag, reconfig, and then kick the beta updates builder. I landed to enable all the necessary builders and reconfiged the build and scheduler masters for it. After that was done, I:

  • Reran bouncer submitter to add necessary bouncer entries with force bulid
  • Ran beta updates builder with force build

After the updates builder ran the downstreams (update verify, ready for cdntest, etc.) seemed to fire correctly.

Build 4

Deal with missing beta channel updates and failures

Similar to build3.

  • relanded attachment 8609368 [edit] on production with retag, to enable all the necessary builders
  • reconfiged the build and scheduler masters
  • reran bouncer submitter to add necessary bouncer entries with force build

This was done before the builds were all done, so the beta updates builder ran automatically. They failed with

release.updates.patcher.PatcherConfigError: Found multiple past-updates with duplicate to/from versions: ['38.0.5b3', '39.0b1', ['betatest', 'releasetest', 'beta']]

Fixed with 73652067bcdc and rebuilt.

The beta final_verification failed, because the uptake check returned quickly and Balrog was still serving content from before the beta_updates job changes. Similar for mac update verify 3 and 4 for beta. Rerunning all three of those returned green results.

What's New Pages

  • there is special handling on what's new needed for both Release (38.0.5) & Developer Edition (40.0). See bug 1143875 & bug 1135820.
    • work for 38.0.5 cdn-test done in blob & balrog (bug 1135820). NB: ja-jp request turns into ja,ja-JP-mac


bug 1157556 requests three funnelcake builds to track effectiveness of firstrun variations. They were built as part of partner repack jobs.

Push to mirrors

# ffxbld@stage
rsync -av /pub/ \
rsync -av /pub/ \
rsync -av /pub/ \
for fc in 37 38 39; do
  for d in '/' '/win32/' '/win32/en-US/'; do
    cp -v /pub/ /pub/${fc}${d}

Set up bouncer links per bug comment, making sure to enable SSL on the product, and limit it to en-US.