Releases/Firefox 33.1/BuildNotes

From MozillaWiki
Jump to: navigation, search

bug 1091282

Firefox Build 1

Expecting a build2 for a further fix



  • win32 update verify 6/10 failed to purge enough space and show some output, so buildbot killed it. PYTHONUNBOFFERED=1 would help
  • missed disabling whatsnew pages for all versions in patcher config, as those are being handled in-app by bug 1077643
    • fix config with a patch for build2
    • if we don't do a build2 we should munge the snippets, doing a
      sed -i -e '/^openURL/d' -e 's,^actions=showURL,actions=silent,'
      on all versions.

Firefox build2



  • win32 update verify 3/6 - 20 minutes with no output purging space, filed bug 1091982. Rerunning.
  • incorrect snippet edit applied in patch from build 1 bug 1093187 (hwine)
    • fix for snippets on beta test:
   # on ffxbld@aus3:staging/
   rsync -a ../pushed/Firefox-33.1-build2-test/ Firefox-33.1-build2-test-silent/
   find Firefox-33.1-build2-test-silent -type f -exec sed -i -e '/^actions=silent/d' {} \;
   diff -ru ../pushed/Firefox-33.1-build2-test/ Firefox-33.1-build2-test-silent/ | less
   ~/bin/pushsnip Firefox-33.1-build2-test-silent
    • fix for production snippets (not yet pushed):
   # on ffxbld@aus3:staging/
   find Firefox-33.1-build2 -type f -exec sed -i -e '/^actions=silent/d' {} \;

Firefox Build 3


Before this starts, we're going to need to decide if we're using the shortened releaseConfig['enUSPlatforms'] or not. So far it's looking very promising. If we are, then we should do these steps as setup:

  • [landed] PRIOR TO START IN SHIP IT - modify buildbot-configs/mozilla/ to have
  releaseConfig['enUSPlatforms'] = ('win32',) 
    • patch tools/release/patcher-configs/mozRelease-branch-patcher2.cfg again to get the whatsnew set up right, patch that needs landing landed
    • copy over the non-windows files from build2:
cd /pub/
mkdir -p 33.1-candidates/build3/update
cd 33.1-candidates/build3/

rsync -av --exclude * ../build2/jsshell* .
rsync -av --exclude win32      ../build2/mar-tools .
rsync -av --exclude win32*     ../build2/*_info.txt .
rsync -av --exclude win32      ../build2/partner-repacks .

for p in linux-i686 linux-x86_64 mac; do
  rsync -av ../build2/$p .
  rsync -av ../build2/update/$p update/



  • release runner has failed: a bunch of http 500 errors because of bug 1094922. Relman is waiting for hg return to a stable status before retry the submission
  • win32 repack 1/10 - timeout cloning m-b, then hit b-2008-ix-0103 with low space and there were a lot of retries


The updates builder failed when generating snippets, when it tried to look up the checksum for a 33.0.3 -> 33.1 build3 partial on not-windows. The testing done in staging (for limited enUSPlatofrms) didn't allow for a 33.0.3 release coming along, because it hadn't been mentioned. The best user experience will be to have partials from 33.0.3 for all platforms, so we need to generate them for mac & linux. Working to follow ...

On b-linux64-hp-0021:

## env setup
mkdir ~/33.1-build3-partials; cd $_
export LC_ALL=C
chmod +x m*
export PATH=`pwd`:$PATH
curl -sL >

## get complete mars (a few mins)
mkdir -p releases/33.0.3/
rsync -av -e 'ssh -i /home/cltbld/.ssh/ffxbld_rsa' --exclude win32 --exclude '*partial.mar*' \ releases/33.0.3/

mkdir -p candidates/33.1-candidates/build3
rsync -av -e 'ssh -i /home/cltbld/.ssh/ffxbld_rsa' --exclude win32 --exclude '*partial.mar*' \ candidates/33.1-candidates/build3/

shipped_locales=`curl -s  | cut -d' ' -f1 | grep -v ja`

## create manifest for fast partial generator
rm manifest
for p in linux-i686 linux-x86_64; do
  for l in $shipped_locales ja; do
    echo "releases/33.0.3/update/$p/$l/firefox-33.0.3.complete.mar,candidates/33.1-candidates/build3/update/$p/$l/firefox-33.1.complete.mar,candidates/33.1-candidates/build3/update/$p/$l/firefox-33.0.3-33.1.partial.mar,libsoftokn3.chk|libnssdbm3.chk|libfreebl3.chk" >> manifest

for l in $shipped_locales ja-JP-mac; do
  echo "releases/33.0.3/update/$p/$l/firefox-33.0.3.complete.mar,candidates/33.1-candidates/build3/update/$p/$l/firefox-33.1.complete.mar,candidates/33.1-candidates/build3/update/$p/$l/firefox-33.0.3-33.1.partial.mar,Contents/MacOS/libsoftokn3.chk|Contents/MacOS/libnssdbm3.chk|Contents/MacOS/libfreebl3.chk|Contents/MacOS/firefox" >> manifest

## generate partials, could have been parallelised with separate manifests (72 min)
time python -u -f manifest 2>&1 | tee generate-partials.log

## setup signing
# get signing token via master
hg clone
export PYTHONPATH=tools/lib/python
export SIGN_CMD="python tools/release/signing/ -t `pwd`/token -n `pwd`/nonce -c tools/release/signing/host.cert -H -H -H"

## sign new partials, the mar file itself and create a detached sig  (13 mins)
find candidates/33.1-candidates/build3/update -name '*partial.mar' -print -exec $SIGN_CMD -f mar -f gpg {} \; 2>&1 | tee sign-partials.log

## update checksums files
for f in `find candidates/33.1-candidates/build3/update -name '*partial.mar'| sort`; do
  csum=`dirname ${f/\/update/}`/firefox-33.1.checksums
  mkdir -pv `dirname $csum`
  wget -q -O $csum$csum
  echo `sha512sum $f | cut -d ' ' -f1` sha512 `stat -c %s $f` `echo $f | sed -e 's/.*update/update/'` >> $csum
  echo `md5sum $f | cut -d ' ' -f1` md5 `stat -c %s $f` `echo $f | sed -e 's/.*update/update/'` >> $csum
  echo `sha1sum $f | cut -d ' ' -f1` sha1 `stat -c %s $f` `echo $f | sed -e 's/.*update/update/'` >> $csum

## resign checksums
find candidates/33.1-candidates/build3/ -name '*checksums' -print -exec $SIGN_CMD -f gpg {} \;

## fix permissions
find candidates/ -type f -exec chmod 644 {} \;
find candidates/ -type d -exec chmod 755 {} \;

## upload partials, checksums+sig
rsync -nav -e 'ssh -i /home/cltbld/.ssh/ffxbld_rsa' candidates/33.1-candidates/build3/ \


  • backed out the automated tools landing so rebumping works, 6e8398759170
  • forced updates builder
  • forced check_permissions builder (green)
  • av scan on new partials (green)
cd /pub/
nice ionice -c2 -n7 -j2 clamdscan -m --no-summary -- \
  {linux*,mac}/*/firefox-33.0.3-33.1.partial.mar 2>&1 | tee /tmp/av.log

Other Issues

  • Bouncer had product submitted by build1 which doesn't match build3:
  • Deleted all products listed above and reran the bouncer builder
  • To fix the state of the uptake_check scheduler ran the following to remove release-mozilla-release-firefox_updates from the list of remainingBuilders:
UPDATE schedulers SET state='{"remainingBuilders": ["release-mozilla-release-firefox_push_to_mirrors"], \
  "upstreamBuilders": ["release-mozilla-release-firefox_push_to_mirrors", "release-mozilla-release-firefox_updates"], \
  "lastReset": 1415366189.750769, "lastCheck": 1415366185}' \
  WHERE name='release-mozilla-release-firefox_uptake_check';

Update Balrog

Because we only built win32 with build3, Balrog's release blob wasn't serving updates for Linux or Mac on the release channel. At the time, we weren't using Balrog at all, but the following week bhearsum wanted to start moving some traffic over, so he did the following to fix the 33.1 build3 blobs:

python -j ~/tmp/build3.json -b Firefox-33.1-build3 -a -c -u

Fennec Build 1



  • email '[release] Fennec 33.1 build1: Updates available on None', bug 1019724 again

Fennec build 2

Potential fix for 1089183



  • email has None instead as channel name, from bug 1019724
Subject: [release] Fennec 33.1 build2: Updates available on None
Content: Updates to Fennec 33.1 build2 are now ready for testing on the None channel.