Releases/Firefox 6.0/BuildNotes

From MozillaWiki
Jump to: navigation, search

Contents

Notes About Releasing

Please update the Notes Template and the Release:Primer for future releases (bug fixes, changes to automation) as needed

Bugs hit

  • bug 678467 - Mobile partner repacks should be triggered automatically
  • bug 678804 - Audit ssh keys on slaves (missing xrbld key)

Build Engineers

rail - Tracking bug: bug 676083

Sync mozilla-beta to mozilla-release

Done on mv-moz2-linux-ix-slave11

  • synced mozilla-beta to mozilla-release (including l10n repos) according to the documentation

Signed-off Revision(s)

L10N changesets

  • Firefox: no new signoffs, copied from beta
  • Fennec: no new signoffs, copied from beta

Tags

Build # Branch, Tags Changeset
1 MOBILE60_2011081116_RELBRANCH, FENNEC_VERSION_RELEASE, FENNEC_VERSION_BUILD1 70592ed81659
GECKO60_2011081114_RELBRANCH, FIREFOX_VERSION_BUILD1 FIREFOX_VERSION_RELEASE 218ed8178b1e

Build data

Firefox

Build # Type Build ID Build machine Time to build
1 Linux 20110811165603 mv-moz2-linux-ix-slave08 2 hrs, 12 mins, 55 secs
Linux64 moz2-linux64-slave09 3 hrs, 16 mins, 28 secs
Mac moz2-darwin10-slave52 4 hrs, 3 mins, 49 secs
Windows mw32-ix-slave09 2 hrs, 32 mins, 46 secs

Fennec

Build # Type Build ID Build machine Time to build
1 android 20110811165229 mv-moz2-linux-ix-slave16 1 hrs, 31 mins, 51 secs
maemo5-gtk 20110811165229 mv-moz2-linux-ix-slave06 1 hrs, 27 mins, 12 secs
linux 20110811172233 mv-moz2-linux-ix-slave19 1 hrs, 50 secs
macosx 20110811165229 bm-xserve16 1 hrs, 45 mins, 23 secs
win32 20110811165229 mw32-ix-slave10 1 hrs, 18 mins, 25 secs

Notes

used buildbot-master07

Build 1

echo "10" > master/reserved_slaves_bm07-build1
  • There is no new l10n changesets available, so I copied beta l10n changesets files to the corresponding release files (submitted as a part of configs patch)
  • Landed configs
  • Tagged tools, buildbotcustom, buildbot-configs, mozharness with FIREFOX_6_0_BUILD1 FIREFOX_6_0_RELEASE FENNEC_6_0_BUILD1 FENNEC_6_0_RELEASE tags
  • updated master
# in /builds/buildbot/build1
make update
  • release sanity:
source ../bin/activate
PYTHONPATH=. ../bin/python ../tools/buildbot-helpers/release_sanity.py -u rail \
 -V 6.0 --branch mozilla-release --build-number 1 -c release-firefox-mozilla-release.py \
 --dryrun --products firefox,fennec localhost:9001
  • failed:
2011-08-11 16:33:43,053 : ERROR : cannot find l10n dashboard at https://l10n-stage-sj.mozilla.org/shipping/l10n-changesets?ms=fx6.0
2011-08-11 16:33:43,053 : ERROR : Error verifying l10n dashboard changesets
  • ignorable, since there is no dashboard entry for 6.0
  • started automation:
source ../bin/activate
PYTHONPATH=. ../bin/python ../tools/buildbot-helpers/release_sanity.py -u rail \
 -V 6.0 --branch mozilla-release --build-number 1 -c release-firefox-mozilla-release.py \
 -b --products firefox,fennec localhost:9001
  • Realized that al win32 builds were pending. There were only 3 slaves attached to bm07 and all were busy.
  • moved mw32-ix-slave10 and mw32-ix-slave09 to bm07. Released when updates started.

E-mail Metrics

E-mail Socorro-dev

Sent an email to socorro-dev with the body "Build IDs will go here: https://wiki.mozilla.org/Releases/Firefox_6.0/BuildNotes#Build_data"

Tag

No problems

Bouncer Submitter

No problems

Added Firefox-6.0-Partial-5.0 and Firefox-6.0-Partial-6.0b5 Bouncer products and the corresponding locations to handle manual updates.

Source

No problems.

Start autosign

Saved the following as ~/tmp/sign_6.0.sh 

PRODUCT=firefox
VERSION=6.0
BUILD=1
TAG=FIREFOX_6_0_RELEASE
REPO='releases/mozilla-release'
EMAIL=release@mozilla.com
KEYDIR=d:/2010-keys
RELCONFIG=mozilla/release-firefox-mozilla-release.py

cd ${HOME}/hg-tools/
hg pull && hg up -r ${TAG}

mkdir -p ~/signing-work/${PRODUCT}-${VERSION}/lib/python
cd ~/signing-work/${PRODUCT}-${VERSION}
cp ~/hg-tools/release/signing/* .
rsync -av ~/hg-tools/lib/python/ ./lib/python/
make autosign PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} RELCONFIG=${RELCONFIG}

First attempt failed. Blat was trying to connect to mail.mozilla.com, but it was blocked. I changed its config by editing registry "HKEYCU/Software/Public domain/Blat/SMTP server". Set to mail.build.mozilla.org.

Build

Firefox

No problems

Fennec

Android

No problems

Maemo5-GTK

No problems

Desktop Builds

No problems

Desktop repacks

No problems

Unittests / Talos

XULRunner

Source

Failed because mv-moz2-linux-ix-slave17.build.mozilla.org was unable to upload files (missing xrbld key). Copied the key from another slaved, rebuilt.

Filed bug 678804 - Audit ssh keys on slaves

Build

No problems

Signing

PRODUCT=xulrunner
VERSION=6.0
BUILD=1
TAG=FIREFOX_6_0_RELEASE
REPO="releases/mozilla-release"
EMAIL=release@mozilla.com
KEYDIR=d:/2010-keys

cd ~/hg-tools
hg pull
hg up -r ${TAG}

mkdir -p ~/signing-work/${PRODUCT}-${VERSION}
cd ~/signing-work/${PRODUCT}-${VERSION}
cp ~/hg-tools/release/signing/* .

make setup PRODUCT=${PRODUCT} VERSION=${VERSION} \
  BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
rsync -av -e "ssh -i /home/cltsign/.ssh/xrbld_dsa" \
  --exclude=*.txt --exclude=*-symbols.zip \
  xrbld@stage.mozilla.org:/home/ftp/pub/${PRODUCT}/nightly/${VERSION}-candidates/build${BUILD}/ \
 unsigned-build${BUILD}
rsync -av --exclude=unsigned unsigned-build${BUILD}/ \
 signed-build${BUILD}/
rsync -av unsigned-build${BUILD}/unsigned/ \
 signed-build${BUILD}/

make checksum-files create-sigs stage verify-sigs \
  PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} \
  REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1

rmdir signed-build${BUILD}/contrib{,-localized}

rsync -av -e "ssh -i /home/cltsign/.ssh/xrbld_dsa" \
  signed-build${BUILD}/ \
  xrbld@stage.mozilla.org:/home/ftp/pub/${PRODUCT}/nightly/${VERSION}-candidates/build${BUILD}/

Repacks

Firefox

No problems

Fennec

No problems

Partner Repack

Firefox

No problems

Fennec

Had to trigger builds manually. Filed bug 678467

Firefox Signing

No problems.

Android Signing

followed instructions

Android Signature Verification

Green. Need to check why r-d email was sent twice.

Maemo5-GTK repos

First attempt failed. Landed syntax fix. Rebuilt.

L10N verify

No problems.

Updates

mozilla-releases updates

Generation

No problems

mozilla-release Update verify

no problems

5.0 -> 6.0 updates

Since most of win32 and linux users are on 5.0 (5.0.1 had mac only update path), I generated 5.0-6.0 partial updates

Generation

Used mv-moz2-linux-ix-slave17 which generated updates 

cd /builds/slave/rel-m-rel-updates/build

cp patcher-configs/mozRelease-branch-patcher2.cfg patcher-configs/mozRelease-from-5.0-branch-patcher2.cfg
# patcher-configs/mozRelease-from-5.0-branch-patcher2.cfg
# removed all past-update lines
# removed all versions except 5.0 and 6.0
# s/5.0.1/5.0/g
# attached as https://bugzilla.mozilla.org/attachment.cgi?id=552629

perl patcher2.pl --download --app=firefox --brand=Firefox --config=patcher-configs/mozRelease-from-5.0-branch-patcher2.cfg
time perl patcher2.pl --create-patches --partial-patchlist-file=patchlist.cfg --app=firefox --brand=Firefox --config=patcher-configs/mozRelease-from-5.0-branch-patcher2.cfg

find temp/firefox/5.0-6.0/ftp/firefox/nightly/6.0-candidates/build1 -type f -exec chmod 644 '{}' ';'
find temp/firefox/5.0-6.0/ftp/firefox/nightly/6.0-candidates/build1 -type d -exec chmod 755 '{}' ';'

rsync -avn -e 'ssh -oIdentityFile=~/.ssh/ffxbld_dsa' '--exclude=*complete.mar' temp/firefox/5.0-6.0/ftp/firefox/nightly/6.0-candidates/build1/update ffxbld@stage-old.mozilla.org:/pub/mozilla.org/firefox/nightly/6.0-candidates/build1/

rsync -av -e 'ssh -oIdentityFile=~/.ssh/cltbld_dsa' temp/firefox/5.0-6.0/aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/Firefox-6.0-build1-5.0-build1-test

rsync -av -e 'ssh -oIdentityFile=~/.ssh/cltbld_dsa' temp/firefox/5.0-6.0/aus2.beta/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/Firefox-6.0-build1-5.0-build1-beta

rsync -av -e 'ssh -oIdentityFile=~/.ssh/cltbld_dsa' temp/firefox/5.0-6.0/aus2/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/Firefox-6.0-build1-5.0-build1

Push to the betatest channel 

# cltbld@aus2-staging.mozilla.org
~/bin/backupsnip Firefox-6.0-build1-5.0-build1-test
~/bin/pushsnip Firefox-6.0-build1-5.0-build1-test

Merge with automatically generated snippets: 

# cltbld@aus2-staging.mozilla.org
cd /opt/aus2/snippets/staging/
# release channel
rsync -avP Firefox-6.0-build1/ Firefox-6.0-build1-with-5.0-partials/
rsync -avP Firefox-6.0-build1-5.0-build1/ Firefox-6.0-build1-with-5.0-partials/
Update verify

Retriggered chunks 1 & 6 of update verify for each platform (chunk #6 contains full tests for de, en-US and ru; chunk #1 contains quick tests for all other locales).

4.x, 5.x, 6.x Beta -> 6.0 updates

We also want to push 6.0 release to beta channel

Generation

Used mv-moz2-linux-ix-slave17 which generated updates 

cd /builds/slave/rel-m-rel-updates/build
# Bump mozBeta-branch-patcher2.cfg, no -u needed
PERL5LIB=../tools/lib/perl perl ../tools/release/patcher-config-bump.pl \
  -p firefox -r Firefox -v 6.0 -a 6.0 -o 6.0b5 -b 1 \
  -c patcher-configs/mozBeta-branch-patcher2.cfg -t stage-old.mozilla.org \
  -f ftp.mozilla.org -d download.mozilla.org -l shipped-locales \
  --platform linux --platform linux64 --platform macosx64 --platform win32 \
  --marname firefox --oldmarname firefox

# Sanity check  
cvs -q diff -u patcher-configs/mozBeta-branch-patcher2.cfg

cvs -q commit -m "Configuration bump: mozBeta-branch-patcher2.cfg, from 6.0b5 to 6.0 build 1" patcher-configs/mozBeta-branch-patcher2.cfg

perl patcher2.pl --download --app=firefox --brand=Firefox --config=patcher-configs/mozBeta-branch-patcher2.cfg
perl patcher2.pl --create-patches --partial-patchlist-file=patchlist.cfg --app=firefox --brand=Firefox --config=patcher-configs/mozBeta-branch-patcher2.cfg

find temp/firefox/6.0b5-6.0/ftp/firefox/nightly/6.0-candidates/build1 -type f -exec chmod 644 '{}' ';'
find temp/firefox/6.0b5-6.0/ftp/firefox/nightly/6.0-candidates/build1 -type d -exec chmod 755 '{}' ';'

rsync -av -e 'ssh -oIdentityFile=~/.ssh/ffxbld_dsa' '--exclude=*complete.mar' temp/firefox/6.0b5-6.0/ftp/firefox/nightly/6.0-candidates/build1/update ffxbld@stage-old.mozilla.org:/pub/mozilla.org/firefox/nightly/6.0-candidates/build1/

rsync -av -e 'ssh -oIdentityFile=~/.ssh/cltbld_dsa' temp/firefox/6.0b5-6.0/aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/Firefox-6.0-build1-6.0b5-build1-test
rsync -av -e 'ssh -oIdentityFile=~/.ssh/cltbld_dsa' temp/firefox/6.0b5-6.0/aus2/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/Firefox-6.0-build1-6.0b5-build1-beta

Push to the betatest channel 

# cltbld@aus2-staging.mozilla.org
~/bin/backupsnip Firefox-6.0-build1-6.0b5-build1-test
~/bin/pushsnip Firefox-6.0-build1-6.0b5-build1-test

Merge with automatically generated snippets: 

# cltbld@aus2-staging.mozilla.org
cd /opt/aus2/snippets/staging/

# beta channel
rsync -avP Firefox-6.0-build1-beta/ Firefox-6.0-build1-with-5.0-6.0b5-partials-beta/
rsync -avP Firefox-6.0-build1-5.0-build1-beta/ Firefox-6.0-build1-with-5.0-6.0b5-partials-beta/
rsync -avP Firefox-6.0-build1-6.0b5-build1-beta/ Firefox-6.0-build1-with-5.0-6.0b5-partials-beta/

Major Update

Update verify

Socorro Product Data Updates

N/A since there is no version change.

Check permissions / AV scan

No problems

Reset reserved slaves

echo 0> reserved_slaves_bm07-build1

Push to beta

Note: push Firefox-6.0-build1-with-5.0-6.0b5-partials-beta directory 

~/bin/backupsnip Firefox-6.0-build1-with-5.0-6.0b5-partials-beta
~/bin/pushsnip Firefox-6.0-build1-with-5.0-6.0b5-partials-beta

QA noticed that beta updates from 4.0b*, 5.0b*, and 6.0b* were getting 404'ed. This is because they were pointing at Bouncer, not directly at FTP. Shut off updates to stop the bustage by doing: 

# cltbld@aus2-staging
cd /opt/aus2/incoming/3/Firefox
chmod 700 4.0* 5.0* 6.0*

Then started the push to internal mirrors. See the subsequent section for details

Push to internal mirrors

Just like the Betas, this is only going to internal mirrors (at first), so we add the following line to stage:/pub/mozilla.org/zz/rsyncd-mozilla-releases.exclude: 

- firefox/releases/6.0

After waiting 10 minutes, forced the "release-mozilla-release-push_to_mirrors" with the following properties: 

release_config: mozilla/release-firefox-mozilla-release.py
script_repo_revision: FIREFOX_6_0_RELEASE

After we got uptake on all of the internal mirrors, re-enabled updates: 

# cltbld@aus2-staging
cd /opt/aus2/incoming/3/Firefox
chmod 775 4.0* 5.0* 6.0*
touch .

Push XULRunner to Mirrors

from a slave: 

ssh -l xrbld -i ~/.ssh/xrbld_dsa stage.mozilla.org

# DRYRUN
rsync -n -av --exclude=*.txt --exclude=*unsigned*  --exclude=*-symbols.zip --exclude=*.checksums \
/pub/mozilla.org/xulrunner/nightly/6.0-candidates/build1/ /pub/mozilla.org/xulrunner/releases/6.0/
# FOR REAL
rsync -av --exclude=*.txt --exclude=*unsigned*  --exclude=*-symbols.zip --exclude=*.checksums \
/pub/mozilla.org/xulrunner/nightly/6.0-candidates/build1/ /pub/mozilla.org/xulrunner/releases/6.0/

Update XULRunner wiki page

Updated Gecko_SDK page

Publish Fennec

Create bouncer entry for maemo

Added Firefox-Mobile-6.0 product with location /mobile/releases/maemo/:lang/firefox.install for maemo platform

Push the files

Used the following script (~ffxbld/push_fennec_release.sh on stage): 

export VERSION=6.0
export BUILDNUM=1
export RD=/home/ftp/pub/mozilla.org/mobile/releases
export CD=/home/ftp/pub/mozilla.org/mobile/candidates

mkdir $RD/$VERSION
cd $RD/$VERSION

rsync -av --exclude=unsigned --exclude=*.txt --exclude=*crashreporter* \
 --exclude=*tests* --exclude=*unaligned* --exclude=*old* --exclude=*install* \
 --exclude=*logs* \
 $CD/$VERSION-candidates/build$BUILDNUM/ $RD/$VERSION/

for p in linux macosx win32; do
    locales=`cd $CD/$VERSION-candidates/build$BUILDNUM/$p/; ls -1 --hide=en-US`
    for i in $locales; do
        cp -a $CD/$VERSION-candidates/build$BUILDNUM/$p/$i/install/*.xpi \
          $RD/$VERSION/$p/$i
    done
done

# repos
mkdir $RD/$VERSION/repos
rsync -av --exclude=*.install $CD/$VERSION-candidates/repos/ $RD/$VERSION/repos/

# symlink repos
rm -f maemo
ln -s repos maemo

# link fremantle to chinook for multi deb repo
cd $RD/$VERSION/maemo/multi/dists
ln -s fremantle chinook

cd $RD
rm -f latest
ln -s $VERSION latest


Publish to the Android Market

IMPORTANT - Apparently it is possible for the old apk to "come back from the dead" and on this release it turned out that a second visit to the (insert expletive here) Market was necessary to re-activate the 6.0b4 apk AGAIN. So watch your back with anything to do with the Market and check it twice.
PROTIP: after publishing new apk on android marketplace, ask anyone in RelEng who has an android device to check for updates. Some delay in seeing new apk might be caused by cache-needing-to-refresh. However, a multi-hour delay in seeing new apk means there is something wrong with the settings in marketplace website. Fixing this may require going to the "upload new apk" tab, then clicking the "advanced" tab and looking at the build ids to compare to the build date you're expecting. Insert non-formatted text here

Push index file to mirrors

Added the following lines to the top of stage:/pub/mozilla.org/zz/rsyncd-mozilla-releases.exclude: 

+ firefox
+ firefox/releases
+ firefox/releases/6.0
+ firefox/releases/6.0/*/
+ firefox/releases/6.0/*/*/
+ firefox/releases/6.0/*/*/*/
+ firefox/releases/6.0/**index.html
- firefox/releases/6.0/**

Waited 10 minutes to ensure the rsync server picks up the changes, then removed the previously added "- firefox/releases/6.0" line. After that, ran the following to lay out the index files: 

# ffxbld@stage
cd /pub/mozilla.org/firefox/releases/6.0
wget --no-check-certificate -O index.html https://bugzilla.mozilla.org/attachment.cgi?id=540139
sed -i -e 's/5/6/g' index.html
for dir in `find . -type d -mindepth 1`; do cp -pv index.html $dir/; done

Push to external mirrors

Removed the following lines to the top of stage:/pub/mozilla.org/zz/rsyncd-mozilla-releases.exclude: 

+ firefox
+ firefox/releases
+ firefox/releases/6.0
+ firefox/releases/6.0/*/
+ firefox/releases/6.0/*/*/
+ firefox/releases/6.0/*/*/*/
+ firefox/releases/6.0/**index.html
- firefox/releases/6.0/**

Updated stage:/pub/mozilla.org/zz/rsyncd-mozilla-current.exclude to carry 6.0 instead of 5.0.1.

Forced email once uptake was over 45k.

Final verification

Had to force builders with properties 

release_config: mozilla/release-firefox-mozilla-release.py
script_repo_revision: FIREFOX_6_0_RELEASE
  • linux was green
  • mac was red - consistent miss from mozilla.cdn.cacheboy.net, disabled in bouncer
  • windows was red - some '301 Moved Permanently' from data.mserverz.de, resulting in a couple of 404's at spock.mserverz.de and a couple 200's at tuvok.mserverz.de. Disabled data.mserverz.de
  • linux64 was red - another 301 -> 404 on spock.mserverz.de

Push to Release Channel

Note: use Firefox-6.0-build1-with-5.0-partials directory 

# cltbld@aus2-staging
~/bin/backupsnip Firefox-6.0-build1-with-5.0-partials
~/bin/pushsnip Firefox-6.0-build1-with-5.0-partials

Update symlinks

# ffxbld@stage
cd /pub/mozilla.org/firefox/releases
ln -s 6.0 latest-6.0
rm latest && ln -s latest-6.0 latest

Remove index.html files

cd /pub/mozilla.org/firefox/releases/6.0
find . -name index.html -exec echo rm -v {} \;
find . -name index.html -exec rm -v {} \;

Update browser choice website

Filed bug 679394 - Please update browserchoice website and download links for EUballot for Fx 6.0

Retrieved from "https://wiki.mozilla.org/index.php?title=Releases/Firefox_6.0/BuildNotes&oldid=340499"