Confirmed users
4,293
edits
(Created page with 'Be sure to use RELEASE_AUTOMATION_M14 in the buildbot master config and when tagging the Bootstrap config file, it fixes up the tagging of the changes from {{bug|483232}} for par...') |
No edit summary |
||
| (40 intermediate revisions by 3 users not shown) | |||
| Line 1: | Line 1: | ||
Be sure to use RELEASE_AUTOMATION_M14 in the buildbot master config and when tagging the Bootstrap config file, it fixes up the tagging of the changes from {{bug|483232}} for partner repacks - nthomas | Be sure to use RELEASE_AUTOMATION_M14 in the buildbot master config and when tagging the Bootstrap config file, it fixes up the tagging of the changes from {{bug|483232}} for partner repacks - nthomas | ||
==Build Engineers== | |||
catlee <br /> | |||
[https://bugzilla.mozilla.org/show_bug.cgi?id=493725 Tracking release bug] | |||
==Bonsai queries== | |||
* [http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=all&branch=HEAD&branchtype=match&dir=&file=&filetype=match&who=&whotype=match&sortby=Date&hours=2&date=explicit&mindate=2009-05-18+15%3A46+PDT&maxdate=2009-05-18+15%3A48+PDT&cvsroot=%2Fcvsroot cvsroot] | |||
* [http://bonsai-l10n.mozilla.org/cvsquery.cgi?treeid=default&module=all&branch=HEAD&branchtype=match&dir=&file=&filetype=match&who=&whotype=match&sortby=Date&hours=2&date=explicit&mindate=2009-05-01+06%3A31+PDT&maxdate=2009-05-01+06%3A33+PDT&cvsroot=%2Fl10n l10n] | |||
==Tags== | |||
Build 1: | |||
{| class="fullwidth-table" | |||
| style="background:#efefef" | '''Module''' | |||
| style="background:#efefef" | '''Branch''' | |||
| style="background:#efefef" | '''Tag''' | |||
| style="background:#efefef" | '''Pull date''' | |||
|- | |||
| cvsroot/mozilla | |||
| HEAD | |||
| GECKO190_20090519_RELBRANCH FIREFOX_3_0_11_BUILD1 FIREFOX_3_0_11_RELEASE | |||
| | |||
|- | |||
| l10n/l10n | |||
| HEAD | |||
| GECKO190_20090519_RELBRANCH FIREFOX_3_0_11_BUILD1 FIREFOX_3_0_11_RELEASE | |||
| | |||
|} | |||
Build 2: | |||
{| class="fullwidth-table" | |||
| style="background:#efefef" | '''Module''' | |||
| style="background:#efefef" | '''Branch''' | |||
| style="background:#efefef" | '''Tag''' | |||
| style="background:#efefef" | '''Pull date''' | |||
|- | |||
| cvsroot/mozilla | |||
| HEAD | |||
| GECKO190_20090519_RELBRANCH FIREFOX_3_0_11_BUILD2 FIREFOX_3_0_11_RELEASE | |||
| | |||
|- | |||
| l10n/l10n | |||
| HEAD | |||
| GECKO190_20090519_RELBRANCH FIREFOX_3_0_11_BUILD2 FIREFOX_3_0_11_RELEASE | |||
| | |||
|} | |||
==Build data== | |||
{| class="fullwidth-table" | |||
| style="background:#efefef" | '''Type''' | |||
| style="background:#efefef" | '''Build ID''' | |||
| style="background:#efefef" | '''SHA1''' | |||
| style="background:#efefef" | '''Push date''' | |||
| style="background:#efefef" | '''Build machine''' | |||
|} | |||
==Notes== | |||
===Build 1=== | |||
Setup before starting: | |||
* On fx-win32-1.9-slave2: | |||
** Deleted /e/xr19rel/WINNT_5.2_Depend | |||
** Deleted /e/fx19l10nrel/WINNT_5.2_Depend | |||
** Deleted /e/fx19rel/WINNT_5.2_Depend | |||
* On fx-linux-1.9-slave2: | |||
** Deleted /builds/verify/firefox-3.0.10 | |||
** Deleted /builds/updates/firefox-3.0.10 | |||
** Deleted /builds/source/firefox-3.0.10 | |||
** Deleted /builds/data/cltbld/firefox-3.0.10 | |||
** Deleted /builds/tinderbox/Xr-Mozilla1.9-Release/Linux_2.6.18-53.1.13.el5_Depend | |||
** Deleted /builds/tags/FIREFOX_3_0_3_BUILD1 | |||
** Deleted /builds/tags/FIREFOX_3_0_4_BUILD1 | |||
* On fx-mac-1.9-slave2: | |||
** Deleted /builds/verify/firefox-3.0.10 | |||
* Space free on slaves: | |||
** fx-mac-1.9-slave2: 28G free on / | |||
** fx-win32-1.9-slave2: 17G free on D, 14G free on E | |||
** fx-linux-1.9-slave2: 21G free on /builds | |||
* production-1.9-master reconfigured with change from https://bugzilla.mozilla.org/attachment.cgi?id=378318 | |||
* Kicked off automation: | |||
buildbot sendchange --username=catlee --master=localhost:9989 -m"Firefox 3.0.11build1 release" go | |||
====Tag==== | |||
No problems | |||
==== Source ==== | |||
No problems | |||
==== Build & Repack ==== | |||
* No problems | |||
==== Sign ==== | |||
* [https://intranet.mozilla.org/Build:Signing Signing doc] | |||
No problems | |||
====L10nVerify==== | |||
No problems | |||
====Generate Updates==== | |||
No problems | |||
====Publish Updates to Test Channels (betatest & releasetest)==== | |||
No problems | |||
====Update Verify==== | |||
Linux failed because pushing to hg would have created a new head. Clobbered /builds/verify/firefox-3.0.11/* on fx-linux-1.9-slave2 and rebuilt failed build. Second build passed. | |||
Windows failed with "Step Updates died: shell call returned bad exit code: 1 at Bootstrap/Step.pm line 110.". Clobbered e:\builds\buildbot\trunk-automation\win32_update_verify\build and rebuilt. Second build failed with normal binary difference errors. | |||
Mac passed on the first try. | |||
====Stage==== | |||
No problems | |||
====Partner Repacks==== | |||
First build failed because 7z couldn't be found. coop fixed this on the slave and forced a new build which ran successfully. | |||
==== Push updates to beta channel ==== | |||
# cltbld@aus2-staging.m.o | |||
# make sure scripts are up to date | |||
cd bin | |||
cvs update | |||
cd /opt/aus2/snippets/staging/ | |||
~/bin/backupsnip 20090519-Firefox-3.0.11-beta | |||
~/bin/pushsnip 20090519-Firefox-3.0.11-beta | |||
At this point the release process was interrupted due to {{bug|495648}}. | |||
===Build 2=== | |||
Setup before starting: | |||
* On fx-win32-1.9-slave2: | |||
** Deleted /e/fx19rel/WINNT_5.2_Depend | |||
** Deleted /e/fx19l10nrel/WINNT_5.2_Depend | |||
* On fx-linux-1.9-slave2: | |||
** Deleted /builds/verify/firefox-3.0.11 | |||
** Deleted /builds/updates/firefox-3.0.11 | |||
** Deleted /builds/source/firefox-3.0.11 | |||
** Deleted /builds/tags/FIREFOX_3_0_5_BUILD1 | |||
** Deleted /builds/data/cltbld/firefox-3.0.11 | |||
* Space free on slaves: | |||
** fx-mac-1.9-slave2: 26GB free on / | |||
** fx-win32-1.9-slave2: 16GB free on D, 14GB free on E | |||
** fx-linux-1.9-slave2: 20GB free on /builds | |||
* Kicked off automation: | |||
buildbot sendchange --username=catlee --master=localhost:9989 -m"Firefox 3.0.11build2 release" go | |||
====Tag==== | |||
No problems | |||
==== Source ==== | |||
No problems | |||
==== Build & Repack ==== | |||
No problems | |||
==== XULRunner ==== | |||
Had to kick off manually. No problems after that. | |||
==== Sign ==== | |||
* [https://intranet.mozilla.org/Build:Signing Signing doc] | |||
No problems | |||
====L10nVerify==== | |||
No problems | |||
====Generate Updates==== | |||
No problems | |||
Need to bring people who already updated to 3.0.11 build1 up to build2, so do this (after the automated run) | |||
# connect as cltbld@fx-linux-1.9-slave2 | |||
cd /builds/updates/firefox-3.0.11/patcher/temp/firefox/3.0.10-3.0.11 | |||
for dir in aus2.test aus2.beta; do | |||
# copy complete snippets from 3.0.10 into a new dir for 3.0.11 | |||
mkdir -p b1-b2-${dir}/Firefox | |||
rsync -a --exclude 'partial.txt' ${dir}/Firefox/3.0.10/ b1-b2-${dir}/Firefox/3.0.11/ | |||
# use the buildid's from build1 | |||
mv b1-b2-${dir}/Firefox/3.0.11/Darwin_Universal-gcc3/2009042315 b1-b2-${dir}/Firefox/3.0.11/Darwin_Universal-gcc3/2009051908 | |||
mv b1-b2-${dir}/Firefox/3.0.11/Linux_x86-gcc3/2009042315 b1-b2-${dir}/Firefox/3.0.11/Linux_x86-gcc3/2009051908 | |||
mv b1-b2-${dir}/Firefox/3.0.11/WINNT_x86-msvc/2009042316 b1-b2-${dir}/Firefox/3.0.11/WINNT_x86-msvc/2009051909 | |||
done | |||
rsync -a -e 'ssh -i ~/.ssh/aus' b1-b2-aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20090605-Firefox-3.0.11build1-build2-test | |||
rsync -a -e 'ssh -i ~/.ssh/aus' b1-b2-aus2.beta/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20090605-Firefox-3.0.11build1-build2-beta | |||
Then go to aus2-staging and push 20090605-Firefox-3.0.11build1-build2-test live. | |||
====Publish Updates to Test Channels (betatest & releasetest)==== | |||
No problems | |||
====Update Verify==== | |||
Linux failed because pushing to hg would have created a new head. Clobbered /builds/verify/firefox-3.0.11/* on fx-linux-1.9-slave2 and rebuilt failed build. The second build passed. | |||
Windows failed with "Step Updates died: shell call returned bad exit code: 1 at Bootstrap/Step.pm line 110.". Clobbered e:\builds\buildbot\trunk-automation\win32_update_verify\build and rebuilt. The second build passed with the normal warnings about binary differences. | |||
Mac passed. | |||
====Stage==== | |||
No problems | |||
====Partner Repacks==== | |||
Had to be forced manually, but ran without problems. | |||
==== Push updates to beta channel ==== | |||
# cltbld@aus2-staging.m.o | |||
# make sure scripts are up to date | |||
cd bin | |||
cvs update | |||
cd /opt/aus2/snippets/staging/ | |||
~/bin/backupsnip 20090605-Firefox-3.0.11build1-build2-test | |||
~/bin/pushsnip 20090603-Firefox-3.0.11-beta | |||
~/bin/pushsnip 20090605-Firefox-3.0.11build1-build2-beta | |||
==== Sign Installers ==== | |||
Done manually using these installer-signing-instructions [https://intranet.mozilla.org/Build:Unified_Release_Process#Sign_builds here]. | |||
* complete stage-merged: | |||
# on stage | |||
cd /data/cltbld/firefox-3.0.11/ | |||
rsync -av batch1/mar/ stage-merged/ | |||
rsync -av batch1/stage-signed/ stage-merged/ | |||
* Create MD5 and SHA1 checksum files | |||
# on stage | |||
cd /data/cltbld/firefox-3.0.11/stage-merged/ | |||
~/bin/checksum-files . | |||
* Fix permissions & ownership (on the two SUM files, and the detached sigs) | |||
chown -R cltbld:firefox . | |||
chmod 644 *SUMS | |||
====Update Bouncer==== | |||
* Done. | |||
* Note for next release: Do not remove the Check Now bit on the Firefox-3.0.10 Products until well after the change to the rsync module (to prevent the likes of bug 464566) | |||
==== Push to mirrors ==== | |||
* push the stage-merged directory to the releases area: | |||
# on stage | |||
rsync -av /data/cltbld/firefox-3.0.11/stage-merged/ /home/ftp/pub/firefox/releases/3.0.11/ | |||
* edit the exclude file /pub/mozilla.org/zz/rsyncd-mozilla-current.exclude to add the new release (3.0.11) and remove the previous release (3.0.10). | |||
====Final Verification==== | |||
* Verify that releasetest points to valid bounce links: | |||
hg clone http://hg.mozilla.org/build/tools | |||
cd tools/release/updates | |||
cat moz19-firefox-*.cfg | grep -v major | sed 's/betatest/releasetest/' > update.cfg | |||
./verify.sh -t update.cfg 2>&1 | tee quickVerify.log | |||
* Look for any HTTP error codes besides 200 ("OK") and 302 ("Found"): | |||
grep HTTP quickVerify.log | grep -v 200 | grep -v 302 | |||
** A few 503 errors from mirrors.directorymix.com that were resolved when I went to check manually. | |||
* Before pushing final updates, verify that "release" and "releasetest" channel match: | |||
# on aus2-staging | |||
$ cd /opt/aus2/snippets/staging/20090603-Firefox-3.0.11 | |||
$ find -type d -iregex '.*release.*' | perl -nle '$a = $_; $a =~ s/release/releasetest/; system("diff -r -u $_ ../20090603-Firefox-3.0.11-test/$a");' | |||
$ | |||
==== Publish Updates to Release Channel ==== | |||
# cltbld@aus2-staging | |||
cd /opt/aus2/snippets/staging | |||
~/bin/backupsnip 20090603-Firefox-3.0.11 | |||
~/bin/pushsnip 20090603-Firefox-3.0.11 | |||
====Release==== | |||
# cltbld@stage | |||
cd /home/ftp/pub/firefox/releases | |||
rm latest-3.0 && ln -s 3.0.11 latest-3.0 | |||
==== Release matching XULRunner build ==== | |||
XULRunner builds are produced by the automation, this is to groom them and push to the mirrors. | |||
# cltbld@stage | |||
VERSION=1.9.0.11 | |||
FX_VERSION=3.0.11 | |||
BUILD=1 | |||
mkdir -p ~/xulrunner-${VERSION}/batch1/stage-unsigned/{runtimes,source,sdk} | |||
cd ~/xulrunner-${VERSION}/batch1/stage-unsigned | |||
rsync -av --exclude=*sdk* --exclude=*info* \ | |||
/home/ftp/pub/xulrunner/nightly/${VERSION}-candidates/build${BUILD}/ runtimes/ | |||
rsync -av --include=*sdk* --exclude=* \ | |||
/home/ftp/pub/xulrunner/nightly/${VERSION}-candidates/build${BUILD}/ sdk/ | |||
cp /home/ftp/pub/firefox/releases/${FX_VERSION}/source/firefox-${FX_VERSION}-source.tar.bz2 \ | |||
source/xulrunner-${VERSION}-source.tar.bz2 | |||
cd ~/xulrunner-${VERSION}/batch1 | |||
rsync -av stage-unsigned/ stage-signed/ | |||
Then create detached signatures per [https://intranet.mozilla.org/Build:Unified_Release_Process#Sign_builds usual process] (with PRODUCT=xulrunner, VERSION=1.9.0.11, don't need BUILD or TAG), skip the win32 signing section. Back on stage: | |||
cd ~/xulrunner-${VERSION}/batch1/stage-signed | |||
cp /home/ftp/pub/firefox/releases/${FX_VERSION}/KEY . | |||
~/bin/checksum-files . | |||
chown -R cltbld:xulrunner . | |||
find . -type f -exec chmod -v 644 {} \; | |||
find . -type d -exec chmod -v 755 {} \; | |||
cd ~/xulrunner-${VERSION}/batch1 | |||
rsync -nav stage-signed/* /home/ftp/pub/xulrunner/releases/${VERSION}/ | |||
If all is fine then | |||
rsync -av stage-signed/* /home/ftp/pub/xulrunner/releases/${VERSION}/ | |||
Ping mfinkle or Mossop to update the links in these two wiki pages for the version change | |||
* [https://developer.mozilla.org/En/XULRunner_1.9_Release_Notes Release Notes] | |||
* [https://developer.mozilla.org/En/Gecko_SDK Gecko_SDK] | |||