Releases/Firefox 3.6.7/BuildNotes: Difference between revisions

(Created page with '== Build Engineers == catlee {{bug|576201|Tracking bug}} == Signed-off Revision(s) == [http://hg.mozilla.org/releases/mozilla-1.9.2/rev/] == Tags == On releases/mozilla-1…')
 
 
(107 intermediate revisions by 3 users not shown)
Line 1: Line 1:
== Build Engineers  ==
== Build Engineers  ==
catlee
bhearsum {{bug|576201|Tracking bug}}
 
 
{{bug|576201|Tracking bug}}


== Signed-off Revision(s)  ==
== Signed-off Revision(s)  ==
[http://hg.mozilla.org/releases/mozilla-1.9.2/rev/]
[http://hg.mozilla.org/releases/mozilla-1.9.2/rev/69bc40a0e08c 69bc40a0e08c]


== Tags  ==
== Tags  ==
Line 19: Line 16:
|-
|-
| rowspan="2" | 1
| rowspan="2" | 1
|  
| GECKO1927_20100701_RELBRANCH
|  
| [http://hg.mozilla.org/releases/mozilla-1.9.2/rev/69bc40a0e08c 69bc40a0e08c]
|-  
|-  
|
| FIREFOX_3_6_7_BUILD1
|  
| [http://hg.mozilla.org/releases/mozilla-1.9.2/rev/cd17c59acd80 cd17c59acd80]
|-
| 2
| FIREFOX_3_6_7_BUILD2 FIREFOX_3_6_7_RELEASE
| [http://hg.mozilla.org/releases/mozilla-1.9.2/rev/2502e5ae6981 2502e5ae6981]
|}
|}


Line 37: Line 38:
| rowspan="3" | 1  
| rowspan="3" | 1  
| Linux  
| Linux  
|  
| 20100701013922
|
| mv-moz2-linux-ix-slave21
|-
|-
| Mac  
| Mac  
|  
| 20100701014848
|
| bm-xserve17
|-
|-
| Windows  
| Windows  
|  
| 20100701024234
|
| mw32-ix-slave03
|-
| rowspan="3" | 2
| Linux
| 20100713120425
| mv-moz2-linux-ix-slave20
|-
| Mac
| 20100713121444
| bm-xserve19
|-
| Windows
| 20100713130626
| mw32-ix-slave23
|}
|}


Line 53: Line 67:
=== Build 1 ===
=== Build 1 ===
* Clobbered 'Any release' on pm with [https://build.mozilla.org/clobberer/index.php clobberer].
* Clobbered 'Any release' on pm with [https://build.mozilla.org/clobberer/index.php clobberer].
** Nick later clobbered all release builders on any master, to be sure that moved slaves would get clobbered.
* Landed [http://hg.mozilla.org/build/buildbot-configs/rev/c8c1450f25d0 automation config]
* Reconfiged pm
* Closed the Firefox3.6 tree
* Started the automation
buildbot sendchange --username=bhearsum --master=localhost:9010 --branch=releases/mozilla-1.9.2 -m "Firefox 3.6.7 build1" Go
* Manually tagged buildbot-configs, buildbotcustom, tools w/ FIREFOX_3_6_7_RELEASE/FIREFOX_3_6_7_BUILD1
* Moved the following slaves to pm:
** mv-moz2-linux-ix-slave14,19,21,22,24
** bm-xserve11,14
** mw32-ix-slave15,19,21,24


==== Tag ====
==== Tag ====
No problems. Tree opened.


==== Source ====
==== Source ====
No problems


==== XULRunner Source ====
==== XULRunner Source ====
No problems


==== Build/Repack ====
==== Build/Repack ====
No problems


==== Partner Repacks ====
==== Partner Repacks ====


==== Unit tests ====
==== Unit tests ====
* Linux: PASS
* Mac: PASS
* Win32: Conditional PASS
** test_playback.html timed out (known orange, {{bug|557432}})


==== Sign  ====
==== Sign  ====
[https://intranet.mozilla.org/Build:CombinedSigning Instructions]
Removed en-US.xpi from stage.
tools repo wasnt tagged, I used default for TAG
<pre>
#!/bin/bash
set -x
set -e
PRODUCT=firefox
VERSION=3.6.7
BUILD=1
TAG=default
REPO="releases/mozilla-1.9.2"
EMAIL=rail@mozilla.com
KEYDIR=d:/2009-keys
mkdir -p ~/signing-work/${PRODUCT}-${VERSION}
cd ~/signing-work/${PRODUCT}-${VERSION}
cd ~/hg-tools
hg pull
hg up -r ${TAG}
cd -
mkdir -p ~/signing-work/${PRODUCT}-${VERSION}
cd ~/signing-work/${PRODUCT}-${VERSION}
cp ~/hg-tools/release/signing/* .
make PRODUCT=${PRODUCT} VERSION=${VERSION} \
  BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} \
  KEYDIR=${KEYDIR} USE_NEW=1
</pre>
Reviewed stdout, win32_signing_build1.log.
make postsign PRODUCT=${PRODUCT} VERSION=${VERSION} \
    BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL}
Got "Signature verification succeeded" email.


==== Generate updates and push betatest snippets ====
==== Generate updates and push betatest snippets ====
Line 72: Line 146:
==== L10N Verification ====
==== L10N Verification ====


==== Update verify ====
This is run separately for each platform since {{bug|571199}}.


==== 3.5.10 -> 3.6.7 MU Generation ====
All platforms are green. Mac has the following (harmless) diff (changes in buildbot path):
<pre>
diff -r firefox-3.6.7-build1/diffs/mac.af.diff firefox-3.6.6-build1/diffs/mac.af.diff
140c140
< < //@line 36 "/builds/slave/macosx_build/build/browser/locales/en-US/firefox-l10n.js"
---
> < //@line 36 "/builds/moz2_slave/macosx_build/build/browser/locales/en-US/firefox-l10n.js"
</pre>


===== Verification =====
==== Update verify ====
No problems


==== 3.0.19 -> 3.6.7 MU ====
==== 3.0.19 -> 3.6.7 MU ====
====== Snippet Generation ======
====== Snippet Generation ======
on moz2-linux-slave45
<pre>
mkdir -p /builds/slave/3.0.19-3.6.7-major-update/snippets
cd /builds/slave/3.0.19-3.6.7-major-update/snippets
cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -d patcher-configs mozilla/tools/patcher-configs
cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R9 -d patcher mozilla/tools/patcher
cd patcher
cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R9 -d MozBuild mozilla/tools/release/MozBuild
export CVSROOT=:ext:cltbld@cvs.mozilla.org:/cvsroot
# build tools
./patcher2.pl --build-tools --app=firefox --tools-rev=UPDATE_PACKAGING_R9 --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_build-tools.log
# download mars
./patcher2.pl --download --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_download.log
# FIXME - patcher needs to see that the MARs that it thinks are partials
#            are there or else it will not attempt to generate patchinfo
cd temp/firefox
ln -s 3.6.7 3.0.19-3.6.7
cd ../..
# Create partial patches and snippets
./patcher2.pl --create-patches --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox-create-patches.log
</pre>
Failed:
<pre>
DownloadFile(): FAILED: 1, output: --09:45:35--  http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/3.0.19-candidates/build1/firefox-3.0.19.af.linux-i686.complete.mar
...
09:45:35 ERROR 404: Not Found.
</pre>
due to {{bug|575966}}.
Fixed:
<pre>
# ffxbld@stage
cd ~
rsync -avP /pub/mozilla.org/firefox/releases/3.0.19-real-real/update ./3.0.19-real-real
mkdir  ./3.0.19-real-real/MU
cd  ./3.0.19-real-real/updates
#
for platform in linux-i686  mac  win32; do
  cd $platform
  for lang in `ls`; do
    cd $lang
        cp firefox-3.0.19.complete.mar  ../../../MU/firefox-3.0.19.${lang}.${platform}.complete.mar
    cd ..
  done
  cd ..
done
cd ../MU
cp -a * /pub/mozilla.org/firefox/nightly/3.0.19-candidates/build1/
</pre>
Reran the first step (took ~35 minutes)


======Quick Verify ======
======Quick Verify ======
Check that releasetest = beta = release
<pre>
cd /builds/slave/3.0.19-3.6.7-major-update/snippets/patcher/temp/firefox/3.0.19-3.6.7/
# releasetest == beta
find aus2.test -type d -iregex '.*/releasetest$' | perl -nle '$a = $_; $a =~ s/releasetest/beta/; $a =~ s/aus2\.test/aus2.beta/; system("diff -r -u $_ $a");'
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/releasetest/; $a =~ s/aus2\.beta/aus2.test/; system("diff -r -u $_ $a");'
# beta == release
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/aus2\.beta/aus2/; $a =~ s/beta/release/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release$/beta/; $a =~ s/aus2/aus2.beta/; system("diff -r -u $_ $a");'
</pre>


======Push to AUS======
======Push to AUS======
<pre>
cd /builds/slave/3.0.19-3.6.7-major-update/snippets/patcher/temp/firefox/3.0.19-3.6.7
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100701-Firefox-3.0.19-3.6.7-MU-test
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.beta/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100701-Firefox-3.0.19-3.6.7-MU-beta
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2/      cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100701-Firefox-3.0.19-3.6.7-MU
</pre>


======Enable test snippets======
======Enable test snippets======
# cltbld@aus2-staging
~/bin/backupsnip 20100701-Firefox-3.0.19-3.6.7-MU-test
~/bin/pushsnip  20100701-Firefox-3.0.19-3.6.7-MU-test


======Update Verify======
======Update Verify======
 
Running update verify on moz2-linux-slave47, mw32-ix-slave23, bm-xserve19
<pre>
mkdir -p /builds/slave/verify/firefox-3019-367-major/
cd /builds/slave/verify/firefox-3019-367-major/
hg clone http://hg.mozilla.org/build/tools
cd tools/release/updates
platform=linux  # or mac or win32
./verify.sh -c moz19-firefox-$platform-major.cfg 2>&1 | tee $platform.log
</pre>
======Results======
======Results======
Checked the logs carefully and found the same differences documented [[Releases/Firefox_3.0.19/BuildNotes#Results | for 3.0.19 -> 3.6's major update]].


==== Socorro product data updates ====
==== Socorro product data updates ====
Added '3.6.7' and '3.6.8pre' to socorro using https://crash-stats.mozilla.com/admin/branch_data_sources
==== Update Bouncer (release entries) ====
Added manually


==== Update Bouncer ====
==== EU Ballot Build ====
This is a win32 partner build for the EU browser ballot.
 
After the builds are created (this time by Kev), and passed QA, they were signed and moved into the candidates dir
# ffxbld@stage
# get copy of signed tarball to /tmp/
mkdir /tmp/ffxbld
cd /tmp/ffxbld
tar xfv ../euballot-367-signed.tar
mv win32 win32-EUballot
mv -v win32-EUballot /pub/mozilla.org/firefox/nightly/3.6.7-candidates/build1/
 
The builds will be pushed when the other ones do, in 'push to mirrors'.
 
==== Virus Scan  ====
# stage
cd /pub/mozilla.org/firefox/nightly/3.6.7-candidates/build1
clamdscan -m `pwd`
----------- SCAN SUMMARY -----------
Infected files: 0
Time: 3135.712 sec (52 m 15 s)
 
==== Push updates for beta channel to mirrors ====
After QA sign off on betatest, and go from driver, we push the update/ directory to the mirrors to cope with the traffic.
# ffxbld@stage
nano /pub/mozilla.org/zz/rsyncd-mozilla-releases.exclude
to add
+ firefox/nightly/3.6.7-candidates/
+ firefox/nightly/3.6.7-candidates/build1
+ firefox/nightly/3.6.7-candidates/build1/update
+ firefox/nightly/3.6.7-candidates/build1/update/**
No installers because we won't be pointing to those from a mozilla.com page.
 
==== Update Bouncer (beta entries) ====
Added Firefox-3.6.7build1-partial-3.6.6 and Firefox-3.6.7build1-complete entries pointing to firefox/nightly/....
 
==== Fix beta & releasetest snippets ====
Switch to using the bouncer entries added just above.
===== Generation =====
# cltbld@aus2-staging
cd /opt/aus2/snippets/staging
# update releasetest snippets
rsync -a --exclude '*betatest*' 20100701-Firefox-3.6.7-test/ 20100701-Firefox-3.6.7build1-test/
touch 20100701-Firefox-3.6.7build1-test/
find 20100701-Firefox-3.6.7build1-test -type f \
  -exec sed -i -e 's/3.6.7-complete/3.6.7build1-complete/' -e 's/3.6.7-partial/3.6.7build1-partial/' {} \;
# duplicate releasetest to beta
rsync -a 20100701-Firefox-3.6.7build1-{test,beta}/
# check for any non-releasetest snippets, got no output
find 20100701-Firefox-3.6.7build1-beta -mindepth 6 -maxdepth 6 -type d | grep -v releasetest$
find 20100701-Firefox-3.6.7build1-beta -mindepth 5 -maxdepth 5 -type d -exec mv -v {}/releasetest {}/beta \;
# check for any non-beta snippets, got no output
find 20100701-Firefox-3.6.7build1-beta -mindepth 6 -maxdepth 6 -type d | grep -v beta$
# push test snippets live
~/bin/backupsnip 20100701-Firefox-3.6.7build1-test
~/bin/pushsnip  20100701-Firefox-3.6.7build1-test
 
===== Update Verify =====
Went to machines that did first update verify, s/betatest/releasetest/ on moz192-firefox-$platform.cfg and reran manually.
 
==== Push updates to beta channel ====
~/bin/backupsnip 20100701-Firefox-3.6.7build1-beta
real    17m2.904s
user    0m1.063s
sys    0m11.161s
~/bin/pushsnip  20100701-Firefox-3.6.7build1-beta
 
real    2m59.730s
user    0m0.129s
sys    0m5.788s


==== Final checks before push  ====
==== Final checks before push  ====
# ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/3.6.7-candidates/build1/
find . ! -user ffxbld #nothing showed up, as expected.
find . ! -group firefox #nothing showed up, as expected.
find . -type f ! -perm 644 # unsigned/partner-repacks have 664 permission
find . -mindepth 1 -type d ! -perm 755 ! -name 'contrib*' # unsigned/partner-repacks have 664 permission
find . -maxdepth 1 -type d ! -perm 2775  -name 'contrib*' # nothing showed up, as expected.
==== XULRunner build ====
No problems
=== Build 2 ===
* Moved the following slaves to pm:
** mv-moz2-linux-ix-slave15-18,20
** bm-xserve19,22, moz2-darwin9-slave44-47
** mw32-ix-slave22,23,win32-slave52-55
* Clobbered 'Any release' on pm with [https://build.mozilla.org/clobberer/index.php clobberer].
* Landed [http://hg.mozilla.org/build/buildbot-configs/rev/a153ad70acfa automation config]
* Reconfiged pm
* Closed the Firefox3.6 tree
* Started the automation
buildbot sendchange --username=bhearsum --master=localhost:9010 --branch=releases/mozilla-1.9.2 -m "Firefox 3.6.7 build2" Go
* Manually tagged buildbot-configs, buildbotcustom, tools w/ FIREFOX_3_6_7_RELEASE/FIREFOX_3_6_7_BUILD2
==== Tag ====
No problems
==== Source/XULRunner Source ====
No problems
==== Build/Repack ====
* Linux: No problems
* Mac: No problems
* Win32: No problems
Removed en-US.xpi from candidates directory.
==== XULRunner Build ====
* Linux: no problems
* Mac: no problems
* Win32: no problems
==== Unittests ====
* Linux: PASS
* Mac: Conditional PASS
** 106886 ERROR TEST-UNEXPECTED-FAIL | /tests/modules/plugin/test/test_streamatclose.html | NPP_URLNotify should be called if streams are active at NPP_Destroy ({{bug|555464}})
* Win32: Conditional PASS
** 41114 ERROR TEST-UNEXPECTED-FAIL | /tests/content/media/test/test_playback.html | Test timed out. ({{bug|558812}})
==== Sign  ====
[https://intranet.mozilla.org/Build:CombinedSigning Instructions]
No problems
==== l10n verify ====
All platforms have no change -> PASS.
==== Updates ====
===== 3.6* -> 3.6.7 =====
====== Generation ======
No problems
====== Update Verify ======
* Linux: PASS
* Mac: PASS
* Win32: PASS
===== 3.6.7build1 -> 3.6.7build2 beta channel =====
====== Generation ======
'''nb: these were originally generated wrongly, causing a 3.6.7build2 -> 3.6.7build2 update loop on betatest. those test snippets were removed, and everything was regenerated from scratch'''
As cltbld@aus2-staging
cd /opt/aus2/snippets/staging
for i in beta test; do
  channel=$i
  if [[ $i == test ]]; then
    channel=betatest
  fi
  export $channel
  mkdir -p 20100714-Firefox-3.6.7-$i-build1-to-build2/Firefox
  rsync -a --exclude 'partial.txt' 20100713-Firefox-3.6.7/Firefox/3.6.6/ 20100714-Firefox-3.6.7-$i-build1-to-build2/Firefox/3.6.7/
  mv 20100714-Firefox-3.6.7-$i-build1-to-build2/Firefox/3.6.7/Linux_x86-gcc3/{20100625222733,20100701013922}
  mv 20100714-Firefox-3.6.7-$i-build1-to-build2/Firefox/3.6.7/Darwin_Universal-gcc3/{20100625223402,20100701014848}
  mv 20100714-Firefox-3.6.7-$i-build1-to-build2/Firefox/3.6.7/WINNT_x86-msvc/{20100625231939,20100701024234}
  # Change the bouncer links
  find 20100714-Firefox-3.6.7-$i-build1-to-build2/ -type f -exec sed -i -e 's/3.6.7-complete/3.6.7build2-complete/' {} \;
  # Create "partials"
  for j in `find 20100714-Firefox-3.6.7-$i-build1-to-build2/ -type f`; do dir=`dirname $j` && cp $j $dir/partial.txt && sed -i -e 's/type=complete/type=partial/' $dir/partial.txt; done
  # Change the update channel
  for j in `find 20100714-Firefox-3.6.7-$i-build1-to-build2/ -mindepth 6 -maxdepth 6 -type d`; do newdir=`echo $j | sed -e "s/release$/$channel/"` && mv $j $newdir; done
done
# Push test snippets
~/bin/backupsnip 20100714-Firefox-3.6.7-test-build1-to-build2
~/bin/pushsnip  20100714-Firefox-3.6.7-test-build1-to-build2
====== Update Verify ======
===== 3.0.19 -> 3.6.7 Major Update =====
====== Generation ======
on moz2-linux-slave46:
<pre>
mkdir -p /builds/slave/3.0.19-3.6.7-major-update/snippets
cd /builds/slave/3.0.19-3.6.7-major-update/snippets
cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -d patcher-configs mozilla/tools/patcher-configs
cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R9 -d patcher mozilla/tools/patcher
cd patcher
cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R9 -d MozBuild mozilla/tools/release/MozBuild
export CVSROOT=:ext:cltbld@cvs.mozilla.org:/cvsroot
# build tools
./patcher2.pl --build-tools --app=firefox --tools-rev=UPDATE_PACKAGING_R9 --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_build-tools.log
# download mars
./patcher2.pl --download --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_download.log
# FIXME - patcher needs to see that the MARs that it thinks are partials
#            are there or else it will not attempt to generate patchinfo
cd temp/firefox
ln -s 3.6.7 3.0.19-3.6.7
cd ../..
# Create partial patches and snippets
./patcher2.pl --create-patches --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox-create-patches.log
</pre>
====== Quick Verify ======
Check that releasetest = beta = release
<pre>
cd temp/firefox/3.0.19-3.6.7
# releasetest == beta
find aus2.test -type d -iregex '.*/releasetest$' | perl -nle '$a = $_; $a =~ s/releasetest/beta/; $a =~ s/aus2\.test/aus2.beta/; system("diff -r -u $_ $a");'
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/releasetest/; $a =~ s/aus2\.beta/aus2.test/; system("diff -r -u $_ $a");'
# beta == release
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/aus2\.beta/aus2/; $a =~ s/beta/release/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release$/beta/; $a =~ s/aus2/aus2.beta/; system("diff -r -u $_ $a");'
</pre>
======Push to AUS======
<pre>
cd /builds/slave/3.0.19-3.6.7-major-update/snippets/patcher/temp/firefox/3.0.19-3.6.7
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100714-Firefox-3.0.19-3.6.7-MU-test
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.beta/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100714-Firefox-3.0.19-3.6.7-MU-beta
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2/      cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100714-Firefox-3.0.19-3.6.7-MU
</pre>
======Enable test snippets======
<pre>
~/bin/backupsnip 20100714-Firefox-3.0.19-3.6.7-MU-test
~/bin/pushsnip  20100714-Firefox-3.0.19-3.6.7-MU-test
</pre>
====== Update Verify ======
Running update verify on moz2-linux-slave46, win32-slave46, moz2-darwin9-slave65
<pre>
mkdir -p /builds/slave/verify/firefox-3019-367-major/
cd /builds/slave/verify/firefox-3019-367-major/
hg clone http://hg.mozilla.org/build/tools
cd tools/release/updates
platform=linux  # or mac or win32
./verify.sh -c moz19-firefox-$platform-major.cfg 2>&1 | tee $platform.log
</pre>
====== Results ======
Checked the logs carefully and found the same differences documented [[Releases/Firefox_3.0.19/BuildNotes#Results | for 3.0.19 -> 3.6's major update]].
==== Update Bouncer (beta) ====
Added "firefox-3.6.7build2-complete" and "firefox-3.6.7build2" entries in bouncer admin, pointing at the candidates directory.
==== Push updates for beta channel to mirrors (NOT!) ====
Based on e-mail from release-drivers (Justin Fitzhugh, July 5, 2010) and conversation with justdave on IRC, we can point beta users directly at ftp.mozilla.org, as we've done in the past.
==== Push updates to beta channel ====
~/bin/backupsnip 20100713-Firefox-3.6.7-beta
~/bin/backupsnip 20100714-Firefox-3.6.7-beta-build1-to-build2
~/bin/pushsnip 20100713-Firefox-3.6.7-beta
~/bin/pushsnip 20100714-Firefox-3.6.7-beta-build1-to-build2
==== EU Ballot Build ====
Generated by Kev. Signed with:
export PRODUCT=firefox
export VERSION=3.6.7
export BUILD=2
export TAG=FIREFOX_3_6_7_RELEASE
export EMAIL=bhearsum@mozilla.com
export KEYDIR=d:/2009-keys
mkdir ~/signing-work/${PRODUCT}-${VERSION}-euballot
cd ~/signing-work/${PRODUCT}-${VERSION}-euballot
cp ~/hg-tools/release/signing/* .
make setup PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
scp cltbld@stage.mozilla.org:/tmp/euballot-367-unsigned.tar .
mkdir unsigned
cd unsigned
tar -xvf ../euballot-367-unsigned.tar
cd win32
../../signing/sign-release ${KEYDIR}
../../signing/sign-files .
cd ..
mv unsigned signed
cd signed
tar -vcf ../euballot-367-signed.tar .
cd ..
scp euballot-367-signed.tar cltbld@stage.mozilla.org:/tmp
Staging:
# ffxbld@stage
# get copy of signed tarball to /tmp/
mkdir /tmp/ffxbld
cd /tmp/ffxbld
tar xfv ../euballot-367-signed.tar
mv win32 win32-EUballot
mv -v win32-EUballot /pub/mozilla.org/firefox/nightly/3.6.7-candidates/build2/
No virus scan here, it will be done with the rest of them before we push to mirrors.
==== Final checks before push  ====
Add back in contrib dir ({{bug|580102}})
# ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/3.6.7-candidates/build2/
mkdir contrib contrib-localized
chmod 2775 contrib contrib-localized
Then do the normal checks
# ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/3.6.6-candidates/build2/
find . ! -user ffxbld
find . ! -group firefox
find . -type f ! -perm 644
find . -mindepth 1 -type d ! -perm 755 ! -name 'contrib*'
find . -maxdepth 1 -type d ! -perm 2775  -name 'contrib*'
Files and dirs in unsigned/partner-repacks have 664 and 775 permissions respectively, so they fail those checks. That's OK because we're not going to push those files.


==== Virus Scan  ====
==== Virus Scan  ====
# stage
cd /pub/mozilla.org/firefox/nightly/3.6.7-candidates/build2
clamdscan -m `pwd`
----------- SCAN SUMMARY -----------
Infected files: 0
Time: 3351.580 sec (55 m 51 s)


==== EU Ballot Build ====
==== Update bouncer ====
Done for build1, entries look OK.


==== Push to mirrors  ====
==== Push to mirrors  ====
# ffxbld@stage
rsync -av --exclude=*.log --exclude=*.txt --exclude=*unsigned*  --exclude=*.crashreporter-symbols.zip --exclude=*.tests.tar.bz2*  --exclude=*partner-repacks*  /pub/mozilla.org/firefox/nightly/3.6.7-candidates/build2/  /pub/mozilla.org/firefox/releases/3.6.7/
Update <tt>cltbld@stage:/pub/mozilla.org/zz/rsyncd-mozilla-current.exclude</tt> to include 3.6.7 instead of 3.6.6.
==== Final Verification ====
Got 404 for easynews.com (automatically disabled later) and mozilla.cs.utah.edu (was in sync state).
mozilla.cs.utah.edu  disabled by hand.
==== Push updates to release channel (and MU to beta) ====
# cltbld @ aus2-staging
~/bin/backupsnip 20100713-Firefox-3.6.7
~/bin/pushsnip 20100713-Firefox-3.6.7


==== Final Verification  ====
The 3.5.11 to 3.6.7 major update is being done [[Releases/Firefox_3.5.11/BuildNotes#Push_updates_to_release_channel| here]].


==== Push updates to release channel ====
For the 3.0.19 -> 3.6.7 major update [https://bugzilla.mozilla.org/attachment.cgi?id=458602 this patch] was landed and deployed before the pushsnip
~/bin/backupsnip 20100714-Firefox-3.0.19-3.6.7-MU
~/bin/pushsnip 20100714-Firefox-3.0.19-3.6.7-MU
~/bin/pushsnip 20100714-Firefox-3.0.19-3.6.7-MU-beta


==== Release ====
==== Release ====
As ffxbld@stage:
cd /home/ftp/pub/firefox/releases
rm latest-3.6 && ln -s 3.6.7 latest-3.6
Sent the following mail to metrics@mozilla.com:
Firefox,firefox,firefox,3.6.7,3.6
Firefox,firefox,firefox,3.6.8pre,3.6
==== Push EU Ballot build ====
Filed {{bug|580364}}. IT did the push around 6pm PDT on July 20
==== Push XULRunner ====
Waiting to hear back from mfinkle/mossop
Waited a day for a response, didn't get one. Pushed out builds anyways.
===== Signing =====
Used the following script:
<pre>
#!/bin/bash
set -x
set -e
PRODUCT=xulrunner
VERSION=1.9.2.7
BUILD=2
TAG=FIREFOX_3_6_7_RELEASE
REPO="releases/mozilla-1.9.2"
EMAIL=bhearsum@mozilla.com
KEYDIR=d:/2009-keys
mkdir -p ~/signing-work/${PRODUCT}-${VERSION}
cd ~/signing-work/${PRODUCT}-${VERSION}
cd ~/hg-tools
hg pull
hg up -r ${TAG}
cd -
cd ~/signing-work/${PRODUCT}-${VERSION}
cp ~/hg-tools/release/signing/* .
make setup PRODUCT=${PRODUCT} VERSION=${VERSION} \
  BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
rsync -av -e "ssh -i /home/cltsign/.ssh/xrbld_dsa" \
  --exclude=*.txt --exclude=*-symbols.zip \
  xrbld@stage.mozilla.org:/home/ftp/pub/${PRODUCT}/nightly/${VERSION}-candidates/build${BUILD}/ \
unsigned-build${BUILD}
rsync -av --exclude=unsigned unsigned-build${BUILD}/ \
signed-build${BUILD}/
rsync -av unsigned-build${BUILD}/unsigned/ \
signed-build${BUILD}/


==== EU Ballot ====
make checksum-files create-sigs stage verify-sigs \
  PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} \
  REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1


==== Push updates to beta channel ====
rmdir signed-build${BUILD}/contrib{,-localized}
 
rsync -av -e "ssh -i /home/cltsign/.ssh/xrbld_dsa" \
  signed-build${BUILD}/ \
  xrbld@stage.mozilla.org:/home/ftp/pub/${PRODUCT}/nightly/${VERSION}-candidates/build${BUILD}/
</pre>
 
===== Push XulRunner to mirrors =====
<pre>
# xrbld@stage
rsync -av --exclude=*.txt --exclude=*unsigned*  --exclude=*-symbols.zip \
  /pub/mozilla.org/xulrunner/nightly/1.9.2.7-candidates/build2/ \
  /pub/mozilla.org/xulrunner/releases/1.9.2.7/
</pre>


==== XULRunner build ====
Edited cltbld@stage:/pub/mozilla.org/zz/rsyncd-mozilla-releases.exclude and added
- xulrunner/releases/1.9.2.6


==== Cleanup ====
===== Update XulRunner wiki =====
Update the links in these two wiki pages for the version change. NOTE: make sure to update all the links on the page!!!
* [https://developer.mozilla.org/En/XULRunner_1.9.2_Release_Notes Release Notes]
* [https://developer.mozilla.org/En/Gecko_SDK Gecko_SDK]

Latest revision as of 16:42, 23 July 2010

Build Engineers

bhearsum Tracking bug

Signed-off Revision(s)

69bc40a0e08c

Tags

On releases/mozilla-1.9.2:

Build # Tag Changeset
1 GECKO1927_20100701_RELBRANCH 69bc40a0e08c
FIREFOX_3_6_7_BUILD1 cd17c59acd80
2 FIREFOX_3_6_7_BUILD2 FIREFOX_3_6_7_RELEASE 2502e5ae6981

Build data

Build # Type Build ID Build machine
1 Linux 20100701013922 mv-moz2-linux-ix-slave21
Mac 20100701014848 bm-xserve17
Windows 20100701024234 mw32-ix-slave03
2 Linux 20100713120425 mv-moz2-linux-ix-slave20
Mac 20100713121444 bm-xserve19
Windows 20100713130626 mw32-ix-slave23

Notes

Build 1

  • Clobbered 'Any release' on pm with clobberer.
    • Nick later clobbered all release builders on any master, to be sure that moved slaves would get clobbered.
  • Landed automation config
  • Reconfiged pm
  • Closed the Firefox3.6 tree
  • Started the automation
buildbot sendchange --username=bhearsum --master=localhost:9010 --branch=releases/mozilla-1.9.2 -m "Firefox 3.6.7 build1" Go
  • Manually tagged buildbot-configs, buildbotcustom, tools w/ FIREFOX_3_6_7_RELEASE/FIREFOX_3_6_7_BUILD1
  • Moved the following slaves to pm:
    • mv-moz2-linux-ix-slave14,19,21,22,24
    • bm-xserve11,14
    • mw32-ix-slave15,19,21,24

Tag

No problems. Tree opened.

Source

No problems

XULRunner Source

No problems

Build/Repack

No problems

Partner Repacks

Unit tests

  • Linux: PASS
  • Mac: PASS
  • Win32: Conditional PASS
    • test_playback.html timed out (known orange, bug 557432)

Sign

Instructions

Removed en-US.xpi from stage.

tools repo wasnt tagged, I used default for TAG

#!/bin/bash
set -x
set -e

PRODUCT=firefox
VERSION=3.6.7
BUILD=1
TAG=default
REPO="releases/mozilla-1.9.2"
EMAIL=rail@mozilla.com
KEYDIR=d:/2009-keys

mkdir -p ~/signing-work/${PRODUCT}-${VERSION}
cd ~/signing-work/${PRODUCT}-${VERSION}
cd ~/hg-tools
hg pull
hg up -r ${TAG}
cd -

mkdir -p ~/signing-work/${PRODUCT}-${VERSION}
cd ~/signing-work/${PRODUCT}-${VERSION}
cp ~/hg-tools/release/signing/* .

make PRODUCT=${PRODUCT} VERSION=${VERSION} \
  BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} \
  KEYDIR=${KEYDIR} USE_NEW=1

Reviewed stdout, win32_signing_build1.log.

make postsign PRODUCT=${PRODUCT} VERSION=${VERSION} \
    BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL}


Got "Signature verification succeeded" email.

Generate updates and push betatest snippets

L10N Verification

This is run separately for each platform since bug 571199.

All platforms are green. Mac has the following (harmless) diff (changes in buildbot path):

diff -r firefox-3.6.7-build1/diffs/mac.af.diff firefox-3.6.6-build1/diffs/mac.af.diff
140c140
< < //@line 36 "/builds/slave/macosx_build/build/browser/locales/en-US/firefox-l10n.js"
---
> < //@line 36 "/builds/moz2_slave/macosx_build/build/browser/locales/en-US/firefox-l10n.js"

Update verify

No problems

3.0.19 -> 3.6.7 MU

Snippet Generation

on moz2-linux-slave45

mkdir -p /builds/slave/3.0.19-3.6.7-major-update/snippets
cd /builds/slave/3.0.19-3.6.7-major-update/snippets

cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -d patcher-configs mozilla/tools/patcher-configs
cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R9 -d patcher mozilla/tools/patcher
cd patcher
cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R9 -d MozBuild mozilla/tools/release/MozBuild

export CVSROOT=:ext:cltbld@cvs.mozilla.org:/cvsroot
# build tools
./patcher2.pl --build-tools --app=firefox --tools-rev=UPDATE_PACKAGING_R9 --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_build-tools.log

# download mars
./patcher2.pl --download --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_download.log

# FIXME - patcher needs to see that the MARs that it thinks are partials
#             are there or else it will not attempt to generate patchinfo
cd temp/firefox
ln -s 3.6.7 3.0.19-3.6.7
cd ../..

# Create partial patches and snippets
./patcher2.pl --create-patches --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox-create-patches.log

Failed:

DownloadFile(): FAILED: 1, output: --09:45:35--  http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/3.0.19-candidates/build1/firefox-3.0.19.af.linux-i686.complete.mar
...
09:45:35 ERROR 404: Not Found.

due to bug 575966.

Fixed:

# ffxbld@stage
cd ~
rsync -avP /pub/mozilla.org/firefox/releases/3.0.19-real-real/update ./3.0.19-real-real
mkdir  ./3.0.19-real-real/MU
cd  ./3.0.19-real-real/updates
# 
for platform in linux-i686  mac  win32; do
  cd $platform
  for lang in `ls`; do
    cd $lang
        cp firefox-3.0.19.complete.mar  ../../../MU/firefox-3.0.19.${lang}.${platform}.complete.mar
    cd ..
  done
  cd ..
done
cd ../MU
cp -a * /pub/mozilla.org/firefox/nightly/3.0.19-candidates/build1/

Reran the first step (took ~35 minutes)

Quick Verify

Check that releasetest = beta = release

cd /builds/slave/3.0.19-3.6.7-major-update/snippets/patcher/temp/firefox/3.0.19-3.6.7/

# releasetest == beta
find aus2.test -type d -iregex '.*/releasetest$' | perl -nle '$a = $_; $a =~ s/releasetest/beta/; $a =~ s/aus2\.test/aus2.beta/; system("diff -r -u $_ $a");'
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/releasetest/; $a =~ s/aus2\.beta/aus2.test/; system("diff -r -u $_ $a");'

# beta == release
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/aus2\.beta/aus2/; $a =~ s/beta/release/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release$/beta/; $a =~ s/aus2/aus2.beta/; system("diff -r -u $_ $a");'
Push to AUS
cd /builds/slave/3.0.19-3.6.7-major-update/snippets/patcher/temp/firefox/3.0.19-3.6.7
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100701-Firefox-3.0.19-3.6.7-MU-test
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.beta/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100701-Firefox-3.0.19-3.6.7-MU-beta
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2/      cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100701-Firefox-3.0.19-3.6.7-MU
Enable test snippets
# cltbld@aus2-staging
~/bin/backupsnip 20100701-Firefox-3.0.19-3.6.7-MU-test
~/bin/pushsnip   20100701-Firefox-3.0.19-3.6.7-MU-test
Update Verify

Running update verify on moz2-linux-slave47, mw32-ix-slave23, bm-xserve19

mkdir -p /builds/slave/verify/firefox-3019-367-major/
cd /builds/slave/verify/firefox-3019-367-major/
hg clone http://hg.mozilla.org/build/tools
 
cd tools/release/updates
platform=linux   # or mac or win32
./verify.sh -c moz19-firefox-$platform-major.cfg 2>&1 | tee $platform.log
Results

Checked the logs carefully and found the same differences documented for 3.0.19 -> 3.6's major update.

Socorro product data updates

Added '3.6.7' and '3.6.8pre' to socorro using https://crash-stats.mozilla.com/admin/branch_data_sources

Update Bouncer (release entries)

Added manually

EU Ballot Build

This is a win32 partner build for the EU browser ballot.

After the builds are created (this time by Kev), and passed QA, they were signed and moved into the candidates dir

# ffxbld@stage
# get copy of signed tarball to /tmp/
mkdir /tmp/ffxbld
cd /tmp/ffxbld
tar xfv ../euballot-367-signed.tar
mv win32 win32-EUballot
mv -v win32-EUballot /pub/mozilla.org/firefox/nightly/3.6.7-candidates/build1/

The builds will be pushed when the other ones do, in 'push to mirrors'.

Virus Scan

# stage
cd /pub/mozilla.org/firefox/nightly/3.6.7-candidates/build1
clamdscan -m `pwd`

----------- SCAN SUMMARY -----------
Infected files: 0
Time: 3135.712 sec (52 m 15 s)

Push updates for beta channel to mirrors

After QA sign off on betatest, and go from driver, we push the update/ directory to the mirrors to cope with the traffic.

# ffxbld@stage
nano /pub/mozilla.org/zz/rsyncd-mozilla-releases.exclude

to add

+ firefox/nightly/3.6.7-candidates/
+ firefox/nightly/3.6.7-candidates/build1
+ firefox/nightly/3.6.7-candidates/build1/update
+ firefox/nightly/3.6.7-candidates/build1/update/**

No installers because we won't be pointing to those from a mozilla.com page.

Update Bouncer (beta entries)

Added Firefox-3.6.7build1-partial-3.6.6 and Firefox-3.6.7build1-complete entries pointing to firefox/nightly/....

Fix beta & releasetest snippets

Switch to using the bouncer entries added just above.

Generation
# cltbld@aus2-staging
cd /opt/aus2/snippets/staging

# update releasetest snippets
rsync -a --exclude '*betatest*' 20100701-Firefox-3.6.7-test/ 20100701-Firefox-3.6.7build1-test/
touch 20100701-Firefox-3.6.7build1-test/
find 20100701-Firefox-3.6.7build1-test -type f \
  -exec sed -i -e 's/3.6.7-complete/3.6.7build1-complete/' -e 's/3.6.7-partial/3.6.7build1-partial/' {} \;

# duplicate releasetest to beta
rsync -a 20100701-Firefox-3.6.7build1-{test,beta}/
# check for any non-releasetest snippets, got no output
find 20100701-Firefox-3.6.7build1-beta -mindepth 6 -maxdepth 6 -type d | grep -v releasetest$
find 20100701-Firefox-3.6.7build1-beta -mindepth 5 -maxdepth 5 -type d -exec mv -v {}/releasetest {}/beta \;
# check for any non-beta snippets, got no output
find 20100701-Firefox-3.6.7build1-beta -mindepth 6 -maxdepth 6 -type d | grep -v beta$

# push test snippets live
~/bin/backupsnip 20100701-Firefox-3.6.7build1-test
~/bin/pushsnip   20100701-Firefox-3.6.7build1-test
Update Verify

Went to machines that did first update verify, s/betatest/releasetest/ on moz192-firefox-$platform.cfg and reran manually.

Push updates to beta channel

~/bin/backupsnip 20100701-Firefox-3.6.7build1-beta

real    17m2.904s
user    0m1.063s
sys     0m11.161s

~/bin/pushsnip   20100701-Firefox-3.6.7build1-beta
 
real    2m59.730s
user    0m0.129s
sys     0m5.788s

Final checks before push

# ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/3.6.7-candidates/build1/
find . ! -user ffxbld #nothing showed up, as expected.
find . ! -group firefox #nothing showed up, as expected.
find . -type f ! -perm 644 # unsigned/partner-repacks have 664 permission
find . -mindepth 1 -type d ! -perm 755 ! -name 'contrib*' # unsigned/partner-repacks have 664 permission
find . -maxdepth 1 -type d ! -perm 2775  -name 'contrib*' # nothing showed up, as expected.

XULRunner build

No problems

Build 2

  • Moved the following slaves to pm:
    • mv-moz2-linux-ix-slave15-18,20
    • bm-xserve19,22, moz2-darwin9-slave44-47
    • mw32-ix-slave22,23,win32-slave52-55
  • Clobbered 'Any release' on pm with clobberer.
  • Landed automation config
  • Reconfiged pm
  • Closed the Firefox3.6 tree
  • Started the automation
buildbot sendchange --username=bhearsum --master=localhost:9010 --branch=releases/mozilla-1.9.2 -m "Firefox 3.6.7 build2" Go
  • Manually tagged buildbot-configs, buildbotcustom, tools w/ FIREFOX_3_6_7_RELEASE/FIREFOX_3_6_7_BUILD2

Tag

No problems

Source/XULRunner Source

No problems

Build/Repack

  • Linux: No problems
  • Mac: No problems
  • Win32: No problems

Removed en-US.xpi from candidates directory.

XULRunner Build

  • Linux: no problems
  • Mac: no problems
  • Win32: no problems

Unittests

  • Linux: PASS
  • Mac: Conditional PASS
    • 106886 ERROR TEST-UNEXPECTED-FAIL | /tests/modules/plugin/test/test_streamatclose.html | NPP_URLNotify should be called if streams are active at NPP_Destroy (bug 555464)
  • Win32: Conditional PASS
    • 41114 ERROR TEST-UNEXPECTED-FAIL | /tests/content/media/test/test_playback.html | Test timed out. (bug 558812)

Sign

Instructions No problems

l10n verify

All platforms have no change -> PASS.

Updates

3.6* -> 3.6.7
Generation

No problems

Update Verify
  • Linux: PASS
  • Mac: PASS
  • Win32: PASS
3.6.7build1 -> 3.6.7build2 beta channel
Generation

nb: these were originally generated wrongly, causing a 3.6.7build2 -> 3.6.7build2 update loop on betatest. those test snippets were removed, and everything was regenerated from scratch

As cltbld@aus2-staging

cd /opt/aus2/snippets/staging
for i in beta test; do
  channel=$i
  if $i == test ; then
    channel=betatest
  fi
  export $channel
  mkdir -p 20100714-Firefox-3.6.7-$i-build1-to-build2/Firefox
  rsync -a --exclude 'partial.txt' 20100713-Firefox-3.6.7/Firefox/3.6.6/ 20100714-Firefox-3.6.7-$i-build1-to-build2/Firefox/3.6.7/
  mv 20100714-Firefox-3.6.7-$i-build1-to-build2/Firefox/3.6.7/Linux_x86-gcc3/{20100625222733,20100701013922}
  mv 20100714-Firefox-3.6.7-$i-build1-to-build2/Firefox/3.6.7/Darwin_Universal-gcc3/{20100625223402,20100701014848}
  mv 20100714-Firefox-3.6.7-$i-build1-to-build2/Firefox/3.6.7/WINNT_x86-msvc/{20100625231939,20100701024234}
  # Change the bouncer links
  find 20100714-Firefox-3.6.7-$i-build1-to-build2/ -type f -exec sed -i -e 's/3.6.7-complete/3.6.7build2-complete/' {} \;
  # Create "partials"
  for j in `find 20100714-Firefox-3.6.7-$i-build1-to-build2/ -type f`; do dir=`dirname $j` && cp $j $dir/partial.txt && sed -i -e 's/type=complete/type=partial/' $dir/partial.txt; done
  # Change the update channel
  for j in `find 20100714-Firefox-3.6.7-$i-build1-to-build2/ -mindepth 6 -maxdepth 6 -type d`; do newdir=`echo $j | sed -e "s/release$/$channel/"` && mv $j $newdir; done
done
# Push test snippets
~/bin/backupsnip 20100714-Firefox-3.6.7-test-build1-to-build2
~/bin/pushsnip   20100714-Firefox-3.6.7-test-build1-to-build2
Update Verify
3.0.19 -> 3.6.7 Major Update
Generation

on moz2-linux-slave46:

mkdir -p /builds/slave/3.0.19-3.6.7-major-update/snippets
cd /builds/slave/3.0.19-3.6.7-major-update/snippets

cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -d patcher-configs mozilla/tools/patcher-configs
cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R9 -d patcher mozilla/tools/patcher
cd patcher
cvs -d:ext:cltbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R9 -d MozBuild mozilla/tools/release/MozBuild

export CVSROOT=:ext:cltbld@cvs.mozilla.org:/cvsroot
# build tools
./patcher2.pl --build-tools --app=firefox --tools-rev=UPDATE_PACKAGING_R9 --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_build-tools.log

# download mars
./patcher2.pl --download --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_download.log

# FIXME - patcher needs to see that the MARs that it thinks are partials
#             are there or else it will not attempt to generate patchinfo
cd temp/firefox
ln -s 3.6.7 3.0.19-3.6.7
cd ../..

# Create partial patches and snippets
./patcher2.pl --create-patches --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox-create-patches.log
Quick Verify

Check that releasetest = beta = release

cd temp/firefox/3.0.19-3.6.7

# releasetest == beta
find aus2.test -type d -iregex '.*/releasetest$' | perl -nle '$a = $_; $a =~ s/releasetest/beta/; $a =~ s/aus2\.test/aus2.beta/; system("diff -r -u $_ $a");'
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/releasetest/; $a =~ s/aus2\.beta/aus2.test/; system("diff -r -u $_ $a");'

# beta == release
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/aus2\.beta/aus2/; $a =~ s/beta/release/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release$/beta/; $a =~ s/aus2/aus2.beta/; system("diff -r -u $_ $a");'
Push to AUS
cd /builds/slave/3.0.19-3.6.7-major-update/snippets/patcher/temp/firefox/3.0.19-3.6.7
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100714-Firefox-3.0.19-3.6.7-MU-test
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.beta/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100714-Firefox-3.0.19-3.6.7-MU-beta
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2/      cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20100714-Firefox-3.0.19-3.6.7-MU
Enable test snippets
~/bin/backupsnip 20100714-Firefox-3.0.19-3.6.7-MU-test
~/bin/pushsnip   20100714-Firefox-3.0.19-3.6.7-MU-test
Update Verify

Running update verify on moz2-linux-slave46, win32-slave46, moz2-darwin9-slave65

mkdir -p /builds/slave/verify/firefox-3019-367-major/
cd /builds/slave/verify/firefox-3019-367-major/
hg clone http://hg.mozilla.org/build/tools
 
cd tools/release/updates
platform=linux   # or mac or win32
./verify.sh -c moz19-firefox-$platform-major.cfg 2>&1 | tee $platform.log
Results

Checked the logs carefully and found the same differences documented for 3.0.19 -> 3.6's major update.

Update Bouncer (beta)

Added "firefox-3.6.7build2-complete" and "firefox-3.6.7build2" entries in bouncer admin, pointing at the candidates directory.

Push updates for beta channel to mirrors (NOT!)

Based on e-mail from release-drivers (Justin Fitzhugh, July 5, 2010) and conversation with justdave on IRC, we can point beta users directly at ftp.mozilla.org, as we've done in the past.

Push updates to beta channel

~/bin/backupsnip 20100713-Firefox-3.6.7-beta
~/bin/backupsnip 20100714-Firefox-3.6.7-beta-build1-to-build2
~/bin/pushsnip 20100713-Firefox-3.6.7-beta
~/bin/pushsnip 20100714-Firefox-3.6.7-beta-build1-to-build2

EU Ballot Build

Generated by Kev. Signed with:

export PRODUCT=firefox
export VERSION=3.6.7
export BUILD=2
export TAG=FIREFOX_3_6_7_RELEASE
export EMAIL=bhearsum@mozilla.com
export KEYDIR=d:/2009-keys

mkdir ~/signing-work/${PRODUCT}-${VERSION}-euballot
cd ~/signing-work/${PRODUCT}-${VERSION}-euballot
cp ~/hg-tools/release/signing/* .
make setup PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
scp cltbld@stage.mozilla.org:/tmp/euballot-367-unsigned.tar .
mkdir unsigned
cd unsigned
tar -xvf ../euballot-367-unsigned.tar
cd win32
../../signing/sign-release ${KEYDIR}
../../signing/sign-files .
cd ..
mv unsigned signed
cd signed
tar -vcf ../euballot-367-signed.tar .
cd ..
scp euballot-367-signed.tar cltbld@stage.mozilla.org:/tmp

Staging:

# ffxbld@stage
# get copy of signed tarball to /tmp/
mkdir /tmp/ffxbld
cd /tmp/ffxbld
tar xfv ../euballot-367-signed.tar
mv win32 win32-EUballot
mv -v win32-EUballot /pub/mozilla.org/firefox/nightly/3.6.7-candidates/build2/

No virus scan here, it will be done with the rest of them before we push to mirrors.

Final checks before push

Add back in contrib dir (bug 580102)

# ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/3.6.7-candidates/build2/
mkdir contrib contrib-localized
chmod 2775 contrib contrib-localized

Then do the normal checks

# ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/3.6.6-candidates/build2/
find . ! -user ffxbld 
find . ! -group firefox 
find . -type f ! -perm 644 
find . -mindepth 1 -type d ! -perm 755 ! -name 'contrib*'
find . -maxdepth 1 -type d ! -perm 2775  -name 'contrib*'

Files and dirs in unsigned/partner-repacks have 664 and 775 permissions respectively, so they fail those checks. That's OK because we're not going to push those files.

Virus Scan

# stage
cd /pub/mozilla.org/firefox/nightly/3.6.7-candidates/build2
clamdscan -m `pwd`

----------- SCAN SUMMARY -----------
Infected files: 0
Time: 3351.580 sec (55 m 51 s)

Update bouncer

Done for build1, entries look OK.

Push to mirrors

# ffxbld@stage
rsync -av --exclude=*.log --exclude=*.txt --exclude=*unsigned*   --exclude=*.crashreporter-symbols.zip --exclude=*.tests.tar.bz2*   --exclude=*partner-repacks*   /pub/mozilla.org/firefox/nightly/3.6.7-candidates/build2/   /pub/mozilla.org/firefox/releases/3.6.7/

Update cltbld@stage:/pub/mozilla.org/zz/rsyncd-mozilla-current.exclude to include 3.6.7 instead of 3.6.6.

Final Verification

Got 404 for easynews.com (automatically disabled later) and mozilla.cs.utah.edu (was in sync state).

mozilla.cs.utah.edu disabled by hand.

Push updates to release channel (and MU to beta)

# cltbld @ aus2-staging
~/bin/backupsnip 20100713-Firefox-3.6.7
~/bin/pushsnip 20100713-Firefox-3.6.7

The 3.5.11 to 3.6.7 major update is being done here.

For the 3.0.19 -> 3.6.7 major update this patch was landed and deployed before the pushsnip

~/bin/backupsnip 20100714-Firefox-3.0.19-3.6.7-MU
~/bin/pushsnip 20100714-Firefox-3.0.19-3.6.7-MU
~/bin/pushsnip 20100714-Firefox-3.0.19-3.6.7-MU-beta

Release

As ffxbld@stage:

cd /home/ftp/pub/firefox/releases
rm latest-3.6 && ln -s 3.6.7 latest-3.6

Sent the following mail to metrics@mozilla.com:

Firefox,firefox,firefox,3.6.7,3.6
Firefox,firefox,firefox,3.6.8pre,3.6

Push EU Ballot build

Filed bug 580364. IT did the push around 6pm PDT on July 20

Push XULRunner

Waiting to hear back from mfinkle/mossop

Waited a day for a response, didn't get one. Pushed out builds anyways.

Signing

Used the following script:

#!/bin/bash
set -x
set -e

PRODUCT=xulrunner
VERSION=1.9.2.7
BUILD=2
TAG=FIREFOX_3_6_7_RELEASE
REPO="releases/mozilla-1.9.2"
EMAIL=bhearsum@mozilla.com
KEYDIR=d:/2009-keys

mkdir -p ~/signing-work/${PRODUCT}-${VERSION}
cd ~/signing-work/${PRODUCT}-${VERSION}
cd ~/hg-tools
hg pull
hg up -r ${TAG}
cd -

cd ~/signing-work/${PRODUCT}-${VERSION}
cp ~/hg-tools/release/signing/* .

make setup PRODUCT=${PRODUCT} VERSION=${VERSION} \
  BUILD=${BUILD} REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1
rsync -av -e "ssh -i /home/cltsign/.ssh/xrbld_dsa" \
  --exclude=*.txt --exclude=*-symbols.zip \
  xrbld@stage.mozilla.org:/home/ftp/pub/${PRODUCT}/nightly/${VERSION}-candidates/build${BUILD}/ \
 unsigned-build${BUILD}
rsync -av --exclude=unsigned unsigned-build${BUILD}/ \
 signed-build${BUILD}/
rsync -av unsigned-build${BUILD}/unsigned/ \
 signed-build${BUILD}/

make checksum-files create-sigs stage verify-sigs \
  PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} \
  REPO=${REPO} EMAIL=${EMAIL} KEYDIR=${KEYDIR} USE_NEW=1

rmdir signed-build${BUILD}/contrib{,-localized}

rsync -av -e "ssh -i /home/cltsign/.ssh/xrbld_dsa" \
  signed-build${BUILD}/ \
  xrbld@stage.mozilla.org:/home/ftp/pub/${PRODUCT}/nightly/${VERSION}-candidates/build${BUILD}/
Push XulRunner to mirrors
# xrbld@stage
rsync -av --exclude=*.txt --exclude=*unsigned*  --exclude=*-symbols.zip \
  /pub/mozilla.org/xulrunner/nightly/1.9.2.7-candidates/build2/ \
  /pub/mozilla.org/xulrunner/releases/1.9.2.7/

Edited cltbld@stage:/pub/mozilla.org/zz/rsyncd-mozilla-releases.exclude and added

- xulrunner/releases/1.9.2.6
Update XulRunner wiki

Update the links in these two wiki pages for the version change. NOTE: make sure to update all the links on the page!!!