Releases/Firefox 3.0.x MUtest/BuildNotes

Build Engineers

nthomas

bug 459878

From 3.0.5 to 3.1b2

The first iteration.

Generate updates

Created patcher and update verify configs

# ssh cltbld@prometheus-vm.build.mozilla.org
mkdir -p /builds/3.0.5-major-update-test/snippets
cd /builds/3.0.5-major-update-test/snippets

cvs -d:ext:ffxbld@cvs.mozilla.org:/cvsroot co -d patcher-configs mozilla/tools/patcher-configs
cvs -d:ext:ffxbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R5 -d patcher mozilla/tools/patcher
cd patcher
cvs -d:ext:ffxbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R5 -d MozBuild mozilla/tools/release/MozBuild

export CVSROOT=:ext:ffxbld@cvs.mozilla.org:/cvsroot
# build tools
./patcher2.pl --build-tools --app=firefox --tools-rev=UPDATE_PACKAGING_R5 --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_build-tools.log

# download mars
./patcher2.pl --download --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_download.log

# FIXME - patcher needs to see that the MARs that it thinks are partials
#             are there or else it will not attempt to generate patchinfo
cd temp/firefox
ln -s 3.1b2 3.0.5-3.1b2
cd ../..

# Create partial patches and snippets
./patcher2.pl --create-patches --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox-create-patches.log

Separate out beta snippets

Skipped over this because they won't be needed.

cd temp/firefox/3.0.5-3.1b2
rsync -a aus2/ aus2.beta/
find aus2.beta/ -type d -name "release" -exec rm -rfv {} \;
find aus2/ -type d -name "beta" -exec rm -rfv {} \; 

Quick verify

Check the snippets are identical for all four channels

# betatest == releasetest
find aus2.test -type d -iregex '.*betatest.*' | perl -nle '$a = $_; $a =~ s/betatest/releasetest/; system("diff -r -u $_ $a");'
# test the opposite direction to catch anything that only exists for releasetest
find aus2.test -type d -iregex '.*releasetest.*' | perl -nle '$a = $_; $a =~ s/releasetest/betatest/; system("diff -r -u $_ $a");'

Also skipped this bit

# betatest == beta
find aus2.test -type d -iregex '.*betatest.*' | perl -nle '$a = $_; $a =~ s/betatest/beta/; $a =~ s/aus2\.test/aus2.beta/; system("diff -r -u $_ $a");'
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/betatest/; $a =~ s/aus2\.beta/aus2.test/; system("diff -r -u $_ $a");'

but did this as a sanity check

# betatest == release
find aus2.test -type d -iregex '.*betatest.*' | perl -nle '$a = $_; $a =~ s/betatest/release/; $a =~ s/aus2\.test/aus2/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release$/betatest/; $a =~ s/aus2/aus2.test/; system("diff -r -u $_ $a");'

Got no output for each test.

Push to AUS

cd /builds/3.0.5-major-update-test/snippets/patcher/temp/firefox/3.0.5-3.1b2
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20090105-Firefox-3.0.5-3.1b2-MU-test

Enable test snippets

# cltbld@aus2-staging
~/bin/backupsnip 20090105-Firefox-3.0.5-3.1b2-MU-test
~/bin/pushsnip 20090105-Firefox-3.0.5-3.1b2-MU-test

Update verify

Ran update verify on fx-{linux,mac,win32}-1.9-slave2 as follows:

mkdir -p /builds/verify/firefox-305-31b2-test-major/
cd /builds/verify/firefox-305-31b2-test-major/
cvs -d cltbld@cvs.mozilla.org:/cvsroot co -d updates mozilla/testing/release/updates/
cvs -d cltbld@cvs.mozilla.org:/cvsroot co -d common mozilla/testing/release/common/

# get patch to trawl "Only in" directories
cd common
# on mac & linux
curl -s https://bugzilla.mozilla.org/attachment.cgi?id=304252 | patch -p0
# on win32
wget --no-check-certificate -O patch https://bugzilla.mozilla.org/attachment.cgi?id=304252
patch -p0 < patch

# everyone ...
platform=linux   # or mac or win32
cd ../updates
./verify.sh -c moz19-firefox-$platform-major.cfg 2>&1 | tee $platform.log

I posted the results and analysis on the bug: win32, linux, and mac. There are a few things to tidy up in a patch.

From 3.0.7 to 3.1b3

Updated the patcher and update verify configs.

# ssh cltbld@prometheus-vm.build.mozilla.org
mkdir -p /builds/3.0.7-major-update-test/snippets
cd /builds/3.0.7-major-update-test/snippets

cvs -d:ext:ffxbld@cvs.mozilla.org:/cvsroot co -d patcher-configs mozilla/tools/patcher-configs
cvs -d:ext:ffxbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R7 -d patcher mozilla/tools/patcher
cd patcher
cvs -d:ext:ffxbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R7 -d MozBuild mozilla/tools/release/MozBuild

export CVSROOT=:ext:ffxbld@cvs.mozilla.org:/cvsroot
# build tools
./patcher2.pl --build-tools --app=firefox --tools-rev=UPDATE_PACKAGING_R7 --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_build-tools.log

# download mars
./patcher2.pl --download --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_download.log

# FIXME - patcher needs to see that the MARs that it thinks are partials
#             are there or else it will not attempt to generate patchinfo
cd temp/firefox
ln -s 3.1b3 3.0.7-3.1b3
cd ../..

# Create partial patches and snippets
./patcher2.pl --create-patches --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox-create-patches.log

Separate out beta snippets

Skipped over this because they won't be needed.

cd temp/firefox/3.0.7-3.1b3
rsync -a aus2/ aus2.beta/
find aus2.beta/ -type d -name "release" -exec rm -rfv {} \;
find aus2/ -type d -name "beta" -exec rm -rfv {} \;

Quick verify

Check the snippets are identical for all four channels

# betatest == releasetest
find aus2.test -type d -iregex '.*betatest.*' | perl -nle '$a = $_; $a =~ s/betatest/releasetest/; system("diff -r -u $_ $a");'
# test the opposite direction to catch anything that only exists for releasetest
find aus2.test -type d -iregex '.*releasetest.*' | perl -nle '$a = $_; $a =~ s/releasetest/betatest/; system("diff -r -u $_ $a");'

Also skipped this bit

# betatest == beta
find aus2.test -type d -iregex '.*betatest.*' | perl -nle '$a = $_; $a =~ s/betatest/beta/; $a =~ s/aus2\.test/aus2.beta/; system("diff -r -u $_ $a");'
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/betatest/; $a =~ s/aus2\.beta/aus2.test/; system("diff -r -u $_ $a");'

but did this as a sanity check

# betatest == release
find aus2.test -type d -iregex '.*betatest.*' | perl -nle '$a = $_; $a =~ s/betatest/release/; $a =~ s/aus2\.test/aus2/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release$/betatest/; $a =~ s/aus2/aus2.test/; system("diff -r -u $_ $a");'

Got no output for each test.

Push to AUS

cd /builds/3.0.7-major-update-test/snippets/patcher/temp/firefox/3.0.7-3.1b3
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20090312-Firefox-3.0.7-3.1b3-MU-test

Enable test snippets

# cltbld@aus2-staging
~/bin/backupsnip 20090312-Firefox-3.0.7-3.1b3-MU-test
~/bin/pushsnip 20090312-Firefox-3.0.7-3.1b3-MU-test

Update verify

Ran update verify on fx-{linux,mac,win32}-1.9-slave2 as follows:

mkdir -p /builds/verify/firefox-307-31b3-test-major/
cd /builds/verify/firefox-307-31b3-test-major/
hg clone http://hg.mozilla.org/build/tools

# get patch to trawl "Only in" directories
cd tools/release/common
# on mac & linux
curl -sL https://bugzilla.mozilla.org/attachment.cgi?id=304252 | patch -p0
# on win32
wget --no-check-certificate -O patch https://bugzilla.mozilla.org/attachment.cgi?id=304252
patch -p0 < patch

# everyone ...
cd ../updates
platform=linux   # or mac or win32
./verify.sh -c moz19-firefox-$platform-major.cfg 2>&1 | tee $platform.log

I posted the results and analysis on the bug: win32, linux, and mac. No packaging changes needed.

Improved patch to list contents of directories in only one side of the diff. Went back and used this on Mac to cope with the space in "Default Plugin".

From 3.0.10 to 3.5b4

Updated the patcher and update verify configs.

# ssh cltbld@prometheus-vm.build.mozilla.org
mkdir -p /builds/3.0.10-3.5b4-major-update-test/snippets
cd /builds/3.0.10-3.5b4-major-update-test/snippets

cvs -d:ext:ffxbld@cvs.mozilla.org:/cvsroot co -d patcher-configs mozilla/tools/patcher-configs
cvs -d:ext:ffxbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R7 -d patcher mozilla/tools/patcher
cd patcher
cvs -d:ext:ffxbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R7 -d MozBuild mozilla/tools/release/MozBuild

export CVSROOT=:ext:ffxbld@cvs.mozilla.org:/cvsroot
# build tools
./patcher2.pl --build-tools --app=firefox --tools-rev=UPDATE_PACKAGING_R7 --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_build-tools.log

# download mars
./patcher2.pl --download --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_download.log

# FIXME - patcher needs to see that the MARs that it thinks are partials
#             are there or else it will not attempt to generate patchinfo
cd temp/firefox
ln -s 3.5b4 3.0.10-3.5b4
cd ../..

# Create partial patches and snippets
./patcher2.pl --create-patches --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox-create-patches.log

Separate out beta snippets

Skipped over this because they won't be needed.

cd temp/firefox/3.0.10-3.5b4
rsync -a aus2/ aus2.beta/
find aus2.beta/ -type d -name "release" -exec rm -rfv {} \;
find aus2/ -type d -name "beta" -exec rm -rfv {} \;

Quick verify

Check the snippets are identical for all four channels

# betatest == releasetest
find aus2.test -type d -iregex '.*betatest.*' | perl -nle '$a = $_; $a =~ s/betatest/releasetest/; system("diff -r -u $_ $a");'
# test the opposite direction to catch anything that only exists for releasetest
find aus2.test -type d -iregex '.*releasetest.*' | perl -nle '$a = $_; $a =~ s/releasetest/betatest/; system("diff -r -u $_ $a");'

Got no output for each test. Then skipped this bit

# betatest == beta
find aus2.test -type d -iregex '.*betatest.*' | perl -nle '$a = $_; $a =~ s/betatest/beta/; $a =~ s/aus2\.test/aus2.beta/; system("diff -r -u $_ $a");'
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/betatest/; $a =~ s/aus2\.beta/aus2.test/; system("diff -r -u $_ $a");'

but did this as a sanity check

# betatest == release
find aus2.test -type d -iregex '.*betatest.*' | perl -nle '$a = $_; $a =~ s/betatest/release/; $a =~ s/aus2\.test/aus2/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release$/betatest/; $a =~ s/aus2/aus2.test/; system("diff -r -u $_ $a");'

Push to AUS

cd /builds/3.0.10-3.5b4-major-update-test/snippets/patcher/temp/firefox/3.0.10-3.5b4
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20090506-Firefox-3.0.10-3.5b4-MU-test

Enable test snippets

# cltbld@aus2-staging
# ~/bin/backupsnip 20090506-Firefox-3.0.10-3.5b4-MU-test
~/bin/pushsnip 20090506-Firefox-3.0.10-3.5b4-MU-test

Skipped the backupsnip, because I ran it just before enabling 2.0.0.20 --> 3.0.10 test snippets.

Update verify

Ran update verify on fx-{linux,mac,win32}-1.9-slave2 as follows:

mkdir -p /builds/verify/firefox-3010-35b4-major-test/
cd /builds/verify/firefox-3010-35b4-major-test/
hg clone http://hg.mozilla.org/build/tools

# get patch to trawl "Only in" directories
cd tools/release/common
# on mac & linux
curl -sL https://bugzilla.mozilla.org/attachment.cgi?id=367544 | patch -p3
# on win32
wget --no-check-certificate -O patch https://bugzilla.mozilla.org/attachment.cgi?id=367544
patch -p3 < patch

# everyone ...
cd ../updates
platform=linux   # or mac or win32
./verify.sh -c moz19-firefox-$platform-major.cfg 2>&1 | tee $platform.log

I posted the results and analysis on the bug: win32, linux, and mac. No packaging changes needed.

From 3.0.11 to 3.5 RC2 build2

Updated the patcher and update verify configs.

# ssh cltbld@prometheus-vm.build.mozilla.org
mkdir -p /builds/3.0.11-3.5rc2build2-major-update-test/snippets
cd /builds/3.0.11-3.5rc2build2-major-update-test/snippets

cvs -d:ext:ffxbld@cvs.mozilla.org:/cvsroot co -d patcher-configs mozilla/tools/patcher-configs
cvs -d:ext:ffxbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R8 -d patcher mozilla/tools/patcher
cd patcher
cvs -d:ext:ffxbld@cvs.mozilla.org:/cvsroot co -r UPDATE_PACKAGING_R8 -d MozBuild mozilla/tools/release/MozBuild

export CVSROOT=:ext:ffxbld@cvs.mozilla.org:/cvsroot
# build tools
./patcher2.pl --build-tools --app=firefox --tools-rev=UPDATE_PACKAGING_R8 --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_build-tools.log

# download mars
./patcher2.pl --download --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox_download.log

# FIXME - patcher needs to see that the MARs that it thinks are partials
#             are there or else it will not attempt to generate patchinfo
cd temp/firefox
ln -s 3.5rc2 3.0.11-3.5rc2
cd ../..

# Create partial patches and snippets
./patcher2.pl --create-patches --app=firefox --config=../patcher-configs/moz19-branch-major-update-patcher2.cfg 2>&1 | tee ../firefox-create-patches.log

Separate out beta snippets

Skipped over this because they won't be needed.

cd temp/firefox/3.0.11-3.5rc2
rsync -a aus2/ aus2.beta/
find aus2.beta/ -type d -name "release" -exec rm -rfv {} \;
find aus2/ -type d -name "beta" -exec rm -rfv {} \;

Quick verify

Check the snippets are identical for all four channels

# betatest == releasetest
find aus2.test -type d -iregex '.*betatest.*' | perl -nle '$a = $_; $a =~ s/betatest/releasetest/; system("diff -r -u $_ $a");'
# test the opposite direction to catch anything that only exists for releasetest
find aus2.test -type d -iregex '.*releasetest.*' | perl -nle '$a = $_; $a =~ s/releasetest/betatest/; system("diff -r -u $_ $a");'

Got no output for each test. Then skipped this bit

# betatest == beta
find aus2.test -type d -iregex '.*betatest.*' | perl -nle '$a = $_; $a =~ s/betatest/beta/; $a =~ s/aus2\.test/aus2.beta/; system("diff -r -u $_ $a");'
find aus2.beta -type d -iregex '.*/beta$' | perl -nle '$a = $_; $a =~ s/beta$/betatest/; $a =~ s/aus2\.beta/aus2.test/; system("diff -r -u $_ $a");'

but did this as a sanity check

# betatest == release
find aus2.test -type d -iregex '.*betatest.*' | perl -nle '$a = $_; $a =~ s/betatest/release/; $a =~ s/aus2\.test/aus2/; system("diff -r -u $_ $a");'
find aus2 -type d -iregex '.*/release$' | perl -nle '$a = $_; $a =~ s/release$/betatest/; $a =~ s/aus2/aus2.test/; system("diff -r -u $_ $a");'

Push to AUS

cd /builds/3.0.11-3.5rc2build2-major-update-test/snippets/patcher/temp/firefox/3.0.11-3.5rc2
rsync -e 'ssh -i /home/cltbld/.ssh/aus' -av aus2.test/ cltbld@aus2-staging.mozilla.org:/opt/aus2/snippets/staging/20090617-Firefox-3.0.11-3.5rc2build2-MU-test

Enable test snippets

# cltbld@aus2-staging
~/bin/backupsnip 20090617-Firefox-3.0.11-3.5rc2build2-MU-test
~/bin/pushsnip   20090617-Firefox-3.0.11-3.5rc2build2-MU-test

Update verify

Ran update verify on fx-{linux,mac,win32}-1.9-slave2 as follows:

mkdir -p /builds/verify/firefox-3011-35rc2build2-major-test/
cd /builds/verify/firefox-3011-35rc2build2-major-test/
hg clone http://hg.mozilla.org/build/tools

# get patch to trawl "Only in" directories
cd tools/release/common
# on mac & linux
curl -sL https://bugzilla.mozilla.org/attachment.cgi?id=367544 | patch -p3
# on win32
wget --no-check-certificate -O patch https://bugzilla.mozilla.org/attachment.cgi?id=367544
patch -p3 < patch

# everyone ...
cd ../updates
platform=linux   # or mac or win32
./verify.sh -c moz19-firefox-$platform-major.cfg 2>&1 | tee $platform.log

Results are posted on the bug. No changes needed.