Releases/Firefox 29.0/BuildNotes: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
 
(17 intermediate revisions by 2 users not shown)
Line 6: Line 6:


== Updates for beta channel ==
== Updates for beta channel ==
<pre>
  # using bld-centos6-hp-016 with screen
  # using bld-centos6-hp-016 with screen
  mkdir ~/bug978746
  mkdir ~/bug978746
Line 13: Line 14:
  export PYTHONPATH=tools/lib/python; export PATH=$PATH:`pwd`
  export PYTHONPATH=tools/lib/python; export PATH=$PATH:`pwd`
  wget -O repack.py https://bugzilla.mozilla.org/attachment.cgi?id=8389037
  wget -O repack.py https://bugzilla.mozilla.org/attachment.cgi?id=8389037
# Edit repack.py to comment out call to writeSettingsFile in the partial update generation section


  # make sure enough disk space is free
  # make sure enough disk space is free
Line 20: Line 23:
  cd ~/bug978746
  cd ~/bug978746
  export PYTHONPATH=tools/lib/python; export PATH=$PATH:`pwd`
  export PYTHONPATH=tools/lib/python; export PATH=$PATH:`pwd`
  time python -u repack.py -f 29.0b9-build1 -t 29.0-build1 -T 29.0b9-build1 -P linux-i686 2>&1 | tee linux.log
  time python -u repack.py -f 29.0b9-build1 -t 29.0-build1 -T 29.0b9-build1 -P linux-i686 --no-completes 2>&1 | tee linux.log
  # repeat for these:
  # repeat for these:
  time python -u repack.py -f 29.0b9-build1 -t 29.0-build1 -T 29.0b9-build1 -P linux-x86_64 --no-env 2>&1 | tee linux64.log
  time python -u repack.py -f 29.0b9-build1 -t 29.0-build1 -T 29.0b9-build1 -P linux-x86_64 --no-env --no-completes 2>&1 | tee linux64.log
  time python -u repack.py -f 29.0b9-build1 -t 29.0-build1 -T 29.0b9-build1 -P win32 --no-env 2>&1 | tee win32.log
  time python -u repack.py -f 29.0b9-build1 -t 29.0-build1 -T 29.0b9-build1 -P win32 --no-env --no-completes 2>&1 | tee win32.log
  time python -u repack.py -f 29.0b9-build1 -t 29.0-build1 -T 29.0b9-build1 -P mac --no-env 2>&1 | tee mac.log
  time python -u repack.py -f 29.0b9-build1 -t 29.0-build1 -T 29.0b9-build1 -P mac --no-env --no-completes 2>&1 | tee mac.log
</pre>
 
<pre>
# fix permissions and upload to stage
find repacked/ -type f -exec chmod 644 {} \;
find repacked/ -type d -exec chmod 755 {} \;
# rsync -n is your friend
rsync -av -e 'ssh -i /home/cltbld/.ssh/ffxbld_dsa' \
  repacked/29.0-candidates/build1/ \
  ffxbld@stage.mozilla.org:/pub/mozilla.org/firefox/candidates/29.0-candidates/build1/
# reran check_permissions builder to make sure all is correct.
 
# run av scan on partials, anyone on stage
cd /pub/mozilla.org/firefox/candidates/29.0-candidates/build1/update
nice ionice -c2 -n7 extract_and_run_command.py -j2 clamdscan -m --no-summary -- \
  `find . -name firefox-29.0b9-29.0.partial.mar` 2>&1 | tee /tmp/av.log
</pre>
 
<pre>
# Generate patcher configs and update verify configs
cd <parent_of_tools_checkout>
export PERL5LIB="tools/lib/perl:$HOME/perl5"  # need Config::General v2.44
wget http://hg.mozilla.org/releases/mozilla-release/raw-file/FIREFOX_29_0_RELEASE/browser/locales/shipped-locales
perl tools/release/patcher-config-bump.pl -p firefox -r Firefox -v 29.0 -a 29.0 -o 29.0b9 -b 1 \
  -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg -t stage.mozilla.org -f ftp.mozilla.org \
  -d download.mozilla.org -l shipped-locales --partial-version 29.0b9 --platform linux --platform linux64 \
  --platform macosx64 --platform win32
# then some manual modifications <29.0> block ——> <29.0-build1>, to -> 29.0-build1, add mar-channel-ids, pretty version
 
# generate update verify configs
python tools/scripts/updates/create-update-verify-configs.py -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg --platform linux64 --output tools/release/updates/mozBeta-firefox-linux64.cfg --release-config-file mozilla/release-firefox-mozilla-release.py -b https://hg.mozilla.org/build/buildbot-configs --channel releasetest -t FIREFOX_29_0_RELEASE
python tools/scripts/updates/create-update-verify-configs.py -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg --platform linux --output tools/release/updates/mozBeta-firefox-linux.cfg --release-config-file mozilla/release-firefox-mozilla-release.py -b https://hg.mozilla.org/build/buildbot-configs --channel releasetest -t FIREFOX_29_0_RELEASE
python tools/scripts/updates/create-update-verify-configs.py -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg --platform macosx64 --output tools/release/updates/mozBeta-firefox-mac64.cfg --release-config-file mozilla/release-firefox-mozilla-release.py -b https://hg.mozilla.org/build/buildbot-configs --channel releasetest -t FIREFOX_29_0_RELEASE
python tools/scripts/updates/create-update-verify-configs.py -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg --platform win32 --output tools/release/updates/mozBeta-firefox-win32.cfg --release-config-file mozilla/release-firefox-mozilla-release.py -b https://hg.mozilla.org/build/buildbot-configs --channel releasetest -t FIREFOX_29_0_RELEASE
 
# hand edit to make the following changes:
#  - fix "to" variable to point at valid candidates directory
</pre>
 
<pre>
# generate snippets (on bld-centos6-hp-016):
cd /home/cltbld/bug978746/tools
hg pull
hg up -r FIREFOX_29_0_RELEASE_RUNTIME
wget -O- https://bugzilla.mozilla.org/attachment.cgi?id=8410637 | patch -p1
cd ..
export PYTHONPATH="tools/lib/python:tools/lib/python/vendor"
python tools/scripts/updates/create-snippets.py --config tools/release/patcher-configs/mozBeta-branch-patcher2.cfg \
  --checksums-dir checksums --snippet-dir aus2 --test-snippet-dir aus2.test -v 2>&1 | tee snippets.log
</pre>
 
<pre>
# ditch the betatest snippets, we're going to use releasetest only before pushing to beta
rm -rf aus2.test/Firefox/*/WIN*/*/*/betatest
rm -rf aus2.test/Firefox/*/Lin*/*/*/betatest
rm -rf aus2.test/Firefox/*/Dar*/*/*/betatest
</pre>
 
<pre>
# Sync to aus3-staging
rsync -av -e 'ssh -i /home/cltbld/.ssh/ffxbld_dsa' aus2.test/ \
  ffxbld@aus3-staging.mozilla.org:/opt/aus2/snippets/staging/Firefox-29.0-build1-forbeta-test/
rsync -av -e 'ssh -i /home/cltbld/.ssh/ffxbld_dsa' aus2/ \
  ffxbld@aus3-staging.mozilla.org:/opt/aus2/snippets/staging/Firefox-29.0-build1-forbeta/
</pre>
 
<pre>
# Bouncer setup (to match patcher config)
# add Products: Firefox-29.0-Partial-29.0b9, Firefox-29.0-build1-complete
# add Locations: 4 locations for each (1 per OS)
</pre>
 
<pre>
# push test snippets live, on aus3-staging
~/bin/pushsnip Firefox-29.0-build1-forbeta-test
</pre>
 
<pre>
# Update verify
# All platforms except linux32:
 
mkdir bug978746 ; cd $_
hg clone http://hg.mozilla.org/build/tools
cd tools
hg up -r FIREFOX_29_0_RELEASE_RUNTIME
wget -O- https://bugzilla.mozilla.org/attachment.cgi?id=8410637 | patch -p1
cd release/updates
# use correct .cfg file for platform
./verify.sh -c mozBeta-firefox-linux64.cfg 2>&1 | tee log
</pre>
 
For posterity, both linux update verifies were run on bld-centos6-hp-016 (64-bit outside of mock, 32-bit inside of mock). bld-lion-r5-005 was used for Mac. w64-ix-slave21 was used for Windows.
 
<pre>
# push beta snippets live, on aus3-staging
~/bin/pushsnip Firefox-29.0-build1-forbeta
</pre>
 
== Funnelcake ==
We're running two funnelcake experiments as part of Firefox 29: funnelcake 30,31 ({{bug|976735}})
 
Funnelcake repacks were created as part of release automation, with the exception of linux64, which wasn't initially requested. Rail created and uploaded the linux64 repacks manually.
 
I created bouncer entries for the funnelcake installers:
 
* https://bounceradmin.mozilla.com/admin/mirror/product/3148/ (Firefox-29.0-SSL-f30)
* https://bounceradmin.mozilla.com/admin/mirror/product/3149/ (Firefox-29.0-SSL-f31)
 
We also needed stub installers for windows. I created those on w64-ix-slave136 (which did the release builds), following the instructions https://bug933847.bugzilla.mozilla.org/attachment.cgi?id=8338878. I used https://bug976735.bugzilla.mozilla.org/attachment.cgi?id=8411340 to patch the stub installer files to include the funnelcake identifiers. The stubs are available in the partner repack directories beside the full installers.
 
I also created bouncer entries for the stub installers:
 
* https://bounceradmin.mozilla.com/admin/mirror/product/3152/ (Firefox-stub-f30)
* https://bounceradmin.mozilla.com/admin/mirror/product/3153/ (Firefox-stub-f31)
 
As well as entries for the products that the stub requests:
 
* https://bounceradmin.mozilla.com/admin/mirror/product/3150/ (Firefox-latest-f30)
* https://bounceradmin.mozilla.com/admin/mirror/product/3151/ (Firefox-latest-f31)
 
These last two must not have the SSL box checked.
 
=== Push live ===
# ffxbld@stage
rsync -av /pub/mozilla.org/firefox/candidates/29.0-candidates/build1/partner-repacks/funnelcake30/ \
            /pub/mozilla.org/firefox/releases/29.0-funnelcake30/
rsync -av /pub/mozilla.org/firefox/candidates/29.0-candidates/build1/partner-repacks/funnelcake31/ \
            /pub/mozilla.org/firefox/releases/29.0-funnelcake31/
for d in `find /pub/mozilla.org/firefox/releases/29.0-funnelcake3{0,1} -type d`; do
    cp /pub/mozilla.org/firefox/releases/29.0/index.html $d
done
Then update all the bouncer links above to point to new location.
 
 
== Release day ==
Manually updated bouncer links for "firefox-stub" and "firefox-latest" since QA signoff was delayed.
 
Zeus caching of AUS made it a bit more difficult to test the updates after unthrottling.

Latest revision as of 14:03, 29 April 2014

bug 978746 - catlee

Build 1

Issues hit:

  • Automation went smoothly for desktop

Updates for beta channel

 # using bld-centos6-hp-016 with screen
 mkdir ~/bug978746
 cd ~/bug978746
 hg clone http://hg.mozilla.org/build/tools
 # get a signing token for release certs, 6 hour expiry
 export PYTHONPATH=tools/lib/python; export PATH=$PATH:`pwd`
 wget -O repack.py https://bugzilla.mozilla.org/attachment.cgi?id=8389037

 # Edit repack.py to comment out call to writeSettingsFile in the partial update generation section

 # make sure enough disk space is free
 python ~/bug978746/tools/buildfarm/maintenance/purge_builds.py -s 55 -n 'rel-*:45d' -n 'tb-rel-*:45d' /builds/slave

 # in separate screen windows, generate other platforms
 cd ~/bug978746
 export PYTHONPATH=tools/lib/python; export PATH=$PATH:`pwd`
 time python -u repack.py -f 29.0b9-build1 -t 29.0-build1 -T 29.0b9-build1 -P linux-i686 --no-completes 2>&1 | tee linux.log
 # repeat for these:
 time python -u repack.py -f 29.0b9-build1 -t 29.0-build1 -T 29.0b9-build1 -P linux-x86_64 --no-env --no-completes 2>&1 | tee linux64.log
 time python -u repack.py -f 29.0b9-build1 -t 29.0-build1 -T 29.0b9-build1 -P win32 --no-env --no-completes 2>&1 | tee win32.log
 time python -u repack.py -f 29.0b9-build1 -t 29.0-build1 -T 29.0b9-build1 -P mac --no-env --no-completes 2>&1 | tee mac.log

# fix permissions and upload to stage
find repacked/ -type f -exec chmod 644 {} \;
find repacked/ -type d -exec chmod 755 {} \;
# rsync -n is your friend
rsync -av -e 'ssh -i /home/cltbld/.ssh/ffxbld_dsa' \
  repacked/29.0-candidates/build1/ \
  ffxbld@stage.mozilla.org:/pub/mozilla.org/firefox/candidates/29.0-candidates/build1/
# reran check_permissions builder to make sure all is correct.

# run av scan on partials, anyone on stage
cd /pub/mozilla.org/firefox/candidates/29.0-candidates/build1/update
nice ionice -c2 -n7 extract_and_run_command.py -j2 clamdscan -m --no-summary -- \
  `find . -name firefox-29.0b9-29.0.partial.mar` 2>&1 | tee /tmp/av.log

# Generate patcher configs and update verify configs
cd <parent_of_tools_checkout>
export PERL5LIB="tools/lib/perl:$HOME/perl5"  # need Config::General v2.44
wget http://hg.mozilla.org/releases/mozilla-release/raw-file/FIREFOX_29_0_RELEASE/browser/locales/shipped-locales
perl tools/release/patcher-config-bump.pl -p firefox -r Firefox -v 29.0 -a 29.0 -o 29.0b9 -b 1 \
  -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg -t stage.mozilla.org -f ftp.mozilla.org \
  -d download.mozilla.org -l shipped-locales --partial-version 29.0b9 --platform linux --platform linux64 \
  --platform macosx64 --platform win32
# then some manual modifications <29.0> block ——> <29.0-build1>, to -> 29.0-build1, add mar-channel-ids, pretty version

# generate update verify configs
python tools/scripts/updates/create-update-verify-configs.py -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg --platform linux64 --output tools/release/updates/mozBeta-firefox-linux64.cfg --release-config-file mozilla/release-firefox-mozilla-release.py -b https://hg.mozilla.org/build/buildbot-configs --channel releasetest -t FIREFOX_29_0_RELEASE
python tools/scripts/updates/create-update-verify-configs.py -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg --platform linux --output tools/release/updates/mozBeta-firefox-linux.cfg --release-config-file mozilla/release-firefox-mozilla-release.py -b https://hg.mozilla.org/build/buildbot-configs --channel releasetest -t FIREFOX_29_0_RELEASE
python tools/scripts/updates/create-update-verify-configs.py -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg --platform macosx64 --output tools/release/updates/mozBeta-firefox-mac64.cfg --release-config-file mozilla/release-firefox-mozilla-release.py -b https://hg.mozilla.org/build/buildbot-configs --channel releasetest -t FIREFOX_29_0_RELEASE
python tools/scripts/updates/create-update-verify-configs.py -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg --platform win32 --output tools/release/updates/mozBeta-firefox-win32.cfg --release-config-file mozilla/release-firefox-mozilla-release.py -b https://hg.mozilla.org/build/buildbot-configs --channel releasetest -t FIREFOX_29_0_RELEASE

# hand edit to make the following changes:
#  - fix "to" variable to point at valid candidates directory

# generate snippets (on bld-centos6-hp-016):
cd /home/cltbld/bug978746/tools
hg pull
hg up -r FIREFOX_29_0_RELEASE_RUNTIME
wget -O- https://bugzilla.mozilla.org/attachment.cgi?id=8410637 | patch -p1 
cd ..
export PYTHONPATH="tools/lib/python:tools/lib/python/vendor"
python tools/scripts/updates/create-snippets.py --config tools/release/patcher-configs/mozBeta-branch-patcher2.cfg \
   --checksums-dir checksums --snippet-dir aus2 --test-snippet-dir aus2.test -v 2>&1 | tee snippets.log

# ditch the betatest snippets, we're going to use releasetest only before pushing to beta
rm -rf aus2.test/Firefox/*/WIN*/*/*/betatest
rm -rf aus2.test/Firefox/*/Lin*/*/*/betatest
rm -rf aus2.test/Firefox/*/Dar*/*/*/betatest

# Sync to aus3-staging 
rsync -av -e 'ssh -i /home/cltbld/.ssh/ffxbld_dsa' aus2.test/ \
  ffxbld@aus3-staging.mozilla.org:/opt/aus2/snippets/staging/Firefox-29.0-build1-forbeta-test/
rsync -av -e 'ssh -i /home/cltbld/.ssh/ffxbld_dsa' aus2/ \
  ffxbld@aus3-staging.mozilla.org:/opt/aus2/snippets/staging/Firefox-29.0-build1-forbeta/

# Bouncer setup (to match patcher config)
# add Products: Firefox-29.0-Partial-29.0b9, Firefox-29.0-build1-complete
# add Locations: 4 locations for each (1 per OS)

# push test snippets live, on aus3-staging
~/bin/pushsnip Firefox-29.0-build1-forbeta-test

# Update verify
# All platforms except linux32:

mkdir bug978746 ; cd $_
hg clone http://hg.mozilla.org/build/tools
cd tools
hg up -r FIREFOX_29_0_RELEASE_RUNTIME
wget -O- https://bugzilla.mozilla.org/attachment.cgi?id=8410637 | patch -p1 
cd release/updates
# use correct .cfg file for platform
./verify.sh -c mozBeta-firefox-linux64.cfg 2>&1 | tee log

For posterity, both linux update verifies were run on bld-centos6-hp-016 (64-bit outside of mock, 32-bit inside of mock). bld-lion-r5-005 was used for Mac. w64-ix-slave21 was used for Windows. 

# push beta snippets live, on aus3-staging
~/bin/pushsnip Firefox-29.0-build1-forbeta

Funnelcake

We're running two funnelcake experiments as part of Firefox 29: funnelcake 30,31 (bug 976735)

Funnelcake repacks were created as part of release automation, with the exception of linux64, which wasn't initially requested. Rail created and uploaded the linux64 repacks manually.

I created bouncer entries for the funnelcake installers:

We also needed stub installers for windows. I created those on w64-ix-slave136 (which did the release builds), following the instructions https://bug933847.bugzilla.mozilla.org/attachment.cgi?id=8338878. I used https://bug976735.bugzilla.mozilla.org/attachment.cgi?id=8411340 to patch the stub installer files to include the funnelcake identifiers. The stubs are available in the partner repack directories beside the full installers.

I also created bouncer entries for the stub installers:

As well as entries for the products that the stub requests:

These last two must not have the SSL box checked.

Push live

# ffxbld@stage
rsync -av /pub/mozilla.org/firefox/candidates/29.0-candidates/build1/partner-repacks/funnelcake30/ \
           /pub/mozilla.org/firefox/releases/29.0-funnelcake30/
rsync -av /pub/mozilla.org/firefox/candidates/29.0-candidates/build1/partner-repacks/funnelcake31/ \
           /pub/mozilla.org/firefox/releases/29.0-funnelcake31/
for d in `find /pub/mozilla.org/firefox/releases/29.0-funnelcake3{0,1} -type d`; do
    cp /pub/mozilla.org/firefox/releases/29.0/index.html $d
done

Then update all the bouncer links above to point to new location.


Release day

Manually updated bouncer links for "firefox-stub" and "firefox-latest" since QA signoff was delayed.

Zeus caching of AUS made it a bit more difficult to test the updates after unthrottling.

Retrieved from "https://wiki.mozilla.org/index.php?title=Releases/Firefox_29.0/BuildNotes&oldid=970712"