Releases/Firefox 12.0/BuildNotes

From MozillaWiki
Jump to: navigation, search

Notes About Releasing

Please update the Notes Template and the Release:Primer for future releases (bug fixes, changes to automation) as needed

Bugs hit

Build Engineers

{bhearsum} - Tracking bug: bug 735392

Signed-off Revision(s)

Build 1: 5bcfa0da3be9

L10N changesets

Instructions on how to get them

Tags

Manually tag the automation code, then record the generated tags below. (details)

Build # Branch, Tags Changeset
1 GECKO120_2012042014_RELBRANCH, FIREFOX_12_0_BUILD1 FIREFOX_12_0_RELEASE 5bcfa0da3be9

Build data

Firefox

Build # Type Build ID Build machine Time to build
1 Linux 20120420145725 mv-moz2-linux-ix-slave06 1 hrs, 5 mins, 5 secs
Linux64 moz2-linux64-slave08 4 hrs, 9 mins, 28 secs
Mac moz2-darwin10-slave50 3 hrs, 17 mins, 18 secs
Windows mw32-ix-slave13 3 hrs, 27 mins, 47 secs

Notes

Build 1

Diff mozconfigs

From a mozilla-release checkout, ran: 

for p in linux32 linux64 macosx-universal win32; do diff -Naur {,../mozilla-beta/}browser/config/mozconfigs/$p/release; done

Got the following output: 

--- browser/config/mozconfigs/linux32/release	2012-04-19 13:09:08.317718465 -0400
+++ ../mozilla-beta/browser/config/mozconfigs/linux32/release	2012-04-19 13:08:35.781619562 -0400
@@ -16,5 +16,8 @@
 
 export MOZ_TELEMETRY_REPORTING=1
 
+# Treat warnings as errors in directories with FAIL_ON_WARNINGS.
+ac_add_options --enable-warnings-as-errors
+
 # Enable parallel compiling
 mk_add_options MOZ_MAKE_FLAGS="-j4"
--- browser/config/mozconfigs/linux64/release	2012-04-19 13:09:08.321718478 -0400
+++ ../mozilla-beta/browser/config/mozconfigs/linux64/release	2012-04-19 13:08:35.781619562 -0400
@@ -16,5 +16,8 @@
 
 export MOZ_TELEMETRY_REPORTING=1
 
+# Treat warnings as errors in directories with FAIL_ON_WARNINGS.
+ac_add_options --enable-warnings-as-errors
+
 # Enable parallel compiling
 mk_add_options MOZ_MAKE_FLAGS="-j4"
--- browser/config/mozconfigs/macosx-universal/release	2012-01-30 08:45:53.000000000 -0500
+++ ../mozilla-beta/browser/config/mozconfigs/macosx-universal/release	2012-04-19 13:08:35.785619573 -0400
@@ -12,5 +12,8 @@
 
 export MOZ_TELEMETRY_REPORTING=1
 
+# Treat warnings as errors in directories with FAIL_ON_WARNINGS.
+ac_add_options --enable-warnings-as-errors
+
 # Enable parallel compiling
 mk_add_options MOZ_MAKE_FLAGS="-j4"

Not a critical change, but it'll get picked up when Release Management does the merge, so didn't take any action.

Preparing to start Automation

  • Requested a clobber for "any master", "mozilla-release", "any builder"
  • Reserved 8 slaves
  • Created a candidates directory symlink:
# ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/
mkdir ../candidates/12.0-candidates
ln -s ../candidates/12.0-candidates 12.0-candidates
  • Clicked "ship it" on l10n dashboard
  • Landed buildbot-configs changes
  • Tagged buildbot-configs, buildbotcustom, and tools with FIREFOX_12_0_{RELEASE,BUILD1}
  • Ran release sanity:
source ../bin/activate
PYTHONPATH=. ../bin/python ../tools/buildbot-helpers/release_sanity.py -u bhearsum \
  -V 12.0 --branch mozilla-release --build-number 1 -c release-firefox-mozilla-release.py \
  --products firefox --l10n-dashboad-version=12 --bypass-l10n-check --dryrun localhost:9001
PYTHONPATH=. ../bin/python ../tools/buildbot-helpers/release_sanity.py -u bhearsum \
  -V 12.0 --branch mozilla-release --build-number 1 -c release-firefox-mozilla-release.py \
  --products firefox --l10n-dashboad-version=12 --bypass-l10n-check localhost:9001

E-mail Metrics

Sent: 

Firefox,firefox,firefox,12.0,12.0

E-mail Mirrors

Emailed mirrors@mozilla.org to warn of Firefox and Thunderbird 12.0

Edit rsync exclude files

Added '- firefox/releases/12.0' to /pub/mozilla.org/rsyncd-mozilla-releases.exclude

Tag

No problems. However, I thought I had messed up the config so I halted tagging at one point, and to restart the automation. We thought we would have to muck around with the schedulers to fix it, but that ended up not being the case.

Bouncer Submitter

No problems.

Source

No problems

Build

No problems

Repacks

Linux repack 1/6 failed part way through, with an hg timeout. The only locales left in that chunk were cs, csb, and cy, so I used the standalone builder to build them. The form was filled out as follows:

  • branch: release/mozilla-release
  • revision: FIREFOX_12_0_RELEASE
  • properties:
    • locale: cs:csb:cy
    • release_config: mozilla/release-firefox-mozilla-release.py
    • script_repo_revision: FIREFOX_12_0_RELEASE

Once that was done, "force build" to manually trigger the linux64_repack_complete builder.

Unittests / Talos

Didn't look at them.

XULRunner

Source

Failed. Rebuilt it.

Build

No problems

Partner Repack

No problems.

L10N verify

No problems, all green.

Checksums

No problems.

Updates

Pushsnip failed with: 

retry: Calling <function run_with_timeout at 0xb7d3f0d4> with args: (['bash', '-c', 'ssh -t -l ffxbld -i ~/.ssh/auspush aus3-staging.mozilla.org ~/bin/pushsnip Firefox-12.0-build1-test'], 7260, None, None, False, True), kwargs: {}, attempt #1
Executing: ['bash', '-c', 'ssh -t -l ffxbld -i ~/.ssh/auspush aus3-staging.mozilla.org ~/bin/pushsnip Firefox-12.0-build1-test']
Pseudo-terminal will not be allocated because stdin is not a terminal.
bash: /home/cltbld/bin/pushsnip: No such file or directory

To finish up this builder, pushed them by hand with: 

# ffxbld@aus3-staging
~/bin/pushsnip Firefox-12.0-build1-test

And had to upload release channel snippets by hand: 

# cltbld@mv-moz2-linux-ix-slave17
cd /builds/slave/rel-m-rel-updates/build/temp/firefox/11.0-12.0
rsync -av -e 'ssh -oIdentityFile=~/.ssh/auspush' aus2/ ffxbld@aus3-staging.mozilla.org:/opt/aus2/snippets/staging/Firefox-12.0-build1

Send mail announcing updates available on betatest.

Update verify

Used "force build" to start because of updates failure. One chunk failed with: 

diff: target/*.app: No such file or directory
FAIL: unknown error from diff: 2
FAIL: check_updates returned unknown error for Darwin_x86_64-gcc3-u-i386-x86_64 downloads/Firefox 11.0.dmg vs. downloads/Firefox 12.0.dmg: 3

Glanced at the mac pl 12.0 build, which looked fine. Rekicked the chunk to see if the error is transient.

Major Update

Used "force build" to start. Pushsnip failed with the same error as above. Did the following to kick it through: 

# ffxbld@aus3-staging
cd /opt/aus2/snippets/staging

# cltbld@mv-moz2-linux-ix-slave20
cd /builds/slave/rel-m-192-mu/build/temp/firefox/3.6.28-12.0
rsync -av -e 'ssh -oIdentityFile=~/.ssh/auspush' aus2.beta/ ffxbld@aus3-staging.mozilla.org:/opt/aus2/snippets/staging/Firefox-3.6.28-build1-12.0-build1-MU-beta
rsync -av -e 'ssh -oIdentityFile=~/.ssh/auspush' aus2/ ffxbld@aus3-staging.mozilla.org:/opt/aus2/snippets/staging/Firefox-3.6.28-build1-12.0-build1-MU

Announced "updates available" manually.

Update verify

Started with "force build". All green.

Reset reserved slaves

Reset to 0.

Check permissions / AV scan

No permissions problems or viruses.

Push to internal mirrors

Used "force build" to start (normally would happen after updates finish successfully).

Final verification

Used "force build" to start. Should've triggered automatically, filed bug 747892 to fix.

All green.

Before Release Day

Run backupsnip for release snippets

# ffxbld@aus3-staging
cd /opt/aus2/snippets/staging
~/bin/backupsnip Firefox-12.0-build1

Push index file to mirrors

Added the following lines to the top of stage:/pub/mozilla.org/zz/rsyncd-mozilla-releases.exclude: 

+ firefox
+ firefox/releases
+ firefox/releases/12.0
+ firefox/releases/12.0/*/
+ firefox/releases/12.0/*/*/
+ firefox/releases/12.0/*/*/*/
+ firefox/releases/12.0/**index.html
- firefox/releases/12.0/**

Removed the previously added "- firefox/releases/12.0" line. After that, ran the following to lay out the index files: 

# ffxbld@stage
cd /pub/mozilla.org/firefox/releases/12.0
wget --no-check-certificate -O index.html https://bugzilla.mozilla.org/attachment.cgi?id=540139
sed -i -e 's/5/12/g' index.html 
for dir in `find . -mindepth 1 -type d `; do cp -pv index.html $dir/; done

Push to external mirrors

Removed '12.0' exclusion lines from the mozilla-releases exclude file. Updated the mozilla-current exclude file to remove Firefox 3.6.28 and add 12.0. Used "force build" to kick off the "start uptake monitoring" builder, because it hadn't started automatically, because of the "updates" failure.

Release Day

Push XULRunner to Mirrors

Update XULRunner wiki page

Updated:

Land throttling change

Verified with: https://aus3.mozilla.org/update/1/Firefox/3.6.28/20120306064154/Darwin_x86-gcc3-u-ppc-i386/af/release/update.xml?force=1 - does return an update https://aus3.mozilla.org/update/1/Firefox/3.6.28/20120306064154/Darwin_x86-gcc3-u-ppc-i386/af/release/update.xml - doesn't return an update

Push to Release Channel

# ffxbld@aus3-staging
cd /opt/aus2/snippets/staging
~/bin/pushsnip Firefox-12.0-build1

Update symlinks

Got rid of pointless 'latest-11.0' symlink at the same time: 

# ffxbld@stage
cd /pub/mozilla.org/firefox/releases
rm latest latest-11.0 && ln -s 12.0 latest

Remove index.html files

# ffxbld@stage
cd /pub/mozilla.org/firefox/releases
find . -iname index.html -exec rm {} \;

Update browser choice website

Filed bug 748367.

Ship 3.6.28 -> 12.0 Major Update

# ffxbld@aus3-staging
cd /opt/aus2/snippets/staging
~/bin/backupsnip Firefox-3.6.28-build1-12.0-build1-MU
~/bin/pushsnip Firefox-3.6.28-build1-12.0-build1-MU
Retrieved from "https://wiki.mozilla.org/index.php?title=Releases/Firefox_12.0/BuildNotes&oldid=423812"