Releases/Firefox 13.0b6/BuildNotes
Notes About Releasing
Bugs hit
Enter any bugs pre-existing or newly discovered and filed during the release:
- no L10n milestone. I took no L10n changes
- change l10n-stage-sj.mozilla.org for l10n.mozilla.org in documentation
- I had to lock few extra darwin10 slaves since there were 4 attached but only 2 ssheable to start the mac builds (xulrunner was picked up first)
- One of the mac repacks failed around Mon May 28 23:26:41 2012
remoteFailed: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other side was lost in a non-clean fashion.
- One of the mac repacls timed out on signing
- http://buildbot-master13.build.mozilla.org:8001/builders/release-mozilla-beta-macosx64_repack_4%2F6/builds/3/steps/run_script/logs/stdio
- command timed out: 2400 seconds without output, attempting to kill
- I started the release before fixing the symlinks
- Once the automation stops we can fix it by doing this:
# do not run until all deliverables and logs are uplodaed cd /pub/mozilla.org/firefox/nightly rsync -av 13.0b6-candidates/ ../candidates/13.0b6-candidates/ mv 13.0b6-candidates 13.0b6-candidates.old && ln -s ../candidates/13.0b6-candidates 13.0b6-candidates rm -rf 13.0b6-candidates.old
- check permissions failed. I assume I can fix this one I fix the candidates symlinks later on
Error while running: find /pub/mozilla.org/firefox/nightly/13.0b6-candidates/build1/ -mindepth 1 -type d ! -perm 755 ! -path '*/contrib*' ! -path '*/partner-repacks*' command: START command: ssh -l ffxbld -i /home/cltbld/.ssh/ffxbld_dsa stage.mozilla.org 'test "0" = "$(find /pub/mozilla.org/firefox/nightly/13.0b6-candidates/build1/ -maxdepth 1 -type d ! -perm 2775 -path \'*/contrib*\' | wc -l)"' command: cwd: /builds/slave/rel-m-beta-chk-prms command: output: Traceback (most recent call last): File "/builds/slave/rel-m-beta-chk-prms/scripts/scripts/release/push-to-mirrors.py", line 176, in <module> buildNumber=buildNumber) File "/builds/slave/rel-m-beta-chk-prms/scripts/scripts/release/push-to-mirrors.py", line 84, in checkStagePermissions username=stageUsername, sshKey=stageSshKey) File "/builds/slave/rel-m-beta-chk-prms/scripts/scripts/release/../../lib/python/util/commands.py", line 57, in run_remote_cmd run_cmd(cmd_prefix + cmd, **kwargs) File "/builds/slave/rel-m-beta-chk-prms/scripts/scripts/release/../../lib/python/util/commands.py", line 42, in run_cmd return subprocess.check_call(cmd, **kwargs) File "/tools/python-2.5.1/lib/python2.5/subprocess.py", line 461, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['ssh', '-l', 'ffxbld', '-i', '/home/cltbld/.ssh/ffxbld_dsa', 'stage.mozilla.org', 'test "0" = "$(find /pub/mozilla.org/firefox/nightly/13.0b6-candidates/build1/ -mindepth 1 -type d ! -perm 755 ! -path \'*/contrib*\' ! -path \'*/partner-repacks*\' | wc -l)"']' returned non-zero exit status 1 command: END (0.18s elapsed)
Build Engineers
armenzg - Tracking bug: bug 744595
Signed-off Revision(s)
Build 1: 046e86aee0c1
L10N changesets
Instructions on how to get them
- Firefox: no milestone. I took no L10n changes
Tags
Manually tag the automation code, then record the generated tags below. (details) XXX: Verify the branch name
Build # | Branch, Tags | Changeset |
GECKO130_#_RELBRANCH, FIREFOX_13_0b6_BUILD1 FIREFOX_13_0b6_RELEASE | 046e86aee0c1 |
Build data
Firefox
XXX: Fill this data
Build # | Type | Build ID | Build machine | Time to build |
1 | Linux | linux-ix-slave28 | 1 hrs, 6 mins, 14 secs | |
Linux64 | linux64-ix-slave21 | 1 hrs, 5 mins, 46 secs | ||
Mac | moz2-darwin10-slave42 | 3 hrs, 18 mins, 57 secs | ||
Windows | w32-ix-slave37 | 3 hrs, 45 mins, 30 secs |
Notes
Build 1
Preparing to start Automation
- Set clobbers for the appropriate masters.
- clobberer
- I didn't do it in advance
- Reserve slaves (you need ~5 each of linux, linux64, win32 (w32), and macosx64 (moz2-darwin10)). Doing this 2-3 hours in advance should speed things up. e.g.:
[cltbld@buildbot-master13 master]$ echo 8 > /builds/buildbot/build1/master/reserved_slaves
- For Firefox build1 only, create a symlink for your $version (eg 11.0b4, 10.0.2esr, 10.0.2, 3.6.27) (until bug 725839 is resolved)
# ffxbld@stage cd /pub/mozilla.org/firefox/nightly/ mkdir ../candidates/13.0b6-candidates ln -s ../candidates/13.0b6-candidates 13.0b6-candidates
- Update l10n changesets for desktop
- Instructions on how to get them
- dashboard
- Land automation configs
- Tag buildbot-configs, buildbotcustom, & tools with build & release tags. (details)
- Sample command for Firefox 10.0 beta 5 build 1:
hg tag -f FIREFOX_13_0b6_RELEASE FIREFOX_13_0b6_BUILD1
- update and reconfigure the master
cd /builds/buildbot/build1/master source ../bin/activate PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u armenzg -V 13.0b6 --branch mozilla-beta --build-number 1 \ --release-config release-firefox-mozilla-beta.py --products firefox \ --dryrun localhost:9001
- start automation ← monitor progress on buildbot (e.g. beta)
- the same command as above but without the "--dryrun"
- I had to run with -b to skip checking for the L10n dashboard
E-mail Metrics
Use the address "metrics-alerts < AT > mozilla < PERIOD > org". Note for first-time-releasers: your email will get held for moderator approval - that is expected. If it happens more than once, get help on #metrics.
Firefox,firefox,firefox,13.0b6,13.0
Tag
No problems.
Bouncer Submitter
No problems.
Source
No problems.
Build
Firefox
No problems.
Firefox repacks
We had a couple of issues with the mac repacks. I re-triggered them.
Unittests / Talos
- Look to see that they ran
- Document any oranges (per platform) for unittests - if possible, try to confirm it's known/random
- Make sure there's no red/failures that we need to have a dev look at
XULRunner
No problems.
Source
No problems.
Build
No problems.
Checksums
No problems.
Updates
- updates generation worked
Update verify
XXX: Running DONE TO HERE
Reset reserved slaves
XXX: TODO
Check permissions / AV scan
It is supposed to be triggered automatically after updates are done.
- check permissions failed. I assume I can fix this one I fix the candidates symlinks later on
Going to Beta
NOTE: Skip to "Before Release Day" for beta releases
Run backupsnip
NOTE: Remember to do this at least 2 hours ahead of the expected "go to beta" email. details
Remember to ssh to aus3-staging using your short LDAP name and then run sudo su - ffxbld
once on. If you intend to use screen, initiate your screen session while you are still yourself.
Push to beta
Wait for "go" from release managers
For Firefox:
- Publish (and record run time here): details
- on completion, email r-d that VERSION updates are live on the beta channel
Wait for confirmation from QA that each of the above releases is okay, then close the tracking ticket.