Releases/Firefox 13.0b6/BuildNotes: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
 
(11 intermediate revisions by 2 users not shown)
Line 4: Line 4:
* no L10n milestone. I took no L10n changes
* no L10n milestone. I took no L10n changes
* change l10n-stage-sj.mozilla.org for l10n.mozilla.org in [https://wiki.mozilla.org/index.php?title=Release%3ARelease_Automation_on_Mercurial%3ADocumentation&action=historysubmit&diff=434857&oldid=429986 documentation]
* change l10n-stage-sj.mozilla.org for l10n.mozilla.org in [https://wiki.mozilla.org/index.php?title=Release%3ARelease_Automation_on_Mercurial%3ADocumentation&action=historysubmit&diff=434857&oldid=429986 documentation]
* I had to lock few extra darwin10 slaves since there were 4 attached but only 2 ssheable to start the mac builds (xulrunner was picked up first)
* I had to lock few extra darwin10 slaves since there were 4 attached but only 2 ssh-able to start the mac builds (xulrunner was picked up first)
* One of the mac repacks failed around Mon May 28 23:26:41 2012
* One of the mac repacks failed around Mon May 28 23:26:41 2012
  remoteFailed: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other side was lost in a non-clean fashion.
  remoteFailed: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other side was lost in a non-clean fashion.
* One of the mac repacls timed out on signing
* One of the mac repacks timed out on signing
** http://buildbot-master13.build.mozilla.org:8001/builders/release-mozilla-beta-macosx64_repack_4%2F6/builds/3/steps/run_script/logs/stdio
** http://buildbot-master13.build.mozilla.org:8001/builders/release-mozilla-beta-macosx64_repack_4%2F6/builds/3/steps/run_script/logs/stdio
** command timed out: 2400 seconds without output, attempting to kill
** command timed out: 2400 seconds without output, attempting to kill
* I started the release before fixing the symlinks
** Once the automation stops we can fix it by doing this:
<pre>
# do not run until all deliverables and logs are uploaded
cd /pub/mozilla.org/firefox/nightly
rsync -av 13.0b6-candidates/ ../candidates/13.0b6-candidates/
mv 13.0b6-candidates 13.0b6-candidates.old && ln -s ../candidates/13.0b6-candidates 13.0b6-candidates
rm -rf 13.0b6-candidates.old
</pre>
* check permissions failed. I assume I can fix this one I fix the candidates symlinks later on
<pre>
Error while running: find /pub/mozilla.org/firefox/nightly/13.0b6-candidates/build1/ -mindepth 1 -type d ! -perm 755 ! -path '*/contrib*' ! -path '*/partner-repacks*'
command: START
command: ssh -l ffxbld -i /home/cltbld/.ssh/ffxbld_dsa stage.mozilla.org 'test "0" = "$(find /pub/mozilla.org/firefox/nightly/13.0b6-candidates/build1/ -maxdepth 1 -type d ! -perm 2775 -path \'*/contrib*\' | wc -l)"'
command: cwd: /builds/slave/rel-m-beta-chk-prms
command: output:
Traceback (most recent call last):
  File "/builds/slave/rel-m-beta-chk-prms/scripts/scripts/release/push-to-mirrors.py", line 176, in <module>
    buildNumber=buildNumber)
  File "/builds/slave/rel-m-beta-chk-prms/scripts/scripts/release/push-to-mirrors.py", line 84, in checkStagePermissions
    username=stageUsername, sshKey=stageSshKey)
  File "/builds/slave/rel-m-beta-chk-prms/scripts/scripts/release/../../lib/python/util/commands.py", line 57, in run_remote_cmd
    run_cmd(cmd_prefix + cmd, **kwargs)
  File "/builds/slave/rel-m-beta-chk-prms/scripts/scripts/release/../../lib/python/util/commands.py", line 42, in run_cmd
    return subprocess.check_call(cmd, **kwargs)
  File "/tools/python-2.5.1/lib/python2.5/subprocess.py", line 461, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['ssh', '-l', 'ffxbld', '-i', '/home/cltbld/.ssh/ffxbld_dsa', 'stage.mozilla.org', 'test "0" = "$(find /pub/mozilla.org/firefox/nightly/13.0b6-candidates/build1/ -mindepth 1 -type d ! -perm 755 ! -path \'*/contrib*\' ! -path \'*/partner-repacks*\' | wc -l)"']' returned non-zero exit status 1
command: END (0.18s elapsed)
</pre>


=Build Engineers=
=Build Engineers=
Line 23: Line 53:
=Tags=
=Tags=
[[Release:Release_Automation_on_Mercurial:Documentation#Starting_the_automation|Manually tag]] the automation code, then record the generated tags below. <small>''([[Release:Release_Automation_on_Mercurial:Documentation#Tag|details]])''</small>
[[Release:Release_Automation_on_Mercurial:Documentation#Starting_the_automation|Manually tag]] the automation code, then record the generated tags below. <small>''([[Release:Release_Automation_on_Mercurial:Documentation#Tag|details]])''</small>
XXX: Verify the branch name
{| class="fullwidth-table"
{| class="fullwidth-table"
| style="background:#efefef" | '''Build #'''
| style="background:#efefef" | '''Build #'''
Line 29: Line 58:
| style="background:#efefef" | '''Changeset'''
| style="background:#efefef" | '''Changeset'''
|-  
|-  
| GECKO130_#_RELBRANCH, FIREFOX_13_0b6_BUILD1 FIREFOX_13_0b6_RELEASE  
| 1
| GECKO130_2012052815_RELBRANCH, FIREFOX_13_0b6_BUILD1 FIREFOX_13_0b6_RELEASE  
| [http://hg.mozilla.org/releases/mozilla-beta/rev/046e86aee0c1 046e86aee0c1]
| [http://hg.mozilla.org/releases/mozilla-beta/rev/046e86aee0c1 046e86aee0c1]
|}
|}
Line 35: Line 65:
=Build data=
=Build data=
== Firefox ==
== Firefox ==
XXX: Fill this data
{| class="fullwidth-table"
{| class="fullwidth-table"
| style="background:#efefef" | '''Build #'''
| style="background:#efefef" | '''Build #'''
Line 47: Line 76:
| rowspan="4" |  
| rowspan="4" |  
| linux-ix-slave28  
| linux-ix-slave28  
|  
| 1 hrs, 6 mins, 14 secs
|-
|-
| Linux64
| Linux64
| linux64-ix-slave21  
| linux64-ix-slave21  
|  
| 1 hrs, 5 mins, 46 secs
|-
|-
| Mac
| Mac
|  
| moz2-darwin10-slave42
|  
| 3 hrs, 18 mins, 57 secs
|-
|-
| Windows  
| Windows  
| w32-ix-slave37  
| w32-ix-slave37  
|  
| 3 hrs, 45 mins, 30 secs
|}
|}


Line 117: Line 146:


===== Firefox repacks =====
===== Firefox repacks =====
We had a couple of issues with the mac repacks
We had a couple of issues with the mac repacks. I re-triggered them.


=== Unittests / Talos ===
=== Unittests / Talos ===
Line 133: Line 162:
====Build====
====Build====
No problems.
No problems.
DONE TO HERE
 
<i>
=== Checksums ===
=== Checksums ===
No problems.


=== Updates ===
=== Updates ===
 
* updates generation worked
==== Update verify ====
==== Update verify ====
No problems.


=== Reset reserved slaves ===
=== Reset reserved slaves ===
[cltbld@buildbot-master13 ~]$ echo 0 > /builds/buildbot/build1/master/reserved_slaves


=== Check permissions / AV scan ===
=== Check permissions / AV scan ===
It is supposed to be triggered automatically after updates are done.
It is supposed to be triggered automatically after updates are done.
File a bug if it doesn't. <small>[[Release:Release_Automation_on_Mercurial:Documentation#Antivirus_check|details]]</small>
* check permissions failed. I assume I can fix this one I fix the candidates symlinks later on
<pre>
# do not run until all deliverables and logs are uplodaed
cd /pub/mozilla.org/firefox/nightly
rsync -av 13.0b6-candidates/ ../candidates/13.0b6-candidates/
mv 13.0b6-candidates 13.0b6-candidates.old && ln -s ../candidates/13.0b6-candidates 13.0b6-candidates
rm -rf 13.0b6-candidates.old
</pre>
* After retriggering it failed again
<pre>
Traceback (most recent call last):
  File "/builds/slave/rel-m-beta-chk-prms/scripts/scripts/release/push-to-mirrors.py", line 176, in <module>
    buildNumber=buildNumber)
  File "/builds/slave/rel-m-beta-chk-prms/scripts/scripts/release/push-to-mirrors.py", line 84, in checkStagePermissions
    username=stageUsername, sshKey=stageSshKey)
  File "/builds/slave/rel-m-beta-chk-prms/scripts/scripts/release/../../lib/python/util/commands.py", line 57, in run_remote_cmd
    run_cmd(cmd_prefix + cmd, **kwargs)
  File "/builds/slave/rel-m-beta-chk-prms/scripts/scripts/release/../../lib/python/util/commands.py", line 42, in run_cmd
    return subprocess.check_call(cmd, **kwargs)
  File "/tools/python-2.5.1/lib/python2.5/subprocess.py", line 461, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['ssh', '-l', 'ffxbld', '-i', '/home/cltbld/.ssh/ffxbld_dsa', 'stage.mozilla.org', 'test "0" = "$(find /pub/mozilla.org/firefox/nightly/13.0b6-candidates/build1/ -mindepth 1 -type d ! -perm 755 ! -path \'*/contrib*\' ! -path \'*/partner-repacks*\' | wc -l)"']' returned non-zero exit status 1
</pre>
* nthomas figured this out:
firefox/nightly is 2775, where 2 is the sticky group bit; firefox/candidates is 755
* fix it up
cd /pub/mozilla.org/firefox/candidates/
find 13.0b6-candidates -type d ! -name 'contrib*' -exec chmod g-s {} \;
* then the builder passes the tests but is orange due to this expected state
WARN: target directory /pub/mozilla.org/firefox/releases/13.0b6/ exists
* redo the rsync to mirrors to fix perms
rsync -av --exclude=*tests* --exclude=*crashreporter* --exclude=*.log --exclude=*.txt --exclude=*unsigned* --exclude=*update-backup* --exclude=*partner-repacks* --exclude=*.checksums --exclude=*.checksums.asc --exclude=logs --exclude=jsshell* --exclude=*.zip --exclude=*.zip.asc /pub/mozilla.org/firefox/nightly/13.0b6-candidates/build1/ /pub/mozilla.org/firefox/releases/13.0b6/


=== Final verification ===
=== Push to internal mirrors ===
Record receipt of "completed_final_verification" emails (1 per platform)
This is done by automation for rapid release betas. '''Note''': if you have to reconfig the release buildbot master in during the release, the uptake monitoring will fail ({{bug|629648}}). That means you'll need to manually send the "ready for releasetest" emails when you believe things are "good enough". Refer to a prior release for email details.
 
* I had to do a reconfig during the day. I sent an email manually
* This can be checked by looking the "release-mozilla-beta-ready_for_releasetest_testing" builder
 
=== Final Verification ===
Record receipt of "completed_final_verification" emails (1 per platform):
    Subject: ***SPAM*** [release] Firefox 13.0b6 build1: completed final_verification on macosx64
    Subject: ***SPAM*** [release] Firefox 13.0b6 build1: completed final_verification on linux
    Subject: ***SPAM*** [release] Firefox 13.0b6 build1: completed final_verification on linux64
    Subject: ***SPAM*** [release] Firefox 13.0b6 build1: completed final_verification on win32


=== Going to Beta ===
=== Going to Beta ===
<font color='orange'>NOTE: Skip to "Before Release Day" for '''beta''' releases</font>
==== Run backupsnip ====
==== Run backupsnip ====
NOTE: Remember to do this at least '''2 hours''' ahead of the expected "go to beta" email. <small>''[[Release:Release_Automation_on_Mercurial:Documentation#Prepare_for_Beta_Release_.28Firefox_only.29|details]]''</small>
NOTE: Remember to do this at least '''2 hours''' ahead of the expected "go to beta" email. <small>''[[Release:Release_Automation_on_Mercurial:Documentation#Prepare_for_Beta_Release_.28Firefox_only.29|details]]''</small>


Remember to ssh to aus3-staging using your short LDAP name and then run <code>sudo su - ffxbld</code> once on.  If you intend to use screen, initiate your screen session while you are still yourself.
'''Done under "screen -R armenzg" on "armenzg" user.'''
<pre>
[armenzg@dp-ausstage01 ~]$ sudo su ffxbld
[ffxbld@dp-ausstage01 armenzg]$  cd /opt/aus2/snippets/staging/
[ffxbld@dp-ausstage01 staging]$ ls -d Firefox-13.0b6*
Firefox-13.0b6-build1      Firefox-13.0b6-build1-test.time
Firefox-13.0b6-build1-test
[ffxbld@dp-ausstage01 staging]$ ~/bin/backupsnip Firefox-13.0b6-build1
</pre>
Remember to ssh to aus3-staging using your short LDAP name and then run <code>sudo su - ffxbld</code> once on.  If you intend to use screen, initiate your screen session while you are still yourself.<br />


==== Push to beta ====
==== Push to beta ====
Line 162: Line 241:
For Firefox:
For Firefox:
* Publish (and record run time here): <small>''[[Release:Release_Automation_on_Mercurial:Documentation#Beta_Release_.28Firefox_only.29|details]]''</small>
* Publish (and record run time here): <small>''[[Release:Release_Automation_on_Mercurial:Documentation#Beta_Release_.28Firefox_only.29|details]]''</small>
** missed the time incantation on the pushsnip, but it was only a few minutes
* on completion, email r-d that VERSION updates are live on the beta channel
* on completion, email r-d that VERSION updates are live on the beta channel


DONE TO HERE
<i>
Wait for confirmation from QA that each of the above releases is okay, then close the tracking ticket.
Wait for confirmation from QA that each of the above releases is okay, then close the tracking ticket.
=== Before Release Day ===
<!-- not for beta -->
Assuming this is not a chemspill.
==== Run backupsnip for release snippets ====
Do this the night before release day as it can take up to two hours.
=== Push index file to mirrors ===
=== Push to external mirrors ===
If this is a chemspill you will want to ping justdave (or other sysadmin) in #release-drivers and ask them to enable CDN.
* Remove previously added blacklisting entry from stage.mozilla.org:/pub/mozilla.org/zz/rsyncd-mozilla-releases.exclude
* Replace old version with the current one in stage.mozilla.org:/pub/mozilla.org/zz/rsyncd-mozilla-current.exclude
See [https://intranet.mozilla.org/RelEngWiki/index.php/Managing_Rsync_Modules details]
=== Release Day ===
==== Publish Fennec to the Play Store (for beta releases and release releases) ====
=====Push the files (for beta releases and release releases)=====
* Use [http://hg.mozilla.org/build/braindump/file/tip/releases-related/push_fennec.sh this script] and run it as ffxbld@stage (please push you changes prior to using it, and grab latest on stage via <tt>curl -O http://hg.mozilla.org/build/braindump/raw-file/tip/releases-related/push_fennec.sh</tt>)
=====Play Store (for beta releases and release releases)=====
'''IMPORTANT -''' Make sure to follow instructions!
* download multi apk (e.g. [http://ftp.mozilla.org/pub/mozilla.org/mobile/candidates/10.0b6-candidates/build1/android/multi/fennec-10.0b6.multi.android-arm.apk multi/fennec-10.0b6.multi.android-arm.apk])
* visit https://market.android.com/publish
* choose "Firefox Beta" or "Firefox"
* select the "APK Files" tab and choose "Upload APK"
** choose the multi apk that you downloaded and hit "upload"
*** NOTE: the progress bar does not seem to show progress (at least on Google Chrome)
** when it finishes uploading, verify VersionCode is the build date (almost anyway, the market reports an hour later than our buildID)
** hit "Save"
* activate new apk (You should see an Error message since both apk's are active)
* deactivate old apk (You should not see the Error message anymore)
* hit "Save"
* If this is a beta 1, go to "Product Details" tab -> Recent Changes, change the url to  http://www.mozilla.com/en-US/mobile/{VERSION}beta/releasenotes/
** and hit "Save"
====Push XULRunner to Mirrors ====
==== Update XULRunner wiki page ====
For major releases or chemspills, update the links on:
* https://developer.mozilla.org/en/XULRunner
* https://developer.mozilla.org/en/Gecko_SDK
Also for older releases:
* https://developer.mozilla.org/en/XULRunner_1.9.2_Release_Notes
* https://developer.mozilla.org/en/Gecko_SDK
==== Check Throttling  ====
See http://update-watch.localgho.st/release/ for example AUS links
Some links to check:
* Automatic (idle time check) update links:
** 10.0.2 en-US [https://aus3.mozilla.org/update/1/Firefox/10.0.2/20120215223356/WINNT_x86-msvc/en-US/release/update.xml win32] [https://aus2.mozilla.org/update/1/Firefox/10.0.2/20120215223356/Linux_x86-gcc3/en-US/release/update.xml linux] [https://aus2.mozilla.org/update/1/Firefox/10.0.2/20120215223356/Darwin_x86_64-gcc3-u-i386-x86_64/en-US/release/update.xml mac] [https://aus2.mozilla.org/update/1/Firefox/10.0.2/20120215223356/Linux_x86_64-gcc3/en-US/release/update.xml linux64]
** 9.0 en-US [https://aus2.mozilla.org/update/1/Firefox/9.0/20111216140209/WINNT_x86-msvc/en-US/release/update.xml win32] [https://aus2.mozilla.org/update/1/Firefox/9.0/20111216140209/Linux_x86-gcc3/en-US/release/update.xml linux] [https://aus2.mozilla.org/update/1/Firefox/9.0/20111216140209/Darwin_x86_64-gcc3-u-i386-x86_64/en-US/release/update.xml mac] [https://aus2.mozilla.org/update/1/Firefox/9.0/20111216140209/Linux_x86_64-gcc3/en-US/release/update.xml linux64]
** 10.0.2esr en-US [https://aus2.mozilla.org/update/1/Firefox/10.0.2/20120216092139/WINNT_x86-msvc/en-US/esr/update.xml win32] [https://aus2.mozilla.org/update/1/Firefox/10.0.2/20120216092139/Linux_x86-gcc3/en-US/esr/update.xml linux] [https://aus2.mozilla.org/update/1/Firefox/10.0.2/20120216092139/Darwin_x86_64-gcc3-u-i386-x86_64/en-US/esr/update.xml mac] [https://aus2.mozilla.org/update/1/Firefox/10.0.2/20120216092139/Linux_x86_64-gcc3/en-US/esr/update.xml linux64]
** 10.0esr en-US [https://aus2.mozilla.org/update/1/Firefox/10.0/20120130064731/WINNT_x86-msvc/en-US/esr/update.xml win32] [https://aus2.mozilla.org/update/1/Firefox/10.0/20120130064731/Linux_x86-gcc3/en-US/esr/update.xml linux] [https://aus2.mozilla.org/update/1/Firefox/10.0/20120130064731/Darwin_x86_64-gcc3-u-i386-x86_64/en-US/esr/update.xml mac] [https://aus2.mozilla.org/update/1/Firefox/10.0/20120130064731/Linux_x86_64-gcc3/en-US/esr/update.xml linux64]
** 3.6.27 en-US [https://aus2.mozilla.org/update/1/Firefox/3.6.27/20120216013254/WINNT_x86-msvc/en-US/release/update.xml win32] [https://aus2.mozilla.org/update/1/Firefox/3.6.27/20120216013254/Linux_x86-gcc3/en-US/release/update.xml linux] [https://aus2.mozilla.org/update/1/Firefox/3.6.27/20120216013254/Darwin_x86-gcc3-u-ppc-i386/en-US/release/update.xml mac]
* Manual update links:
** 10.0.2 en-US [https://aus3.mozilla.org/update/1/Firefox/10.0.2/20120215223356/WINNT_x86-msvc/en-US/release/update.xml?force=1 win32] [https://aus2.mozilla.org/update/1/Firefox/10.0.2/20120215223356/Linux_x86-gcc3/en-US/release/update.xml?force=1 linux] [https://aus2.mozilla.org/update/1/Firefox/10.0.2/20120215223356/Darwin_x86_64-gcc3-u-i386-x86_64/en-US/release/update.xml?force=1 mac] [https://aus2.mozilla.org/update/1/Firefox/10.0.2/20120215223356/Linux_x86_64-gcc3/en-US/release/update.xml?force=1 linux64]
** 9.0 en-US [https://aus2.mozilla.org/update/1/Firefox/9.0/20111216140209/WINNT_x86-msvc/en-US/release/update.xml?force=1 win32] [https://aus2.mozilla.org/update/1/Firefox/9.0/20111216140209/Linux_x86-gcc3/en-US/release/update.xml?force=1 linux] [https://aus2.mozilla.org/update/1/Firefox/9.0/20111216140209/Darwin_x86_64-gcc3-u-i386-x86_64/en-US/release/update.xml?force=1 mac] [https://aus2.mozilla.org/update/1/Firefox/9.0/20111216140209/Linux_x86_64-gcc3/en-US/release/update.xml?force=1 linux64]
** 10.0.2esr en-US [https://aus2.mozilla.org/update/1/Firefox/10.0.2/20120216092139/WINNT_x86-msvc/en-US/esr/update.xml?force=1 win32] [https://aus2.mozilla.org/update/1/Firefox/10.0.2/20120216092139/Linux_x86-gcc3/en-US/esr/update.xml?force=1 linux] [https://aus2.mozilla.org/update/1/Firefox/10.0.2/20120216092139/Darwin_x86_64-gcc3-u-i386-x86_64/en-US/esr/update.xml?force=1 mac] [https://aus2.mozilla.org/update/1/Firefox/10.0.2/20120216092139/Linux_x86_64-gcc3/en-US/esr/update.xml?force=1 linux64]
** 10.0esr en-US [https://aus2.mozilla.org/update/1/Firefox/10.0/20120130064731/WINNT_x86-msvc/en-US/esr/update.xml?force=1 win32] [https://aus2.mozilla.org/update/1/Firefox/10.0/20120130064731/Linux_x86-gcc3/en-US/esr/update.xml?force=1 linux] [https://aus2.mozilla.org/update/1/Firefox/10.0/20120130064731/Darwin_x86_64-gcc3-u-i386-x86_64/en-US/esr/update.xml?force=1 mac] [https://aus2.mozilla.org/update/1/Firefox/10.0/20120130064731/Linux_x86_64-gcc3/en-US/esr/update.xml?force=1 linux64]
** 3.6.27 en-US [https://aus2.mozilla.org/update/1/Firefox/3.6.27/20120216013254/WINNT_x86-msvc/en-US/release/update.xml?force=1 win32] [https://aus2.mozilla.org/update/1/Firefox/3.6.27/20120216013254/Linux_x86-gcc3/en-US/release/update.xml?force=1 linux] [https://aus2.mozilla.org/update/1/Firefox/3.6.27/20120216013254/Darwin_x86-gcc3-u-ppc-i386/en-US/release/update.xml?force=1 mac]
==== Push to Release Channel (for beta releases and release releases) ====
Once there is enough uptake and we get "go" from release driver.
==== Update symlinks ====
Once we're signed off on the release channel.
===== Remove index.html files =====
If you created them earlier to hide the release.
==== Update browser choice website ====
</i>
</i>

Latest revision as of 23:22, 29 May 2012

Notes About Releasing

Bugs hit

Enter any bugs pre-existing or newly discovered and filed during the release:

  • no L10n milestone. I took no L10n changes
  • change l10n-stage-sj.mozilla.org for l10n.mozilla.org in documentation
  • I had to lock few extra darwin10 slaves since there were 4 attached but only 2 ssh-able to start the mac builds (xulrunner was picked up first)
  • One of the mac repacks failed around Mon May 28 23:26:41 2012
remoteFailed: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other side was lost in a non-clean fashion.

# do not run until all deliverables and logs are uploaded
cd /pub/mozilla.org/firefox/nightly
rsync -av 13.0b6-candidates/ ../candidates/13.0b6-candidates/
mv 13.0b6-candidates 13.0b6-candidates.old && ln -s ../candidates/13.0b6-candidates 13.0b6-candidates
rm -rf 13.0b6-candidates.old
  • check permissions failed. I assume I can fix this one I fix the candidates symlinks later on
Error while running: find /pub/mozilla.org/firefox/nightly/13.0b6-candidates/build1/ -mindepth 1 -type d ! -perm 755 ! -path '*/contrib*' ! -path '*/partner-repacks*'
command: START
command: ssh -l ffxbld -i /home/cltbld/.ssh/ffxbld_dsa stage.mozilla.org 'test "0" = "$(find /pub/mozilla.org/firefox/nightly/13.0b6-candidates/build1/ -maxdepth 1 -type d ! -perm 2775 -path \'*/contrib*\' | wc -l)"'
command: cwd: /builds/slave/rel-m-beta-chk-prms
command: output:
Traceback (most recent call last):
  File "/builds/slave/rel-m-beta-chk-prms/scripts/scripts/release/push-to-mirrors.py", line 176, in <module>
    buildNumber=buildNumber)
  File "/builds/slave/rel-m-beta-chk-prms/scripts/scripts/release/push-to-mirrors.py", line 84, in checkStagePermissions
    username=stageUsername, sshKey=stageSshKey)
  File "/builds/slave/rel-m-beta-chk-prms/scripts/scripts/release/../../lib/python/util/commands.py", line 57, in run_remote_cmd
    run_cmd(cmd_prefix + cmd, **kwargs)
  File "/builds/slave/rel-m-beta-chk-prms/scripts/scripts/release/../../lib/python/util/commands.py", line 42, in run_cmd
    return subprocess.check_call(cmd, **kwargs)
  File "/tools/python-2.5.1/lib/python2.5/subprocess.py", line 461, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['ssh', '-l', 'ffxbld', '-i', '/home/cltbld/.ssh/ffxbld_dsa', 'stage.mozilla.org', 'test "0" = "$(find /pub/mozilla.org/firefox/nightly/13.0b6-candidates/build1/ -mindepth 1 -type d ! -perm 755 ! -path \'*/contrib*\' ! -path \'*/partner-repacks*\' | wc -l)"']' returned non-zero exit status 1
command: END (0.18s elapsed)

Build Engineers

armenzg - Tracking bug: bug 744595

Signed-off Revision(s)

Build 1: 046e86aee0c1

L10N changesets

Instructions on how to get them

  • Firefox: no milestone. I took no L10n changes

Tags

Manually tag the automation code, then record the generated tags below. (details)

Build # Branch, Tags Changeset
1 GECKO130_2012052815_RELBRANCH, FIREFOX_13_0b6_BUILD1 FIREFOX_13_0b6_RELEASE 046e86aee0c1

Build data

Firefox

Build # Type Build ID Build machine Time to build
1 Linux linux-ix-slave28 1 hrs, 6 mins, 14 secs
Linux64 linux64-ix-slave21 1 hrs, 5 mins, 46 secs
Mac moz2-darwin10-slave42 3 hrs, 18 mins, 57 secs
Windows w32-ix-slave37 3 hrs, 45 mins, 30 secs

Notes

Build 1

Preparing to start Automation

  • Set clobbers for the appropriate masters.
  • Reserve slaves (you need ~5 each of linux, linux64, win32 (w32), and macosx64 (moz2-darwin10)). Doing this 2-3 hours in advance should speed things up. e.g.:
[cltbld@buildbot-master13 master]$ echo 8 > /builds/buildbot/build1/master/reserved_slaves
  • For Firefox build1 only, create a symlink for your $version (eg 11.0b4, 10.0.2esr, 10.0.2, 3.6.27) (until bug 725839 is resolved)
# ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/
mkdir ../candidates/13.0b6-candidates
ln -s ../candidates/13.0b6-candidates 13.0b6-candidates
  • Update l10n changesets for desktop
  • Land automation configs
  • Tag buildbot-configs, buildbotcustom, & tools with build & release tags. (details)
    • Sample command for Firefox 10.0 beta 5 build 1:
hg tag -f FIREFOX_13_0b6_RELEASE FIREFOX_13_0b6_BUILD1
  • update and reconfigure the master
cd /builds/buildbot/build1/master
source ../bin/activate
PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u armenzg -V 13.0b6 --branch mozilla-beta --build-number 1 \
    --release-config release-firefox-mozilla-beta.py --products firefox  \
    --dryrun localhost:9001
  • start automation ← monitor progress on buildbot (e.g. beta)
    • the same command as above but without the "--dryrun"
    • I had to run with -b to skip checking for the L10n dashboard

E-mail Metrics

Use the address "metrics-alerts < AT > mozilla < PERIOD > org". Note for first-time-releasers: your email will get held for moderator approval - that is expected. If it happens more than once, get help on #metrics. 

Firefox,firefox,firefox,13.0b6,13.0

Tag

No problems.

Bouncer Submitter

No problems.

Source

No problems.

Build

Firefox

No problems.

Firefox repacks

We had a couple of issues with the mac repacks. I re-triggered them.

Unittests / Talos

details

  1. Look to see that they ran
  2. Document any oranges (per platform) for unittests - if possible, try to confirm it's known/random
  3. Make sure there's no red/failures that we need to have a dev look at

XULRunner

No problems.

Source

No problems.

Build

No problems.

Checksums

No problems.

Updates

  • updates generation worked

Update verify

No problems.

Reset reserved slaves

[cltbld@buildbot-master13 ~]$ echo 0 > /builds/buildbot/build1/master/reserved_slaves

Check permissions / AV scan

It is supposed to be triggered automatically after updates are done.

  • check permissions failed. I assume I can fix this one I fix the candidates symlinks later on
# do not run until all deliverables and logs are uplodaed
cd /pub/mozilla.org/firefox/nightly
rsync -av 13.0b6-candidates/ ../candidates/13.0b6-candidates/
mv 13.0b6-candidates 13.0b6-candidates.old && ln -s ../candidates/13.0b6-candidates 13.0b6-candidates
rm -rf 13.0b6-candidates.old
  • After retriggering it failed again
Traceback (most recent call last):
  File "/builds/slave/rel-m-beta-chk-prms/scripts/scripts/release/push-to-mirrors.py", line 176, in <module>
    buildNumber=buildNumber)
  File "/builds/slave/rel-m-beta-chk-prms/scripts/scripts/release/push-to-mirrors.py", line 84, in checkStagePermissions
    username=stageUsername, sshKey=stageSshKey)
  File "/builds/slave/rel-m-beta-chk-prms/scripts/scripts/release/../../lib/python/util/commands.py", line 57, in run_remote_cmd
    run_cmd(cmd_prefix + cmd, **kwargs)
  File "/builds/slave/rel-m-beta-chk-prms/scripts/scripts/release/../../lib/python/util/commands.py", line 42, in run_cmd
    return subprocess.check_call(cmd, **kwargs)
  File "/tools/python-2.5.1/lib/python2.5/subprocess.py", line 461, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['ssh', '-l', 'ffxbld', '-i', '/home/cltbld/.ssh/ffxbld_dsa', 'stage.mozilla.org', 'test "0" = "$(find /pub/mozilla.org/firefox/nightly/13.0b6-candidates/build1/ -mindepth 1 -type d ! -perm 755 ! -path \'*/contrib*\' ! -path \'*/partner-repacks*\' | wc -l)"']' returned non-zero exit status 1
  • nthomas figured this out:
firefox/nightly is 2775, where 2 is the sticky group bit; firefox/candidates is 755
  • fix it up
cd /pub/mozilla.org/firefox/candidates/
find 13.0b6-candidates -type d ! -name 'contrib*' -exec chmod g-s {} \;
  • then the builder passes the tests but is orange due to this expected state
WARN: target directory /pub/mozilla.org/firefox/releases/13.0b6/ exists
  • redo the rsync to mirrors to fix perms
rsync -av --exclude=*tests* --exclude=*crashreporter* --exclude=*.log --exclude=*.txt --exclude=*unsigned* --exclude=*update-backup* --exclude=*partner-repacks* --exclude=*.checksums --exclude=*.checksums.asc --exclude=logs --exclude=jsshell* --exclude=*.zip --exclude=*.zip.asc /pub/mozilla.org/firefox/nightly/13.0b6-candidates/build1/ /pub/mozilla.org/firefox/releases/13.0b6/

Push to internal mirrors

This is done by automation for rapid release betas. Note: if you have to reconfig the release buildbot master in during the release, the uptake monitoring will fail (bug 629648). That means you'll need to manually send the "ready for releasetest" emails when you believe things are "good enough". Refer to a prior release for email details.

  • I had to do a reconfig during the day. I sent an email manually
  • This can be checked by looking the "release-mozilla-beta-ready_for_releasetest_testing" builder

Final Verification

Record receipt of "completed_final_verification" emails (1 per platform): 

   Subject: ***SPAM*** [release] Firefox 13.0b6 build1: completed final_verification on macosx64
   Subject: ***SPAM*** [release] Firefox 13.0b6 build1: completed final_verification on linux
   Subject: ***SPAM*** [release] Firefox 13.0b6 build1: completed final_verification on linux64
   Subject: ***SPAM*** [release] Firefox 13.0b6 build1: completed final_verification on win32

Going to Beta

Run backupsnip

NOTE: Remember to do this at least 2 hours ahead of the expected "go to beta" email. details

Done under "screen -R armenzg" on "armenzg" user. 

[armenzg@dp-ausstage01 ~]$ sudo su ffxbld
[ffxbld@dp-ausstage01 armenzg]$  cd /opt/aus2/snippets/staging/ 
[ffxbld@dp-ausstage01 staging]$ ls -d Firefox-13.0b6*
Firefox-13.0b6-build1       Firefox-13.0b6-build1-test.time
Firefox-13.0b6-build1-test
[ffxbld@dp-ausstage01 staging]$ ~/bin/backupsnip Firefox-13.0b6-build1

Remember to ssh to aus3-staging using your short LDAP name and then run sudo su - ffxbld once on. If you intend to use screen, initiate your screen session while you are still yourself.

Push to beta

Wait for "go" from release managers
For Firefox:

  • Publish (and record run time here): details
    • missed the time incantation on the pushsnip, but it was only a few minutes
  • on completion, email r-d that VERSION updates are live on the beta channel

DONE TO HERE

Wait for confirmation from QA that each of the above releases is okay, then close the tracking ticket.

Retrieved from "https://wiki.mozilla.org/index.php?title=Releases/Firefox_13.0b6/BuildNotes&oldid=435383"