|
|
| (27 intermediate revisions by 5 users not shown) |
| Line 1: |
Line 1: |
| | {{Obsolete RelEng Page|It was last edited in 2008, and is thus certainly out of date by now.}} |
| | |
| =Intro= | | =Intro= |
| Firefox and Thunderbird releases are currently done using the [http://wiki.mozilla.org/Build:Release_Automation#Bootstrap Bootstrap] automation scripts, which call into [http://mxr.mozilla.org/seamonkey/source/tools/tinderbox/ Tinderbox client] to do the actual build. | | Firefox and Thunderbird releases are currently done using the [http://wiki.mozilla.org/Build:Release_Automation#Bootstrap Bootstrap] automation scripts, which call into [http://mxr.mozilla.org/seamonkey/source/tools/tinderbox/ Tinderbox client] to do the actual build. |
| Line 20: |
Line 22: |
| =Current status= | | =Current status= |
|
| |
|
| For the Firefox 2.0.0.8 release, we are using Buildbot to drive the release. Instead of a human operater logging into [http://wiki.mozilla.org/index.php?title=Build:Release_Automation#Roles_and_resource_requirements each machine] used in the release process, the machines run Buildbot slaves instead. Most of the slaves simply check out and run Bootstrap, at this point.
| | This system is used for all current Firefox branches. Public documentation for each release is posted under the "Build" links on the [[Releases]] page. |
| | |
| Both the [http://mxr.mozilla.org/mozilla/source/tools/buildbot-configs/automation/staging/ staging] and [http://mxr.mozilla.org/mozilla/source/tools/buildbot-configs/automation/production production] configs are checked into CVS.
| |
| | |
| There are still several manual processes, which we are working on:
| |
|
| |
|
| * Buildbot config
| | All configs are checked into CVS under [http://mxr.mozilla.org/mozilla/source/tools/buildbot-configs/automation/ mozilla/tools/buildbot-configs/automation]. |
| ** if necessary, tag new version of mozilla/tools/release (used RELEASE_AUTOMATION_M5_3)
| |
| *** make sure buildbot-configs/automation/production/master.cfg uses this tag
| |
| ** need to file bug (e.g. {{Bug|400770}}) and attach diff for bootstrap.cfg (e.g. [http://mxr.mozilla.org/mozilla/source/tools/release/configs/fx-moz18-bootstrap.cfg fx-moz18-bootstrap.cfg])
| |
| ** need to "cvs update" /home/buildmaster/Automation/buildbot-configs/ and /home/buildmaster/Automation/bootstrap-configs/ after checkin
| |
| ** need to manually insert passwords into master.cfg, as they are intentionally not in the checked-in version.
| |
| ** make sure Tinderbox is up-to-date on all slaves {{Bug|397554}}
| |
| *** used RELEASE_AUTOMATION_M5_3
| |
| ** ensure that machines have [http://wiki.mozilla.org/Build:Release_Automation#Roles_and_resource_requirements enough resources] {{Bug|393274}}
| |
| ** kick off buildbot (run as cltbld):
| |
| *** buildbot sendchange --master=localhost:9989 -u joduinn -m"Firefox 2.0.0.9 RC1" release
| |
| | |
| * Tag
| |
| * Bootstrap::Tag::Bump does not handle already-bumped files on relbranch
| |
| ** need to make client.mk *_CO_TAGs use MOZILLA_1_8_BRANCH
| |
| ** need to add "2.0.0.8pre" tags to version bump files:
| |
| *** browser/app/module.ver browser/config/version.txt config/milestone.txt
| |
| | |
| * Sign
| |
| ** still manual
| |
| ** need to sync signed bits manually back to build-console e.g. as cltbld@build-console:
| |
| *** rsync -av stage.mozilla.org:/home/ftp/pub/firefox/nightly/2.0.0.9-candidates/ /home/ftp/pub/firefox/nightly/2.0.0.9-candidates/
| |
| | |
| * Updates
| |
| ** <strike>call push, "./release -o Updates -p", manually (should be fixed on bootstrap trunk {{Bug|389206}})</strike>
| |
| ** <strike>had to correct permissions for both snippets and MARs (should be fixed on bootstrap trunk {{Bug|399628}})</strike>
| |
| ** <strike>update verification config is still manual (should be fixed on bootstrap trunk {{Bug|373995}})</strike>
| |
| | |
| * Stage
| |
| ** <strike>had to correct permissions (should be fixed on bootstrap trunk {{Bug|399628}})</strike>
| |
| ** <strike>need to rsync /data/cltbld/firefox-2.0.0.8/ stage:/data/cltbld/firefox-2.0.0.8/ (not covered by {{Bug|396438}}).</strike>
| |
| ** created bouncer links manually {{Bug|372746}}
| |
| ** created "latest" and "latest-2.0" symlinks manually after final release
| |
|
| |
|
| =Roles and resource requirements= | | =Roles and resource requirements= |
| Line 94: |
Line 60: |
|
| |
|
| =Updating Staging release version= | | =Updating Staging release version= |
| * Bump config versions in mozilla/tools/release/Makefile, mozilla/tools/configs/fx-moz18-staging-bootstrap.cfg, mozilla/tools/buildbot-configs/automation/staging/master.cfg e.g. {{bug|397425}} | | * Bump config versions in mozilla/tools/release/Makefile, mozilla/tools/configs/fx-moz18-staging-bootstrap.cfg, mozilla/tools/buildbot-configs/automation/staging/master.cfg e.g. {{bug|407672}} |
|
| |
|
| * Disable cltbld's nightly cronjob | | * As cltbld@staging-build-console, disable cltbld's nightly cronjob to prevent accidently starting mid-way through update. |
|
| |
|
| * Refresh cvsmirror | | * As cltbld@staging-build-console, refresh cvsmirror by doing: |
| | |
| As cltbld@staging-build-console: | |
| cd /home/cltbld/mozilla/tools/release | | cd /home/cltbld/mozilla/tools/release |
| cvs up | | cvs up |
| Line 106: |
Line 70: |
| make cvsmirror | | make cvsmirror |
|
| |
|
| * Update bootstrap and buildbot configs. These are symlinked from bootstrap-configs and buildbot-configs checkouts (of mozilla/tools/release/configs/ and mozilla/tools/buildbot-configs/automation/staging/, respectively). | | * Update buildbot configs. These are symlinked from the buildbot-configs checkout (of mozilla/tools/buildbot-configs/automation/staging/). |
|
| |
|
| As buildmaster@staging-build-console: | | As buildmaster@staging-build-console: |
| | # pull configs and restart buildbot |
| cd /home/buildmaster/TestBot | | cd /home/buildmaster/TestBot |
| buildbot stop `pwd` | | buildbot stop `pwd` |
| cd bootstrap-configs && cvs up && cd ../
| |
| cd buildbot-configs && cvs up && cd ../ | | cd buildbot-configs && cvs up && cd ../ |
| buildbot start `pwd` | | buildbot start `pwd` |
|
| |
|
| NOTE - the Talkback symbol server is hardcoded in /builds/cvsmirror.clean/mofo/talkback/fullsoft/Makefile.in, this should be changed like so:
| | As cltbld@staging-build-console: |
| | # update mofo scripts |
| | cd /data/cltbld/bin |
| | cvs up |
|
| |
|
| FC_TUNNEL = ssh -$(FC_SSH_VERSION) -f -L 8080:hal:80 $(LSSH_USER)staging-build-console.build.mozilla.org sleep 20
| | * Remove and re-checkout tinderbox-configs directories on build machines. Since the mirror takes checkins, the local checkout thinks that it has a higher revision and won't update to the now-refreshed mirror. |
| SYM_TUNNEL = ssh -$(SYM_SSH_VERSION) -f -L 2222:localhost:22 $(LSSH_USER)staging-build-console.build.mozilla.org sleep 20
| |
|
| |
|
| * re-enable cltbld's nightly cronjob
| | # e.g. linux |
| | cd /builds/tinderbox/Fx-Mozilla1.8-release |
| | rm -rf tinderbox-configs |
| | cvs -d :ext:cltbld@staging-build-console.build.mozilla.org:/builds/cvsmirror/cvsroot co -d tinderbox-configs -r MOZILLA_1_8_BRANCH_release mozilla/tools/tinderbox-configs/firefox/linux |
|
| |
|
| =Enhancements=
| | For l10n use the MOZILLA_1_8_BRANCH_l10n_release branch. |
|
| |
|
| * ([https://bugzilla.mozilla.org/show_bug.cgi?id=394507 bug 394507]) should set buildbot up to mail based on any failures, currently just depend on bootstrap | | * re-enable cltbld's nightly cronjob |
| * ({{bug|372746}}) Automatically configure bouncer
| |
| * ([https://bugzilla.mozilla.org/show_bug.cgi?id=373995 bug 373995]) l10n needs the URL it downloads builds from to be configurable as well
| |
| * ([https://bugzilla.mozilla.org/show_bug.cgi?id=394498 bug 394498]) should report on mirror saturation after release
| |
| * ({{bug|397554}}) Automatically check out, set up, and keep Tinderbox installs up to date
| |
| * [http://buildbot.net/trac/ticket/68 buildbot bug#68] buildbot default timeout too short. 5sec isnt always enough, and you can get a "timed out" message in the slave logs, even though slave started "normally".
| |
| * [http://buildbot.net/trac/ticket/85 buildbot bug#85] sometimes buildmaster sees buildslave correctly, confirms ping ok, but never assigns pending work to the slave. Doing "buildmaster refresh" is not enough, you need to do "buildmaster stop/start". Restarting the slave does not help.
| |
| * [http://buildbot.net/trac/ticket/92 buildbot bug#92] on win32, console output is not logged (goes to the DOS console running buildbot :( )
| |
| * [http://buildbot.net/trac/ticket/77 buildbot bug#77] file buildbot bug to handle kill on win32. Add details linking to bsmedberg fix.
| |
| * [http://buildbot.net/trac/ticket/67 buildbot bug#67] link to history for old builds at bottom of page (ala tinderbox server).
| |
| * [http://buildbot.net/trac/ticket/69 buildbot bug#69] meta-refresh tag for waterfall page
| |
| * [http://buildbot.net/trac/ticket/78 buildbot bug#78] buildbot UI to contain way to force build dependent steps instead of just doing current step.
| |
| * [http://buildbot.net/trac/ticket/91 buildbot bug#91] When using the CVS Source step on a Mac OSX slave, if a CVS directory is found on the path, buildbot will attempt to use it as if it were a CVS binary.
| |
| * [http://buildbot.net/trac/ticket/88 buildbot bug#88] steps which start within a few seconds of each other show as same start time on waterfall page
| |
| * (needs bug filed) tinderbox symbol server should be configurable
| |
|
| |
|
| =Setup notes (historical)= | | =Setup notes (historical)= |
| * [[Build:Release Automation:Notes on Staging Setup|Notes on Staging Setup]] | | * [[Build:Release Automation:Notes on Staging Setup|Notes on Staging Setup]] |
| * [[Build:Release Automation:Notes on Production Setup|Notes on Production Setup]] | | * [[Build:Release Automation:Notes on Production Setup|Notes on Production Setup]] |
| | * [[Build:Release Automation:Trunk|Notes on Trunk Automation Setup]] |