|
|
| (5 intermediate revisions by one other user not shown) |
| Line 1: |
Line 1: |
| | {{Obsolete RelEng Page|It was last edited in 2008, and is thus certainly out of date by now.}} |
| | |
| =Intro= | | =Intro= |
| Firefox and Thunderbird releases are currently done using the [http://wiki.mozilla.org/Build:Release_Automation#Bootstrap Bootstrap] automation scripts, which call into [http://mxr.mozilla.org/seamonkey/source/tools/tinderbox/ Tinderbox client] to do the actual build. | | Firefox and Thunderbird releases are currently done using the [http://wiki.mozilla.org/Build:Release_Automation#Bootstrap Bootstrap] automation scripts, which call into [http://mxr.mozilla.org/seamonkey/source/tools/tinderbox/ Tinderbox client] to do the actual build. |
| Line 20: |
Line 22: |
| =Current status= | | =Current status= |
|
| |
|
| For the Firefox 2.0.0.12, we are using Buildbot to drive the release. Instead of a human operater logging into [http://wiki.mozilla.org/index.php?title=Build:Release_Automation#Roles_and_resource_requirements each machine] used in the release process, the machines run Buildbot slaves instead. Most of the slaves simply check out and run Bootstrap, at this point.
| | This system is used for all current Firefox branches. Public documentation for each release is posted under the "Build" links on the [[Releases]] page. |
| | |
| Both the [http://mxr.mozilla.org/mozilla/source/tools/buildbot-configs/automation/staging/ staging] and [http://mxr.mozilla.org/mozilla/source/tools/buildbot-configs/automation/production production] configs are checked into CVS.
| |
| | |
| There are still several manual processes, which we are working on automating:
| |
| | |
| * Buildbot config
| |
| ** if necessary, tag new version of mozilla/tools/release (used RELEASE_AUTOMATION_M7)
| |
| *** make sure buildbot-configs/automation/production/master.cfg uses this tag
| |
| ** need to file bug (e.g. {{Bug|414327}}) and attach diff for bootstrap.cfg (e.g. [http://mxr.mozilla.org/mozilla/source/tools/release/configs/fx-moz18-bootstrap.cfg fx-moz18-bootstrap.cfg])
| |
| ** need to "cvs update" /home/buildmaster/Automation/buildbot-configs/ and /home/buildmaster/Automation/bootstrap-configs/ after checkin
| |
| ** need to manually insert passwords into master.cfg, as they are intentionally not in the checked-in version.
| |
| ** make sure Tinderbox is up-to-date on all slaves {{Bug|397554}}
| |
| *** used RELEASE_AUTOMATION_M7 (after tagging mozilla/tools/tinderbox with this)
| |
| *** was RELEASE_AUTOMATION_M6_1 before
| |
| ** reconfig (or restart if idle) buildbot master (<tt>buildbot reconfig `pwd`</tt>), so new automation tag is detected
| |
| ** ensure that machines have [http://wiki.mozilla.org/Build:Release_Automation#Roles_and_resource_requirements enough resources] {{Bug|393274}}
| |
| ** cd /data/cltbld/bin and run "cvs up" to update mofo scripts
| |
| *** FIXME - these should really be auto-updated and moved to public repo
| |
| ** kick off buildbot (run as cltbld):
| |
| *** buildbot sendchange --master=localhost:9989 -u joduinn -m"Firefox 2.0.0.12 RC1" release
| |
|
| |
|
| * Tag
| | All configs are checked into CVS under [http://mxr.mozilla.org/mozilla/source/tools/buildbot-configs/automation/ mozilla/tools/buildbot-configs/automation]. |
| ** *only applicable for relbranch-based builds* - Bootstrap::Tag::Bump does not handle already-bumped files on relbranch
| |
| *** need to make client.mk *_CO_TAGs use MOZILLA_1_8_BRANCH
| |
| *** need to add "2.0.0.12pre" tags to version bump files:
| |
| **** browser/app/module.ver browser/config/version.txt config/milestone.txt
| |
| | |
| * Sign
| |
| ** still manual
| |
| ** need to sync signed bits manually back to build-console e.g. as cltbld@build-console:
| |
| *** rsync -av stage.mozilla.org:/home/ftp/pub/firefox/nightly/2.0.0.12-candidates/ /home/ftp/pub/firefox/nightly/2.0.0.12-candidates/
| |
| | |
| * Updates
| |
| | |
| * Stage
| |
| ** created bouncer links manually {{Bug|414327}}
| |
| ** created "latest" and "latest-2.0" symlinks manually after final release
| |
|
| |
|
| =Roles and resource requirements= | | =Roles and resource requirements= |
| Line 103: |
Line 70: |
| make cvsmirror | | make cvsmirror |
|
| |
|
| * Update bootstrap and buildbot configs. These are symlinked from bootstrap-configs and buildbot-configs checkouts (of mozilla/tools/release/configs/ and mozilla/tools/buildbot-configs/automation/staging/, respectively). | | * Update buildbot configs. These are symlinked from the buildbot-configs checkout (of mozilla/tools/buildbot-configs/automation/staging/). |
|
| |
|
| As buildmaster@staging-build-console: | | As buildmaster@staging-build-console: |
| Line 109: |
Line 76: |
| cd /home/buildmaster/TestBot | | cd /home/buildmaster/TestBot |
| buildbot stop `pwd` | | buildbot stop `pwd` |
| cd bootstrap-configs && cvs up && cd ../
| |
| cd buildbot-configs && cvs up && cd ../ | | cd buildbot-configs && cvs up && cd ../ |
| buildbot start `pwd` | | buildbot start `pwd` |
| Line 117: |
Line 83: |
| cd /data/cltbld/bin | | cd /data/cltbld/bin |
| cvs up | | cvs up |
|
| |
| * Update the verify configs, removing the latest entry as appropriate.
| |
| These configs will be bumped during the update verify runs. e.g. for 1.8 branch Firefox:
| |
| cvs -d /builds/cvsmirror.clean/cvsroot co mozilla/testing/release/updates
| |
| cd mozilla/testing/release/updates
| |
| # remove top two lines, from moz18-firefox-*.cfg e.g.:
| |
| # 2.0.0.10 linux
| |
| release="2.0.0.10" product="Firefox" platform="Linux_x86-gcc3" build_id="2007111504" locales="af ar be bg ca cs da de el en-GB en-US es-AR es-ES eu fi fr fy-NL ga-IE gu-IN he hu it ja ka ko ku lt mk mn nb-NO nl nn-NO pa-IN pl pt-BR pt-PT ro ru sk sl sv-SE tr uk zh-CN zh-TW" channel="betatest" from="/firefox/releases/2.0.0.10/linux-i686/%locale%/firefox-2.0.0.10.tar.gz" aus_server="https://aus2.mozilla.org" ftp_server="stage.mozilla.org/pub/mozilla.org" to="/firefox/nightly/2.0.0.11-candidates/rc1/firefox-2.0.0.11.%locale%.linux-i686.tar.gz"
| |
| cvs ci -m"remove latest release from staging config"
| |
|
| |
|
| * Remove and re-checkout tinderbox-configs directories on build machines. Since the mirror takes checkins, the local checkout thinks that it has a higher revision and won't update to the now-refreshed mirror. | | * Remove and re-checkout tinderbox-configs directories on build machines. Since the mirror takes checkins, the local checkout thinks that it has a higher revision and won't update to the now-refreshed mirror. |
| Line 135: |
Line 92: |
|
| |
|
| For l10n use the MOZILLA_1_8_BRANCH_l10n_release branch. | | For l10n use the MOZILLA_1_8_BRANCH_l10n_release branch. |
|
| |
| * <del>MOZILLA_1_8_BRANCH ONLY</del>
| |
| ** <del>The Talkback symbol server is hardcoded in /builds/cvsmirror.clean/mofo/talkback/fullsoft/Makefile.in, replace all occurrences of "talkback-upload.mozilla.org" with "staging-build-console.build.mozilla.org".</del> - pre-built Talkback packages on the staging machines already contain this change ({{bug|403701}})
| |
|
| |
|
| * re-enable cltbld's nightly cronjob | | * re-enable cltbld's nightly cronjob |
|
| |
| =Enhancements=
| |
|
| |
| * ([https://bugzilla.mozilla.org/show_bug.cgi?id=394507 bug 394507]) should set buildbot up to mail based on any failures, currently just depend on bootstrap
| |
| * ({{bug|372746}}) Automatically configure bouncer
| |
| * ([https://bugzilla.mozilla.org/show_bug.cgi?id=373995 bug 373995]) l10n needs the URL it downloads builds from to be configurable as well
| |
| * ([https://bugzilla.mozilla.org/show_bug.cgi?id=394498 bug 394498]) should report on mirror saturation after release
| |
| * ({{bug|397554}}) Automatically check out, set up, and keep Tinderbox installs up to date
| |
| * [http://buildbot.net/trac/ticket/68 buildbot bug#68] buildbot default timeout too short. 5sec isnt always enough, and you can get a "timed out" message in the slave logs, even though slave started "normally".
| |
| * [http://buildbot.net/trac/ticket/85 buildbot bug#85] sometimes buildmaster sees buildslave correctly, confirms ping ok, but never assigns pending work to the slave. Doing "buildmaster refresh" is not enough, you need to do "buildmaster stop/start". Restarting the slave does not help.
| |
| * [http://buildbot.net/trac/ticket/92 buildbot bug#92] on win32, console output is not logged (goes to the DOS console running buildbot :( )
| |
| * [http://buildbot.net/trac/ticket/77 buildbot bug#77] file buildbot bug to handle kill on win32. Add details linking to bsmedberg fix.
| |
| * [http://buildbot.net/trac/ticket/67 buildbot bug#67] link to history for old builds at bottom of page (ala tinderbox server).
| |
| * [http://buildbot.net/trac/ticket/69 buildbot bug#69] meta-refresh tag for waterfall page
| |
| * [http://buildbot.net/trac/ticket/78 buildbot bug#78] buildbot UI to contain way to force build dependent steps instead of just doing current step.
| |
| * [http://buildbot.net/trac/ticket/91 buildbot bug#91] When using the CVS Source step on a Mac OSX slave, if a CVS directory is found on the path, buildbot will attempt to use it as if it were a CVS binary.
| |
| * [http://buildbot.net/trac/ticket/88 buildbot bug#88] steps which start within a few seconds of each other show as same start time on waterfall page
| |
| * (needs bug filed) tinderbox symbol server should be configurable
| |
|
| |
|
| =Setup notes (historical)= | | =Setup notes (historical)= |