Confirmed users
4,293
edits
Software Update:Nightly update infrastructure: Difference between revisions
Software Update:Nightly update infrastructure (view source)
Revision as of 00:50, 1 July 2006
, 1 July 2006assorted updates
(334011 landed, FTP server changed) |
(assorted updates) |
||
| Line 3: | Line 3: | ||
Updates are available for nightly builds of Firefox & Thunderbird. Keeping this system running requires the coordination of tinderbox configuration, partial update generation, and the AUS server that provides update information the client applications. This document describes how that all works, to the extent that publicly available information allows. Hopefully the build crew and Mike Morgan can fill out the details to make this page more complete. | Updates are available for nightly builds of Firefox & Thunderbird. Keeping this system running requires the coordination of tinderbox configuration, partial update generation, and the AUS server that provides update information the client applications. This document describes how that all works, to the extent that publicly available information allows. Hopefully the build crew and Mike Morgan can fill out the details to make this page more complete. | ||
Scope: Currently, updates are setup for the en-US locale on the branches for 1.5.x.y and 2.0, and on the trunk. The | Scope: Currently, updates are setup for the en-US locale on the branches for 1.5.x.y and 2.0, and on the trunk. The two Tier-1 platforms of Windows and Linux, plus a mixture of Macintosh PPC & Universal, are supported for the two main toolkit apps, Firefox & Thunderbird. | ||
=Tinderbox= | =Tinderbox= | ||
| Line 15: | Line 15: | ||
ac_add_options --enable-update-channel=nightly | ac_add_options --enable-update-channel=nightly | ||
For 1.5 the beta releases shipped with a "beta" channel, while the release candidates and final shipped with "release". | For Firefox 1.5, the beta releases shipped with a "beta" channel, while the release candidates and final shipped with "release". | ||
To pull the code for complete update generation you need: | To pull the code for complete update generation you need: | ||
| Line 63: | Line 63: | ||
The snippet is scp'ed to | The snippet is scp'ed to | ||
aus2-staging.mozilla.org:/opt/aus2/build/0/$update_product/... | |||
$update_version/$update_platform/$buildid/$locale/complete.txt | $update_version/$update_platform/$buildid/$locale/complete.txt | ||
A complete-snippet for a Firefox build from the Mozilla1.8 branch looks like: | A complete-snippet for a Firefox build from the Mozilla1.8 branch looks like: | ||
complete | complete | ||
http://mozilla | http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2006-05-10-02-mozilla1.8/firefox-2.0a2.en-US.win32.mar | ||
SHA1 | SHA1 | ||
202f8c4a5c6af9bd00569ed087e245dec87d67fb | 202f8c4a5c6af9bd00569ed087e245dec87d67fb | ||
| Line 79: | Line 79: | ||
Gathering complete update info... | Gathering complete update info... | ||
Got build ID 2006051002. | Got build ID 2006051002. | ||
Pushing third-gen update info... | Pushing third-gen update info... | ||
ssh -i /home/cltbld/.ssh/aus cltbld@ | ssh -i /home/cltbld/.ssh/aus cltbld@aus2-staging.mozilla.org mkdir -p /opt/aus2/build/0/Firefox/2.0/WINNT_x86-msvc/2006051002/en-US | ||
scp -i /home/cltbld/.ssh/aus /cygdrive/c/builds/tinderbox/Fx-Mozilla1.8/WINNT_5.2_Depend/mozilla/dist/update/update.snippet.1 cltbld@ | scp -i /home/cltbld/.ssh/aus /cygdrive/c/builds/tinderbox/Fx-Mozilla1.8/WINNT_5.2_Depend/mozilla/dist/update/update.snippet.1 cltbld@aus2-staging.mozilla.org:/opt/aus2/build/0/Firefox/2.0/WINNT_x86-msvc/2006051002/en-US/complete.txt | ||
Completed pushing update info... | Completed pushing update info... | ||
| Line 90: | Line 88: | ||
Update build completed. | Update build completed. | ||
First and second generation update info is deprecated and no longer | First and second generation update info is deprecated and no longer in the tinderbox code ([https://bugzilla.mozilla.org/show_bug.cgi?id=341752 bug 341752]). Third generation update info is only sent to AUS if the $update_pushinfo variable is set. | ||
=Partial Update Generation= | =Partial Update Generation= | ||
Not much information is available, apart from | Not much information is available, apart from | ||
* the work is done on | * the work is done on the tinderbox prometheus, using cron to start it every 5 minutes | ||
* the [http://lxr.mozilla.org/mozilla/source/tools/update-packaging/unwrap_full_update.sh unwrap_full_update.sh] | * the script [http://lxr.mozilla.org/mozilla/source/tools/update-packaging/unwrap_full_update.sh unwrap_full_update.sh] is used to unpack two complete MAR files. Then [http://lxr.mozilla.org/mozilla/source/tools/update-packaging/make_incremental_update.sh make_incremental_update.sh] is run against the two resulting directories, and that calls (m)bsdiff to create binary diff's for files that have changed in the later build. The resulting patches are combined with a manifest to create the partial MAR file. | ||
* it pushes partial | * it pushes partial update to the FTP server (into the <i>from</i> directory of any <i>from</i>--><i>to</i> update), and a partial snippet to AUS | ||
* Chase Phillips wrote the code; Paul Reed (preed) is a current build engineer who knows how to drive it | * Chase Phillips wrote the code; Paul Reed (preed) is a current build engineer who knows how to drive it | ||
=AUS= | =AUS= | ||
This server returns update information to the client, based on the snippets it has from the build systems. It was written in PHP by Mike Morgan (morgamic). | This server returns update information to the client, based on the snippets it has from the build systems. It was written in PHP by Mike Morgan (morgamic). The source is now available [http://lxr.mozilla.org/mozilla/source/webtools/aus/ from the Mozilla CVS server]. | ||
==Application update request== | ==Application update request== | ||
| Line 137: | Line 135: | ||
'2.0', '2.0a1', '2.0a2', '2.0a3', '2.0b1', '2.0b2' => '2.0', | '2.0', '2.0a1', '2.0a2', '2.0a3', '2.0b1', '2.0b2' => '2.0', | ||
'1.6a1', '3.0a1' => 'trunk' | '1.6a1', '3.0a1' => 'trunk' | ||
(This is a more compact form of the actual definition, each mapping should be on it's own line) | (This is a more compact form of the [http://lxr.mozilla.org/mozilla/source/webtools/aus/xml/inc/config-dist.php actual definition], each mapping should be on it's own line) | ||
So there is a many to one relationship between application versions from a code branch to an AUS disk location. Eg: The Mozilla1.8 branch is being used for developement of v2.0 and will have versions 2.0a1, a2, a3, b1,b2, RC1, ... and finally 2.0. The situation with the Mozilla1.8.0 branch needs fixing up (partly [https://bugzilla.mozilla.org/show_bug.cgi?id=328497 Bug 328497]), an $update_version of "1.5.x.y" would be flexible. | So there is a many to one relationship between application versions from a code branch to an AUS disk location. Eg: The Mozilla1.8 branch is being used for developement of v2.0 and will have versions 2.0a1, a2, a3, b1,b2, RC1, ... and finally 2.0. The situation with the Mozilla1.8.0 branch needs fixing up (partly [https://bugzilla.mozilla.org/show_bug.cgi?id=328497 Bug 328497]), an $update_version of "1.5.x.y" would be flexible. | ||