Confirmed users
4,293
edits
(→Fennec 34.0 build1: repack green, wonky emal) |
m (→More scheduler fallout: typo) |
||
| (36 intermediate revisions by 4 users not shown) | |||
| Line 5: | Line 5: | ||
* <s>(RelMan) [[Release:Release_Automation_on_Mercurial:Preparation#L10N_Changesets | Ship the l10n milestone]] | * <s>(RelMan) [[Release:Release_Automation_on_Mercurial:Preparation#L10N_Changesets | Ship the l10n milestone]] | ||
* (RelMan) [[Release:Release_Automation_on_Mercurial:Starting_a_Release#Submit_to_Ship_It | Submit to Ship It]]</s> | * (RelMan) [[Release:Release_Automation_on_Mercurial:Starting_a_Release#Submit_to_Ship_It | Submit to Ship It]]</s> | ||
* [[Releases/Firefox_34.0/BuildNotes#Updates_for_beta | Updates for beta]] | * [[Releases/Firefox_34.0/BuildNotes#Updates_for_beta | Updates for beta]] - prep done, waiting to be able to test | ||
* | * Started build2 (prior to shipping to beta channel) | ||
== Issues == | == Issues == | ||
* a single locale, ta, has failed in chunk 9/10 win32, restarted standalone repack (bad slave/disk space issue?), triggered release-mozilla-release-win32_repack_complete manually | |||
== Updates for beta == | == Updates for beta == | ||
| Line 18: | Line 15: | ||
* partial from 34.0b11 generated at build time (from [http://hg.mozilla.org/build/buildbot-configs/rev/418f1ba06dc4 this patch]) | * partial from 34.0b11 generated at build time (from [http://hg.mozilla.org/build/buildbot-configs/rev/418f1ba06dc4 this patch]) | ||
* | * [https://aus4-admin.mozilla.org/releases/Firefox-34.0-build1/data release blob] looks good | ||
* added bouncer product/locations for beta-cdntest/beta: | |||
<pre> | |||
Product OS Path | |||
Firefox-34.0-Partial-34.0b11 win /firefox/candidates/34.0-candidates/build1/update/win32/:lang/firefox-34.0b11-34.0.partial.mar | |||
Firefox-34.0-Partial-34.0b11 osx /firefox/candidates/34.0-candidates/build1/update/mac/:lang/firefox-34.0b11-34.0.partial.mar | |||
Firefox-34.0-Partial-34.0b11 linux64 /firefox/candidates/34.0-candidates/build1/update/linux-x86_64/:lang/firefox-34.0b11-34.0.partial.mar | |||
Firefox-34.0-Partial-34.0b11 linux /firefox/candidates/34.0-candidates/build1/update/linux-i686/:lang/firefox-34.0b11-34.0.partial.mar | |||
</pre> | |||
=== Update verify === | === Update verify === | ||
| Line 42: | Line 47: | ||
==== Running tests ==== | ==== Running tests ==== | ||
Rather than grabbing 3 slaves and running these manually, we ''may'' be able to twist the automation's arm: | |||
* wait until we get the 'updates available on betatest' email | |||
* commit the config patch to tools (no fallout from this on next release) | |||
* force the beta update verify jobs (eg release-mozilla-beta-linux_update_verify_1/6). We'd need to set the '''script_repo_revision''' property to the revision when that landed | |||
The emails we get back would be labelled 34.0b11, but I don't think it's worth bumping the release config and reconfiging just to avoid that. | |||
--> Turns out this doesn't work, script_repo_revision was still set to FIREFOX_34_0b11_RELEASE_RUNTIME. | |||
= Firefox 34.0 build2 = | |||
== Checklist == | |||
* <s>(RelMan) [[Release:Release_Automation_on_Mercurial:Preparation#L10N_Changesets | Ship the l10n milestone]] | |||
* (RelMan) [[Release:Release_Automation_on_Mercurial:Starting_a_Release#Submit_to_Ship_It | Submit to Ship It]]</s> | |||
* <s>[[Releases/Firefox_34.0/BuildNotes#Updates_for_beta_2 | Updates for beta]]</s> | |||
* <s>[[Releases/Firefox_34.0/BuildNotes#Push_to_beta | Push to beta ]]</s> | |||
* <s>[[Release:Release_Automation_on_Mercurial:Updates#Push_to_mirrors | Push to mirrors]] ''(hwine)''</s> | |||
* <s>[[Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Do_I_need_to_throttle.3F | Apply Throttling ]]</s> | |||
* <s>[[Release:Release_Automation_on_Mercurial:Updates#Verifying_Throttling | Verify throttling ]]</s> | |||
* <s>[[Release:Release_Automation_on_Mercurial:Updates#Push_snippets | Run pushsnip]]</s> | |||
* <s>[[Releases/Firefox_34.0/BuildNotes#Post_release | Special Post-release]]</s> | |||
== Issues == | |||
* didn't need to modify extraPartials, as we didn't ship build1 to beta channel | |||
* linux64 partner repack hit a 500 error grabbing pkg-dmg from hg.m.o, rebuild | |||
* the sendchange for 34.0.5 matched the schedulers for build2, firing some extra builds | |||
** tagging, which moved the release tags on m-r, and was up to te on locales before being cancelled. Remediated by putting m-r back on 456394191c90 - [http://hg.mozilla.org/releases/mozilla-release/rev/4a942164fddc changeset] | |||
** schedulers, the reset job ran, and it would be very helpful if things just worked come release time. See [[Releases/Firefox_34.0/BuildNotes#Fix scheduling]] for resolution. | |||
* first 2 final verifies (after mirror push) failed due to issues with aus3. (First run, 1 failure; second run, 4 failures; third run passed.) | |||
** contacted gozer in #webops who will double check for any issues that may impact us Monday | |||
* [[Releases/Firefox_34.0/BuildNotes#More_scheduler_fallout | more fallout from scheduler-resetting extra-sendchange]] | |||
== Updates for beta == | |||
* partial from 34.0b11 generated at build time (from [http://hg.mozilla.org/build/buildbot-configs/rev/418f1ba06dc4 this patch]) | |||
* [https://aus4-admin.mozilla.org/releases/Firefox-34.0-build2/data release blob] looks good | |||
* modified the bouncer locations for Firefox-34.0-Partial-34.0b11 to s/build1/build2/ | |||
=== Update verify === | |||
==== Prep ==== | |||
On nthomas' laptop, after en-US builds done: | |||
<pre>cd <parent_of_tools_checkout, updated to tip> | |||
export PERL5LIB="tools/lib/perl:$HOME/perl5/lib/perl5" # need Config::General v2.44 | |||
wget http://hg.mozilla.org/releases/mozilla-release/raw-file/FIREFOX_34_0_RELEASE/browser/locales/shipped-locales | |||
perl tools/release/patcher-config-bump.pl -p firefox -r Firefox -v 34.0 -a 34.0 -o 34.0b11 -b 2 \ | |||
-c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg -t stage.mozilla.org -f ftp.mozilla.org \ | |||
-d download.mozilla.org -l shipped-locales --partial-version 34.0b11 --platform linux --platform linux64 \ | |||
--platform macosx64 --platform win32 | |||
# generate update verify configs | |||
python tools/scripts/updates/create-update-verify-configs.py -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg --platform linux64 --output tools/release/updates/mozBeta-firefox-linux64.cfg --release-config-file mozilla/release-firefox-mozilla-release.py -b https://hg.mozilla.org/build/buildbot-configs --channel beta-cdntest -t FIREFOX_34_0_RELEASE | |||
python tools/scripts/updates/create-update-verify-configs.py -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg --platform linux --output tools/release/updates/mozBeta-firefox-linux.cfg --release-config-file mozilla/release-firefox-mozilla-release.py -b https://hg.mozilla.org/build/buildbot-configs --channel beta-cdntest -t FIREFOX_34_0_RELEASE | |||
python tools/scripts/updates/create-update-verify-configs.py -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg --platform macosx64 --output tools/release/updates/mozBeta-firefox-mac64.cfg --release-config-file mozilla/release-firefox-mozilla-release.py -b https://hg.mozilla.org/build/buildbot-configs --channel beta-cdntest -t FIREFOX_34_0_RELEASE | |||
python tools/scripts/updates/create-update-verify-configs.py -c tools/release/patcher-configs/mozBeta-branch-patcher2.cfg --platform win32 --output tools/release/updates/mozBeta-firefox-win32.cfg --release-config-file mozilla/release-firefox-mozilla-release.py -b https://hg.mozilla.org/build/buildbot-configs --channel beta-cdntest -t FIREFOX_34_0_RELEASE | |||
hg -R tools/ diff tools/release/updates > patch | |||
</pre> | |||
Attached to {{bug|1071606}} as [https://bugzilla.mozilla.org/attachment.cgi?id=8528998 attachment 8528998]. | |||
In Balrog, updated beta-localtest and beta-cdntest rules to point at Firefox-34.0-build2. | |||
Notice we have a problem with the complete's not being available (locations are in releases/ dir), filed {{bug|1105228}} to handle creating a new blob to point to. bhearsum landed that in Balrog, and set up new Firefox-34.0build2-Complete product & locations in candidates dir. | |||
There was some confusion about whatsnew pages. We wanted them for en-US, and set up another blob (Firefox-34.0-build2-beta-2-whatsnew) which included action and openURL. We then discovered that the in-app code only uses that when the app version is changing, so the rules part was undone. | |||
==== Running tests ==== | |||
Reverting to manual checks. | |||
<pre> | |||
# linux32 setup; others don't need anything | |||
sh -c 'rm -f /builds/mock_mozilla/mozilla-centos6-x86_64/buildroot.lock; mock_mozilla -r mozilla-centos6-x86_64 --orphanskill' | |||
mock_mozilla -r mozilla-centos6-x86_64 --init | |||
mock_mozilla -r mozilla-centos6-x86_64 --install autoconf213 python zip mozilla-python27-mercurial git ccache glibc-static.i686 libstdc++-static.i686 perl-Test-Simple perl-Config-General gtk2-devel.i686 libnotify-devel.i686 yasm alsa-lib-devel.i686 libcurl-devel.i686 wireless-tools-devel.i686 libX11-devel.i686 libXt-devel.i686 mesa-libGL-devel.i686 gnome-vfs2-devel.i686 GConf2-devel.i686 wget mpfr xorg-x11-font* imake gcc45_0moz3 gcc454_0moz1 gcc472_0moz1 gcc473_0moz1 yasm ccache valgrind pulseaudio-libs-devel.i686 gstreamer-devel.i686 gstreamer-plugins-base-devel.i686 glibc-devel.i686 libgcc.i686 libstdc++-devel.i686 ORBit2-devel.i686 atk-devel.i686 cairo-devel.i686 check-devel.i686 dbus-devel.i686 dbus-glib-devel.i686 fontconfig-devel.i686 glib2-devel.i686 hal-devel.i686 libICE-devel.i686 libIDL-devel.i686 libSM-devel.i686 libXau-devel.i686 libXcomposite-devel.i686 libXcursor-devel.i686 libXdamage-devel.i686 libXdmcp-devel.i686 libXext-devel.i686 libXfixes-devel.i686 libXft-devel.i686 libXi-devel.i686 libXinerama-devel.i686 libXrandr-devel.i686 libXrender-devel.i686 libXxf86vm-devel.i686 libdrm-devel.i686 libidn-devel.i686 libpng-devel.i686 libxcb-devel.i686 libxml2-devel.i686 pango-devel.i686 perl-devel.i686 pixman-devel.i686 zlib-devel.i686 freetype-2.3.11-6.el6_1.8.i686 freetype-devel-2.3.11-6.el6_1.8.i686 freetype-2.3.11-6.el6_1.8.x86_64 | |||
mock_mozilla -r mozilla-centos6-x86_64 --cwd /builds/slave --unpriv --shell 'PATH="/tools/buildbot/bin:/usr/local/bin:/usr/lib/ccache:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/tools/git/bin:/tools/python27/bin:/tools/python27-mercurial/bin:/home/cltbld/bin" bash' | |||
# end linux32 setup | |||
mkdir bug1071606 ; cd $_ | |||
hg clone http://hg.mozilla.org/build/tools | |||
cd tools | |||
wget -O- https://bugzilla.mozilla.org/attachment.cgi?id=8528998 | patch -p1 | |||
cd release/updates | |||
</pre> | |||
On b-linux64-hp-021 (screen 0 is linux32, screen 1 is linux64), bld-lion-r5-072, b-2008-ix-0082. Adjust to right config for platform: | |||
./verify.sh -c mozBeta-firefox-linux64.cfg 2>&1 | tee log | |||
Results: expected differences in channel-prefs.js and update-settings.ini, no failures: | |||
<pre>diff -r source/firefox/defaults/pref/channel-prefs.js target/firefox/defaults/pref/channel-prefs.js | |||
5c5 | |||
< pref("app.update.channel", "beta"); | |||
--- | |||
> pref("app.update.channel", "release"); | |||
diff -r source/firefox/update-settings.ini target/firefox/update-settings.ini | |||
5c5 | |||
< ACCEPTED_MAR_CHANNEL_IDS=firefox-mozilla-beta,firefox-mozilla-release | |||
--- | |||
> ACCEPTED_MAR_CHANNEL_IDS=firefox-mozilla-release | |||
WARN: non-binary files found in diff | |||
WARN: check_updates returned warning for Linux_x86-gcc3 downloads/firefox-34.0b11.tar.bz2 vs. downloads/firefox-34.0.tar.bz2: 2 | |||
</pre> | |||
==== Push to beta ==== | |||
bhearsum updated the rule directly in Balrog, changing the mapping to Firefox-34.0-build2-beta. | |||
== Fix scheduling == | |||
Because we hit an issue when 34.0.5 started 34.0 tagging what reset the schedulers, scheduling fix required: | |||
UPDATE schedulers SET state='{"remainingBuilders": ["release-mozilla-release-firefox_push_to_mirrors"], \ | |||
"upstreamBuilders": ["release-mozilla-release-firefox_push_to_mirrors", "release-mozilla-release-firefox_updates"], \ | |||
"lastReset": 1417001746.997826, "lastCheck": 1417001746.997826}' WHERE name='release-mozilla-release-firefox_uptake_check'; | |||
== Post release == | |||
We're using the 34.0.5 post release to get the installers in the right place, and don't want the latest symlink to be updated to 34.0, so do this by hand: | |||
# ffxbld@stage | |||
find /pub/mozilla.org/firefox/releases/34.0/ -name index.html -exec rm -v {} \; | |||
== More scheduler fallout == | |||
checksums, check permissions, and antivirus builders never ran | |||
* ran checksums by forcing the job, manually copy files over | |||
<pre># ffxbld@stage | |||
cp -pv /pub/mozilla.org/firefox/candidates/34.0-candidates/build2/{*SUMS*,KEYS} /pub/mozilla.org/firefox/releases/34.0/ | |||
rsync -nav /pub/mozilla.org/firefox/candidates/34.0-candidates/build2/contrib* /pub/mozilla.org/firefox/releases/34.0/</pre> | |||
* did permissions checks by hand | |||
<pre># ffxbld@stage | |||
find /pub/mozilla.org/firefox/releases/34.0/ ! -user ffxbld ! -path '*/contrib*' | wc -l | |||
find /pub/mozilla.org/firefox/releases/34.0/ ! -group `id -g -n ffxbld` ! -path '*/contrib*' | wc -l | |||
find /pub/mozilla.org/firefox/releases/34.0/ -type f ! -perm 644 | wc -l | |||
find /pub/mozilla.org/firefox/releases/34.0/ -mindepth 1 -type d ! -perm 755 ! -path '*/contrib*' ! -path '*/partner-repacks*' | wc -l | |||
find /pub/mozilla.org/firefox/releases/34.0/ -maxdepth 1 -type d ! -perm 2775 -path '*/contrib*' | wc -l</pre> | |||
* forced antivirus job | |||
= Firefox 34.0.5 = | |||
build notes [[Releases/Firefox_34.0.5/BuildNotes|here]] | |||
= Fennec 34.0 build1 = | = Fennec 34.0 build1 = | ||
| Line 48: | Line 185: | ||
* <s>(RelMan) [[Release:Release_Automation_on_Mercurial:Preparation#L10N_Changesets | Ship the l10n milestone]] | * <s>(RelMan) [[Release:Release_Automation_on_Mercurial:Preparation#L10N_Changesets | Ship the l10n milestone]] | ||
* (RelMan) [[Release:Release_Automation_on_Mercurial:Starting_a_Release#Submit_to_Ship_It | Submit to Ship It]]</s> | * (RelMan) [[Release:Release_Automation_on_Mercurial:Starting_a_Release#Submit_to_Ship_It | Submit to Ship It]]</s> | ||
* | * Started build2 for l10n fixes | ||
== Issues == | == Issues == | ||
* android repacks all failed in configure with java < 1.7, we missed {{bug|1082325}}. Patch landed, updated tags and re-triggered jobs, all green. | * android repacks all failed in configure with java < 1.7, we missed {{bug|1082325}}. Patch landed, updated tags and re-triggered jobs, all green. | ||
* email '[release] Fennec 34.0 build1: Updates available on None' with content 'Updates to Fennec 34.0 build1 are now ready for testing on the None channel.' Commented on {{bug|1019724}}. | * email '[release] Fennec 34.0 build1: Updates available on None' with content 'Updates to Fennec 34.0 build1 are now ready for testing on the None channel.' Commented on {{bug|1019724}}. | ||
= Fennec 34.0 build2 = | |||
== Checklist == | |||
* <s>(RelMan) [[Release:Release_Automation_on_Mercurial:Preparation#L10N_Changesets | Ship the l10n milestone]] | |||
* (RelMan) [[Release:Release_Automation_on_Mercurial:Starting_a_Release#Submit_to_Ship_It | Submit to Ship It]]</s> | |||
* Doing a build3 | |||
== Issues == | |||
* None | |||
= Fennec 34.0 build3 = | |||
Took the [https://hg.mozilla.org/build/buildbot-configs/rev/bdd8e9617403 fix for sendchanges] before starting this off. | |||
== Checklist == | |||
* <s>(RelMan) [[Release:Release_Automation_on_Mercurial:Preparation#L10N_Changesets | Ship the l10n milestone]] | |||
* (RelMan) [[Release:Release_Automation_on_Mercurial:Starting_a_Release#Submit_to_Ship_It | Submit to Ship It]]</s> | |||
* <s>[[Release:Release_Automation_on_Mercurial:Updates#Push_to_mirrors | Push to mirrors]] | |||
* [[Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Post-release_tasks | Post-release tasks]]</s> | |||
== Issues == | |||
None. | |||
= Fennec armv6 = | |||
This is done as the 31.3.0esr release. Tracked [[Releases/Fennec 31.3.0esr/BuildNotes|here]] | |||