canmove, Confirmed users
6,441
edits
No edit summary |
|||
| (13 intermediate revisions by 3 users not shown) | |||
| Line 1: | Line 1: | ||
* <s>[[Release:Release_Automation_on_Mercurial:Preparation#L10N_Changesets | Ship the l10n milestone]] {{DesktopTag}} {{MobileTag}} {{ReleaseTag}} {{BetaTag}}</s> | * <s>[[Release:Release_Automation_on_Mercurial:Preparation#L10N_Changesets | Ship the l10n milestone]] {{DesktopTag}} {{MobileTag}} {{ReleaseTag}} {{BetaTag}}</s> | ||
* <s>[[Release:Release_Automation_on_Mercurial:Starting_a_Release#Setting_reserved_slaves | Set reserved_slaves]] {{DesktopTag}} {{MobileTag}} {{AllTag}}</s> | * <s>[[Release:Release_Automation_on_Mercurial:Starting_a_Release#Setting_reserved_slaves | Set reserved_slaves]] {{DesktopTag}} {{MobileTag}} {{AllTag}}</s> | ||
* [[Release:Release_Automation_on_Mercurial:Starting_a_Release#Locking_slaves | Check to make sure you have all of the different types of slaves that you need]] {{DesktopTag}} {{MobileTag}} {{AllTag}} | * <s>[[Release:Release_Automation_on_Mercurial:Starting_a_Release#Locking_slaves | Check to make sure you have all of the different types of slaves that you need]] {{DesktopTag}} {{MobileTag}} {{AllTag}}</s> | ||
* <s>[[Release:Release_Automation_on_Mercurial:Preparation#Starting_the_automation | Start the automation]] {{DesktopTag}} {{MobileTag}} {{AllTag}}</s> | * <s>[[Release:Release_Automation_on_Mercurial:Preparation#Starting_the_automation | Start the automation]] {{DesktopTag}} {{MobileTag}} {{AllTag}}</s> | ||
* <s>[https://intranet.mozilla.org/RelEngWiki/index.php/Signing#Android_Signing Sign Android builds] {{MobileTag}} {{ReleaseTag}} {{BetaTag}}</s> | * <s>[https://intranet.mozilla.org/RelEngWiki/index.php/Signing#Android_Signing Sign Android builds] {{MobileTag}} {{ReleaseTag}} {{BetaTag}}</s> | ||
* [[Release:Release_Automation_on_Mercurial:Build#Reset_reserved_slaves | Reset reserved_slaves and unlock slaves]] {{DesktopTag}} {{MobileTag}} {{AllTag}} | * <s>[[Release:Release_Automation_on_Mercurial:Build#Reset_reserved_slaves | Reset reserved_slaves and unlock slaves]] {{DesktopTag}} {{MobileTag}} {{AllTag}}</s> | ||
* [[Release:Release_Automation_on_Mercurial:Updates#Push_to_mirrors | Push to mirrors]] {{DesktopTag}} {{ReleaseTag}} | * <s>[[Release:Release_Automation_on_Mercurial:Updates#Push_to_mirrors | Push to mirrors]] {{DesktopTag}} {{ReleaseTag}}</s> | ||
* [[Release:Release_Automation_on_Mercurial:Updates#Publish_Fennec | Publish Fennec]] {{MobileTag}} {{ReleaseTag}} {{BetaTag}} | * <s>[[Release:Release_Automation_on_Mercurial:Updates#Publish_Fennec | Publish Fennec]] {{MobileTag}} {{ReleaseTag}} {{BetaTag}}</s> | ||
* [[Release:Release_Automation_on_Mercurial:Updates#Push_snippets | Run pushsnip]] {{DesktopTag}} {{AllTag}} | * <s>[[Release:Release_Automation_on_Mercurial:Updates#Push_snippets | Run pushsnip]] {{DesktopTag}} {{AllTag}}</s> | ||
* <s>[[Release:Release_Automation_on_Mercurial:Updates#Update_Bouncer | Update bouncer links]] {{DesktopTag}} {{ReleaseTag}} {{BetaTag}}</s> | |||
= Bugs hit = | = Bugs hit = | ||
| Line 17: | Line 18: | ||
* {{bug|799988}} - Install android-sdk-r16 RPM to IX machines | * {{bug|799988}} - Install android-sdk-r16 RPM to IX machines | ||
* [no bug] - Add --enable-stdcxx-compat to l10n release mozconfigs for Linux mozilla-beta, to fix compiler errors. | * [no bug] - Add --enable-stdcxx-compat to l10n release mozconfigs for Linux mozilla-beta, to fix compiler errors. | ||
* {{bug|800329}} - multiple locales donn't build properly on win32, empty values break | |||
* {{bug|800457}} - 401 error hitting bouncer | |||
* {{bug|800925}} - update verify config bumper doesn't handle dropped locales properly | |||
* {{bug|800924}} - PatcherConfig class returns wrong update paths when a locale is dropped | |||
= Notes = | = Notes = | ||
| Line 40: | Line 45: | ||
=== Fix repacks === | === Fix repacks === | ||
First we hit {{bug|799806}}. nthomas wrote a patch to resolve the | First we hit {{bug|799806}}. nthomas wrote a patch to resolve the issue. | ||
After that, Windows and Mac repacks worked fine but Linux hit: | After that, Windows and Mac repacks worked fine but Linux hit: | ||
| Line 95: | Line 100: | ||
</pre> | </pre> | ||
=== | === Update generation failed because of missing 'km' build === | ||
Update generation failed with: | |||
<pre> | <pre>2012-10-11 08:22:03,160 - DEBUG - requests.packages.urllib3.connectionpool._make_request#267: "GET /pub/mozilla.org/firefox/nightly/17.0b1-candidates/build2/win32/km/firefox-17.0b1.checksums HTTP/1.1" 404 288 | ||
Traceback (most recent call last): | Traceback (most recent call last): | ||
File " | File "tools/scripts/updates/create-snippets.py", line 90, in <module> | ||
info = getChecksum(platform, locale, checksumsFile) | |||
File " | File "tools/scripts/updates/create-snippets.py", line 72, in getChecksum | ||
contents = requests.get(checksumsUrl, config={'danger_mode': True}).content | |||
File " | File "/builds/slave/rel-m-beta-updates/tools/lib/python/vendor/requests-0.10.8/requests/api.py", line 51, in get | ||
return request('get', url, **kwargs) | |||
File "/builds/slave/rel-m-beta-updates/tools/lib/python/vendor/requests-0.10.8/requests/api.py", line 39, in request | |||
return s.request(method=method, url=url, **kwargs) | |||
File "/builds/slave/rel-m-beta-updates/tools/lib/python/vendor/requests-0.10.8/requests/sessions.py", line 203, in request | |||
r.send(prefetch=prefetch) | |||
File "/builds/slave/rel-m-beta-updates/tools/lib/python/vendor/requests-0.10.8/requests/models.py", line 585, in send | |||
self.response.raise_for_status() | |||
File "/builds/slave/rel-m-beta-updates/tools/lib/python/vendor/requests-0.10.8/requests/models.py", line 810, in raise_for_status | |||
raise http_error | |||
requests.exceptions.HTTPError: 404 Client Error | |||
</pre> | </pre> | ||
To fix, had to manually change the patcher config to exclude 'km' from win32 for 17.0b1: https://hg.mozilla.org/build/tools/rev/f85e15d49942. Retagged tools with FIREFOX_17_0b1_{RELEASE,BUILD2} and used "force build" to start updates again. | |||
Failed again because "updates" regenerated the patcher config from shipped locales. Manually changed shipped-locales this time, on the relbranch: https://hg.mozilla.org/releases/mozilla-beta/rev/a3cbcfe120bd. Retagged tools with FIREFOX_17_0b1_{RELEASE,BUILD2} and used "force build" to start updates again. | |||
Failed again because it still tried to download checksums file. FIXME there's a bug in the new patcher config code that means we don't support removed locales. Next around attempt: faking out the checksums and removing the snippets later: | |||
# ffxbld@stage | |||
cd /pub/mozilla.org/firefox/candidates/17.0b1-candidates/build2/win32 | |||
mkdir km | |||
cd km | |||
cp ../ach/*.checksums . | |||
Then forced the builder again. | |||
After it completed, had to do some cleanup on stage: | |||
# ffxbld@stage | |||
cd /pub/mozilla.org/firefox/candidates/17.0b1-candidates/build2/win32 | |||
rm -rf km | |||
And in the snippets: | |||
# ffxbld@aus3-staging | |||
cd /opt/aus2/snippets/staging | |||
find Firefox-17.0b1-build2 -maxdepth 5 -type d -wholename '*WINNT*km*' -exec rm -rf {} \; | |||
Didn't bother deleting km test snippets from the test snippets. They'll get overwritten in 17.0b2. | |||
=== Push to Mirrors === | |||
[2:37pm] bhearsum: jhopkins: because of the bug noted above {{bug|800457}}, you'll need | |||
to manually monitor 17.0b1's mirror uptake | |||
(https://bounceradmin.mozilla.com/stats/uptake/?p=2038&p=2039&p=2041&p=2040), | |||
and force build the final verify builders + "ready for releasetest" + "ready for release" | |||
when there's uptake | |||
[2:41pm] jhopkins: bhearsum: i thought CDN uptake was instant? | |||
[2:41pm] Callek: catlee-buildduty: bhearsum: just to say it -- bouncer in prod is working now -- at least for my seamonkey bouncer script | |||
[2:42pm] bhearsum: jhopkins: near-instant | |||
[2:42pm] bhearsum: sentry takes a few minutes to notice that it has uptake | |||
[2:42pm] bhearsum: bounceradmin seems to be showing 50000002 availability now though, which means you have uptake | |||
[2:43pm] jhopkins: so it's sentry we're waiting for, really | |||
[2:43pm] bhearsum: kind of | |||
[2:43pm] bhearsum: we're waiting for the bouncer system (which is bounceradmin + download.m.o + sentry) to be fully aware of the release | |||
[2:43pm] bhearsum: if we don't, we'll serve 404s | |||
[2:44pm] rail: plus sync from ftp.m.o to cdn? | |||
[2:44pm] bhearsum: rail: that's near-instant, aiui | |||
[2:46pm] jhopkins: i think how it works is the CDN only caches requested files. | |||
so it'll be cached on the CDN when the first user requests a file. it will probably | |||
pull from us multiple times due to more than one distribution node plus when TTL expires | |||
for a file | |||
[2:47pm] jhopkins: the other way to do it is to upload to the CDN manually and they store | |||
all the files but we're not operating that way | |||
Bouncer indicates that uptake is over 50000000 for all platforms. | |||
Kicked off final_verification builders. Some time later, IT fixed the bouncer issue and buildbot started a duplicate set of final_verification builders. | |||
Investigated win32_update_verify failures. All '404' errors were due to the missing 'km' locale. | |||
win32_final_verification '404' errors also failed predictably on the 'km' locale. | |||
[5:12pm EST] QA has given approval for 17.0b1 _desktop_. lsblakk talking to PR and getting ready to give a final release 'go' - we have not received one yet. No word on Fennec QA progress yet. | |||
=== Publish Fennec === | |||
(Done by Hwine due to jhopkins wanting to have a life.) | |||
Grabbed: | |||
$ curl -O 'https://ftp.mozilla.org/pub/mozilla.org/mobile/candidates/17.0b1-candidates/build2/android-armv6/mult | |||
i/fennec-17.0b1.multi.android-arm-armv6.apk' | |||
% Total % Received % Xferd Average Speed Time Time Time Current | |||
Dload Upload Total Spent Left Speed | |||
100 20.8M 100 20.8M 0 0 140k 0 0:02:31 0:02:31 --:--:-- 142k | |||
$ curl -O 'https://ftp.mozilla.org/pub/mozilla.org/mobile/candidates/17.0b1-candidates/build2/android/multi/fennec-17.0b1.multi.android-arm.apk' | |||
% Total % Received % Xferd Average Speed Time Time Time Current | |||
Dload Upload Total Spent Left Speed | |||
100 19.5M 100 19.5M 0 0 206k 0 0:01:36 0:01:36 --:--:-- 568k | |||
0 [Hal@Hals-MacBook-Air Desktop] | |||
$ ll *apk | |||
-rw-r--r-- 1 Hal staff 21840767 Oct 11 14:59 fennec-17.0b1.multi.android-arm-armv6.apk | |||
-rw-r--r-- 1 Hal staff 20449419 Oct 11 15:25 fennec-17.0b1.multi.android-arm.apk | |||
0 [Hal@Hals-MacBook-Air Desktop] | |||
Posted per instructions - neither the "recent changes" nor "description" content needed version specific updates. (Tried to [https://wiki.mozilla.org/index.php?title=Release:Release_Automation_on_Mercurial:Updates_through_Shipping&diff=479050&oldid=478992 clarify] in notes.) | |||
Pushed snippets w/o incident. | |||
=== Publish Desktop === | |||
Pushsnips | |||
[ffxbld@dp-ausstage01 ~]$ cd /opt/aus2/snippets/staging | |||
[ffxbld@dp-ausstage01 staging]$ ~/bin/pushsnip Firefox-17.0b1-build2/ | |||
Ran w/o incident | |||
Bouncer Admin update worked fine: | |||
$ curl -I 'http://download.mozilla.org/?product=firefox-beta-latest&os=win&lang=en-US' | |||
HTTP/1.1 302 Found | |||
Server: Apache | |||
X-Backend-Server: bouncer2.webapp.phx1.mozilla.com | |||
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0, private | |||
Content-Type: text/html; charset=UTF-8 | |||
Date: Thu, 11 Oct 2012 23:46:21 GMT | |||
Location: http://download.cdn.mozilla.net/pub/mozilla.org/firefox/releases/17.0b1/win32/en-US/Firefox%20Setup%2017.0b1.exe | |||
Pragma: no-cache | |||
Transfer-Encoding: chunked | |||
Connection: Keep-Alive | |||