Releases/Firefox 3.6.2/BuildNotes: Difference between revisions

Revision as of 02:41, 23 March 2010

Build Engineers

Signed-off Revision(s)

Build data

Build #	Type	Build ID	Build machine
1	Linux		moz2-linux-slave20
	Mac		bm-xserve11
	Windows		win32-slave42
2	Linux		moz2-linux-slave21
	Mac		moz2-darwin9-slave13
	Windows		win32-slave34
3	Linux	20100316060223	moz2-linux-slave21
	Mac	20100316055951	moz2-darwin9-slave10
	Windows	20100316074819	win32-slave18

Notes

Build 1

Clobbered 1.9.2 on pm with the clobberer page
Landed automation patch
Tag buildbot-configs and buildbotcustom
- hg tag -r 7848b054e13a FIREFOX_3_6_2_BUILD1; hg tag -r 7848b054e13a FIREFOX_3_6_2_RELEASE (buildbot-configs)
- hg tag -r 5e5c12c1e83a FIREFOX_3_6_2_RELEASE; hg tag -r 5e5c12c1e83a FIREFOX_3_6_2_BUILD1 (buildbotcustom)
Triggered the builds

buildbot sendchange --username=armenzg --master=localhost:9010 --branch=releases/mozilla-1.9.2 -m "Firefox 3.6.2 build1" Go

I noticed that I used the wrong changeset before Mac and Linux builds were available so I stopped and started a build 2

Build 2

Clobbered 1.9.2 on pm with the clobberer page
Landed build 2 patch
Tagged buildbot-config
- hg tag -r 4b4fad2279f6 FIREFOX_3_6_2_BUILD2
Triggered the builds

buildbot sendchange --username=armenzg --master=localhost:9010 --branch=releases/mozilla-1.9.2 -m "Firefox 3.6.2 build2" Go

All builds are available but win32 repacks did not trigger:
- Specific instructions to resume l10n repacks
- python force_release_l10n.py -m http://production-master.build.mozilla.org:8010 -t FIREFOX_3_6_2_RELEASE -v -b releases/mozilla-1.9.2 -p win32 -n armenzg

Build 3

Clobbered 1.9.2 on pm with the clobberer page
bhearsum landed the two missing changesets that had landed after changeset on build1 (cd857b3b0e33) on the release branch
Landed automation patch
Triggered the builds

buildbot sendchange --username=armenzg --master=localhost:9010 --branch=releases/mozilla-1.9.2 -m "Firefox 3.6.2 build3" Go

Tag builder finished but did not trigger the rest of the builders
souce, linux_build, mac_build and win32_build were triggered manually
Tagging our repos:
- hg tag -r hg tag -f -r 5b24dd7ca36f FIREFOX_3_6_2_RELEASE; hg tag -r 5b24dd7ca36f FIREFOX_3_6_2_BUILD3 (buildbotcustom)
- hg tag -r 6e420bf20bec FIREFOX_3_6_2_BUILD3; hg tag -r 6e420bf20bec FIREFOX_3_6_2_RELEASE (tools)
- hg tag -r f0db7a70f0eb FIREFOX_3_6_2_BUILD3; hg tag -f -r f0db7a70f0eb FIREFOX_3_6_2_RELEASE (buildbot-configs)
We hit bug in "updates" generation
Updates have been generated
Snipptes have been pushed to beta

Tag

Good

Source

Good
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/3.6.2-candidates/build3/source/

Build/Repack

One of linux's sendchanges was not timing out so I had to stop it manually
- It failed the step but it did not fail the build
- The sendchange that wasn't timing out was for talos-staging-master02.build.mozilla.org:9010
- The same sendchange was affecting all platforms. I stopped the step in all 3 platforms
To trigger the locales do the following from the master:
- python force_release_l10n.py -m localhost:8010 -t FIREFOX_3_6_2_RELEASE -v -b releases/mozilla-1.9.2 -p linux -n armenzg
- python force_release_l10n.py -m localhost:8010 -t FIREFOX_3_6_2_RELEASE -v -b releases/mozilla-1.9.2 -p macosx -n armenzg
- python force_release_l10n.py -m localhost:8010 -t FIREFOX_3_6_2_RELEASE -v -b releases/mozilla-1.9.2 -p win32 -n armenzg
I retriggered the "id" locale which failed downloading the candidates zip
- http://production-master.build.mozilla.org:8010/builders/win32_repack/builds/2033

--12:11:13--  http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/3.6.2-candidates/build3/unsigned/win32/en-US/firefox-3.6.2.zip
           => `firefox.zip'
Resolving stage.mozilla.org... 10.2.74.116
Connecting to stage.mozilla.org|10.2.74.116|:80... connected.
HTTP request sent, awaiting response... 502 Proxy Error
12:11:21 ERROR 502: Proxy Error.

partner repack step failed to upload the generated builds. Given all the other network failures at the time, this isn't surprising. Re-ran the upload step by hand without issue.

Remove build1/en-US.xpi bug 485860

Unit tests

Make sure that you check to see that all unit tests were green

Sign

I tried to start signing before all-locales were ready (WRONG!)
Before started signing I got an rsync failure and after talking with bhearsum I figured that I need all locales for all platforms

Saving to: `app.tag'

100%[======================================>] 81          --.-K/s   in 0s

2010-03-16 10:40:17 (36.5 MB/s) - `app.tag' saved [81/81]

/home/cltsign/signing-work/firefox-3.6.2/upx/upx.exe --best -o checkouts/stubs/7z/7zSD.sfx.compressed checkouts/stubs/7z/7zSD.sfx
                      Ultimate Packer for eXecutables
   Copyright (C) 1996,1997,1998,1999,2000,2001,2002,2003,2004,2005,2006
UPX 2.03w       Markus Oberhumer, Laszlo Molnar & John Reiser    Nov 7th 2006

       File size         Ratio      Format      Name
  --------------------   ------   -----------   -----------
   122368 ->     70144   57.32%    win32/pe     7zSD.sfx.compressed

Packed 1 file.
blat -to armenzg@mozilla.com \
            -subject "sign-release.py for firefox 3.6.2 starting" \
                -body "EOM"
Blat v2.5.0 w/GSS encryption (build : Sep 14 2005 22:46:29) 

Sending stdin.txt to armenzg@mozilla.com
Subject: sign-release.py for firefox 3.6.2 starting
Login name is cltsign@cm-keymaster01.mozilla.org
./sign-release.py --keydir d:/2009-keys -j4 --product firefox -o signed-build3 unsigned-build3 >> win32_signing_build3.log 2>&1
# And copy over the info file and XPIs.
cp -pv ./unsigned-build3/unsigned/win32_info.txt ./signed-build3/
`./unsigned-build3/unsigned/win32_info.txt' -> `./signed-build3/win32_info.txt'
rsync -av ./unsigned-build3/unsigned/win32/xpi ./signed-build3/win32
sending incremental file list
rsync: link_stat "/cygdrive/d/signing-work/firefox-3.6.2/unsigned-build3/unsigned/win32/xpi" failed: No such file or directory (2)

sent 18 bytes  received 12 bytes  60.00 bytes/sec
total size is 0  speedup is 0.00
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at /home/lapo/packaging/rsync-3.0.4-1/src/rsync-3.0.4/main.c(1040) [sender=3.0.4]
make: *** [sign-files] Error 23

I triggered the missing windows locales
I have to clobber everything on keymaster and start again once all repacks are done
This second time had no complains. Signed builds are currently being uploaded.

Partner Signing

to sign the repack partners (I believe this would have not been necessary if I had had the partner repacks in the unsigned folder on ftp with the other win32 unsigned files).

this is the first time that we are signing repacks without having received a tar ball

PRODUCT=firefox
VERSION=3.6.2
BUILD=3
TAG=FIREFOX_3_6_2_RELEASE
REPO='releases/mozilla-1.9.2'
EMAIL=armenzg@mozilla.com
KEYDIR=d:/2009-keys
cd ~/hg-tools
hg pull && hg up -r ${TAG}
#start signcode
mkdir -p ~/signing-work/${PRODUCT}-${VERSION}-repacks/partners
cd ~/signing-work/${PRODUCT}-${VERSION}-repacks/partners
cp ~/hg-tools/release/signing/* .
make PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} TAG=${TAG} REPO=${REPO} EMAIL=${EMAIL} USE_NEW=1 setup
mkdir unsigned-build${BUILD}
scp cltbld@stage.mozilla.org:/pub/mozilla.org/${PRODUCT}/nightly/${VERSION}-candidates/${BUILD}/unsigned/partner-repacks/* .
rsync -av --exclude-from download-exclude.list -e "ssh -i ${HOME}/.ssh/ffxbld_dsa" ffxbld@stage.mozilla.org:/home/ftp/pub/${PRODUCT}/nightly/${VERSION}-candidates/build${BUILD}/unsigned/partner-repacks unsigned-build${BUILD}

Here is where I have left the signing. I based this off Build:CombinedSigning#Partner_Repack_Signing which can't be used directly in here.

# download partner bundle, usually from people.m.c cd unsigned tar xvf ../partners-355-unsigned.tar make PRODUCT=${PRODUCT} VERSION=${VERSION} BUILD=${BUILD} TAG=${TAG} REPO=${REPO} EMAIL=${EMAIL} USE_NEW=1 setup # This next step will fail to find the win32 log and/or xpis. # This is expected. cd win32 && ../../signing/sign-release ${KEYDIR} && cd ../ ../signing/sign-files . cd .. mv unsigned signed cd signed tar -vcf ../partners-355-signed.tar . cd ../

L10nVerify

No problems

Generate updates and push betatest snippets

We hit a bug. Nick helped me with this.
bug 552831 - patcher-config-bump.pl fails when flipping useBeta to True

perl ../tools/release/patcher-config-bump.pl -p firefox -r Firefox -v 3.6.2 -a 3.6.2 -o 3.6rc2 -b 3 -c patcher-configs/moz192-branch-patcher2.cfg -t stage-old.mozilla.org -f ftp.mozilla.org -d download.mozilla.org -l shipped-locales -u
...
Use of uninitialized value in substitution (s///) at ../tools/release/patcher-config-bump.pl line 127.
Use of uninitialized value in pattern match (m//) at ../tools/release/patcher-config-bump.pl line 129.
Use of uninitialized value in pattern match (m//) at ../tools/release/patcher-config-bump.pl line 129.
Use of uninitialized value in concatenation (.) or string at ../tools/release/patcher-config-bump.pl line 140.
ASSERT: BumpFilePath() - Unknown file type for  at ../tools/release/patcher-config-bump.pl line 140.

Once this landed, I deleted moz2-linux-slave12:/builds/slave/updates/* and force the builder
updates generation is completed

Update verify

Waiting on mac_update_verify
mac_update_verify took forever because of the network issues but it finished

Publish updates to beta channel

THU. 18th of March - 6:30 PDT
Backing up snippets

# cltbld@aus2-staging.m.o
# make sure scripts are up to date
cd bin
cvs update
cd /opt/aus2/snippets/staging/
# NOTE: That that backupsnip only backs up the folder you specify and nothing older as it used to
~/bin/backupsnip 20100316-Firefox-3.6.2-beta
~/bin/pushsnip 20100316-Firefox-3.6.2-beta

Socorro product data updates

We can do this ourselves (a subset of relengs, ask nthomas or bhearsum). No need to file an IT bug for this.

Nick added for me the entries.

Added 3.6.2 and 3.6.3pre with the crash-stats admin page.

Update Bouncer

Added the following products:
- Firefox-3.6.2
- Firefox-3.6.2-Complete
- Firefox-3.6.2-Partial-3.6rc2
Added files to bouncer

Final checks before push

# ffxbld@stage
cd /pub/mozilla.org/firefox/nightly/3.6.2-candidates/build3/
find . ! -user ffxbld #only the solaris contributed builds showed up
find . ! -group firefox
find . -type f ! -perm 644 # unsigned/partner-repacks have 664 permission
find . -mindepth 1 -type d ! -perm 755 ! -name 'contrib*' # unsigned/partner-repacks have 664 permission
find . -maxdepth 1 -type d ! -perm 2775  -name 'contrib*'

Remove throttling on previous releases

No need to remove throttling from 3.6.0

Virus Scan

# stage
$ cd /pub/mozilla.org/firefox/nightly/3.6.2-candidates/build3
$ clamdscan -m
/mnt/netapp/stage/archive.mozilla.org/pub/firefox/nightly/3.6.2-candidates/build3: OK

----------- SCAN SUMMARY -----------
Infected files: 0
Time: 2917.127 sec (48 m 37 s)

Push to mirrors

Check https://nagios.mozilla.org/sentry or bouncer's mirror uptake
We want to push partner repacks to candidates dirs but we want to make sure we don't send it to the mirrors. This will be the first time we do this and that is why I am adding a note beforehand in this section

After GO from driver:

# ffxbld@stage
rsync -av --exclude=*.log --exclude=*.txt --exclude=*unsigned* \
 --exclude=*.crashreporter-symbols.zip --exclude=*.tests.tar.bz2* \
 --exclude=*partner-repacks* \
 /pub/mozilla.org/firefox/nightly/3.6.2-candidates/build3/ \
 /pub/mozilla.org/firefox/releases/3.6.2/

Updated cltbld@stage:/pub/mozilla.org/zz/rsyncd-mozilla-current.exclude to include 3.6.2 instead of 3.6.

Final Verification

Got 9 failures like this

Using  https://aus2.mozilla.org/update/1/Firefox/3.6b5/20091204132459/Linux_x86-gcc3/nn-NO/releasetest/update.xml?force=1
Testing http://download.mozilla.org/?product=firefox-3.6.2-complete&os=linux&lang=nn-NO
HTTP/1.1 302 Found
Date: Tue, 23 Mar 2010 00:22:22 GMT
Server: Apache
Set-Cookie: dmo=10.2.84.4.1269303742243225; path=/; expires=Wed, 23-Mar-11 00:22:22 GMT
X-Powered-By: PHP/5.1.6
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0, private
Pragma: no-cache
Location: http://3347-mozilla.voxcdn.com/pub/mozilla.org/firefox/releases/3.6.2/update/linux-i686/nn-NO/firefox-3.6.2.complete.mar
Content-Type: text/html; charset=UTF-8

FAIL: download_mars returned non-zero exit code: 7

Exit status of 7 comes from curl and means 'Failed to connect to host'. nthomas and reed have attempted to debug this in the past and reed has not been able to reproduce it on his end. We are assuming that it's something between the build network and voxcdn only.

Push updates to release channel

# cltbld@aus2-staging
cd /opt/aus2/snippets/staging/
~/bin/backupsnip 20100316-Firefox-3.6.2
~/bin/pushsnip 20100316-Firefox-3.6.2

Release

# ffxbld@stage
rm latest-3.6
ln -s 3.6.2 latest-3.6

Build #	Tag	Changeset
1	GECKO1922_20100315_RELBRANCH	f14062c981ba
1	FIREFOX_3_6_2_BUILD1	f14062c981ba
2	FIREFOX_3_6_2_BUILD2	4b4fad2279f6
3	FIREFOX_3_6_2_BUILD3 FIREFOX_3_6_2_RELEASE	cd857b3b0e33

Releases/Firefox 3.6.2/BuildNotes: Difference between revisions

Revision as of 02:41, 23 March 2010

Contents

Build Engineers

Signed-off Revision(s)

Tags

Build data

Notes

Build 1

Build 2

Build 3

Tag

Source

Build/Repack

Unit tests

Sign

Partner Signing

L10nVerify

Generate updates and push betatest snippets

Update verify

Publish updates to beta channel

Socorro product data updates

Update Bouncer

Final checks before push

Remove throttling on previous releases

Virus Scan

Push to mirrors

Final Verification

Push updates to release channel

Release

XULRunner build

Navigation menu

Releases/Firefox 3.6.2/BuildNotes: Difference between revisions

Revision as of 02:41, 23 March 2010

Build Engineers

Signed-off Revision(s)

Tags

Build data

Notes

Build 1

Build 2

Build 3

Tag

Source

Build/Repack

Unit tests

Sign

Partner Signing

L10nVerify

Generate updates and push betatest snippets

Update verify

Publish updates to beta channel

Socorro product data updates

Update Bouncer

Final checks before push

Remove throttling on previous releases

Virus Scan

Push to mirrors

Final Verification

Push updates to release channel

Release

XULRunner build

Navigation menu

Search