Releases/Firefox 32.0b9/BuildNotes

Bugs hit

• fallout from bug 1032978
• 'hg update -C' failure on some win32 ix boxes bug 1057549

Build 1

• release-runner failure from bug 1032978

[bumps and lands everything, reconfigs masters, then gets to the sendchange ...]
2014-08-21 23:22:22,585 - INFO - updating status for Firefox-32.0b9-build1 to Running sendchange command
2014-08-21 23:22:22,585 - DEBUG - Request to https://ship-it.mozilla.org/releases/Firefox-32.0b9-build1
2014-08-21 23:22:22,585 - DEBUG - Data sent: {'status': 'Running sendchange command', 'csrf_token': ...
2014-08-21 23:22:22,587 - INFO - Resetting dropped connection: ship-it.mozilla.org
2014-08-21 23:22:22,877 - DEBUG - "POST /releases/Firefox-32.0b9-build1 HTTP/1.1" 200 0
2014-08-21 23:22:22,878 - INFO - updating status for Firefox-32.0b9-build1 to Sendchange failed
2014-08-21 23:22:22,878 - DEBUG - Request to https://ship-it.mozilla.org/releases/Firefox-32.0b9-build1
2014-08-21 23:22:22,878 - DEBUG - Data sent: {'status': 'Sendchange failed', 'csrf_token': ...
2014-08-21 23:22:22,930 - DEBUG - "POST /releases/Firefox-32.0b9-build1 HTTP/1.1" 200 0
2014-08-21 23:22:22,930 - ERROR - Sendchange failed for {'status': 'Pending', 'comment': 'Bug 1057061 is still outstanding for beta9 as the patch was leaking ...
'32.0b8build1,32.0b7build1', 'ready': True, 'mozillaRevision': '58eb677e55f3', 'name': 'Firefox-32.0b9-build1'}:
Traceback (most recent call last):
  File "release-runner.py", line 428, in main
    enUSPlatforms = readReleaseConfig(cfgFile)['enUSPlatforms']
  File "/builds/releaserunner/tools/lib/python/release/info.py", line 66, in readReleaseConfig
    return readConfig(configfile, keys=['releaseConfig'], required=required)
  File "/builds/releaserunner/tools/lib/python/release/info.py", line 84, in readConfig
    execfile(configfile, c)
IOError: [Errno 2] No such file or directory: 'release-firefox-mozilla-beta.py'

Then two later errors from RelMan hitting ready on the release again, and release-runner complaining that the config was already bumped.

pmoore backed out the tools part of bug 1032978, nthomas updated release runner so that it's in working state. pmoore backed out the buildbotcustom part of bug 1032978, which looks like it isn't strictly required but nice for consistency. Reconfig for that.

sendchange to get us going again on bm81:

/builds/buildbot/build_scheduler/bin/buildbot sendchange --username lmandel@mozilla.com --master buildbot-master81.build.mozilla.org:9301 --branch releases/mozilla-beta -p products:firefox -p script_repo_revision:FIREFOX_32_0b9_RELEASE release_build

Try (and fail) to set the status in ship-it:

cd /builds/releaserunner
. bin/activate
cd tools/buildfarm/release
python

from os import path
import site
site.addsitedir(path.join(path.dirname('.'), "../../lib/python"))

from kickoff.api import Release
from util.file import load_config

config = load_config('../../../release-runner.ini')
api_root = config.get('api', 'api_root')
username = config.get('api', 'username')
password = config.get('api', 'password')
release_api = Release((username, password), api_root=api_root)

release_api.update('Firefox-32.0b9-build1', complete=True, status='Started')

This leaves the state in the db like this:

mysql> select name, version, ready, complete, status from firefox_release order by submittedAt desc limit 4;
+-----------------------+---------+-------+----------+---------+
| name                  | version | ready | complete | status  |
+-----------------------+---------+-------+----------+---------+
| Firefox-32.0b9-build1 | 32.0b9  |     0 |        1 | Started |
| Firefox-32.0b8-build1 | 32.0b8  |     1 |        1 | Started |
| Firefox-32.0b7-build1 | 32.0b7  |     1 |        1 | Started |
| Firefox-32.0b6-build1 | 32.0b6  |     1 |        1 | Started |
+-----------------------+---------+-------+----------+---------+

so 32.0b9 still shows up in the 'Submitted' section of the ship-it UI. Updated ready to 1 in the db.

compilation error on win 32

crashinject.exe
c:/builds/moz2_slave/rel-m-beta-w32_bld-00000000000/build/obj-firefox/_virtualenv/Scripts/python.exe c:/builds/moz2_slave/rel-m-beta-w32_bld-00000000000/build/config/expandlibs_exec.py --depend .deps/crashinject.exe.pp --target crashinject.exe --uselist -- link -NOLOGO -OUT:crashinject.exe -PDB:crashinject.pdb  -LARGEADDRESSAWARE -NXCOMPAT -RELEASE -DYNAMICBASE -SAFESEH  -DEBUG -DEBUGTYPE:CV -DEBUG -OPT:REF      crashinject.obj ./module.res kernel32.lib user32.lib gdi32.lib winmm.lib wsock32.lib advapi32.lib secur32.lib netapi32.lib  kernel32.lib user32.lib gdi32.lib winmm.lib wsock32.lib advapi32.lib secur32.lib netapi32.lib
c:/builds/moz2_slave/rel-m-beta-w32_bld-00000000000/build/config/rules.mk:729: recipe for target 'crashinject.exe' failed
mozmake.exe[6]: Leaving directory 'c:/builds/moz2_slave/rel-m-beta-w32_bld-00000000000/build/obj-firefox/build/win32'
c:/builds/moz2_slave/rel-m-beta-w32_bld-00000000000/build/config/recurse.mk:179: recipe for target 'libs' failed
mozmake.exe[5]: Leaving directory 'c:/builds/moz2_slave/rel-m-beta-w32_bld-00000000000/build/obj-firefox/build'
c:/builds/moz2_slave/rel-m-beta-w32_bld-00000000000/build/config/recurse.mk:153: recipe for target 'libs' failed
mozmake.exe[4]: Leaving directory 'c:/builds/moz2_slave/rel-m-beta-w32_bld-00000000000/build/obj-firefox'
c:/builds/moz2_slave/rel-m-beta-w32_bld-00000000000/build/config/rules.mk:592: recipe for target 'default' failed
mozmake.exe[3]: Leaving directory 'c:/builds/moz2_slave/rel-m-beta-w32_bld-00000000000/build/obj-firefox'
c:/builds/moz2_slave/rel-m-beta-w32_bld-00000000000/build/client.mk:400: recipe for target 'realbuild' failed
mozmake.exe[2]: Leaving directory 'c:/builds/moz2_slave/rel-m-beta-w32_bld-00000000000/build'
c:/builds/moz2_slave/rel-m-beta-w32_bld-00000000000/build/client.mk:246: recipe for target 'profiledbuild' failed
mozmake.exe[1]: Leaving directory 'c:/builds/moz2_slave/rel-m-beta-w32_bld-00000000000/build'
client.mk:187: recipe for target 'build' failed
Executing: link -NOLOGO -OUT:crashinject.exe -PDB:crashinject.pdb -LARGEADDRESSAWARE -NXCOMPAT -RELEASE -DYNAMICBASE -SAFESEH -DEBUG -DEBUGTYPE:CV -DEBUG -OPT:REF @c:\builds\moz2_slave\rel-m-beta-w32_bld-00000000000\build\obj-firefox\build\win32\tmpwmu4ns.list module.res kernel32.lib user32.lib gdi32.lib winmm.lib wsock32.lib advapi32.lib secur32.lib netapi32.lib kernel32.lib user32.lib gdi32.lib winmm.lib wsock32.lib advapi32.lib secur32.lib netapi32.lib
c:\builds\moz2_slave\rel-m-beta-w32_bld-00000000000\build\obj-firefox\build\win32\tmpwmu4ns.list:
   crashinject.obj
LINK : fatal error LNK1123: failure during conversion to COFF: file invalid or corrupt

Full log here: http://buildbot-master84.srv.releng.scl3.mozilla.com:8001/builders/release-mozilla-beta-win32_build/builds/14/steps/compile/logs/stdio

I've (massimo) re-triggered this build and it has passed the LNK1123 point... still building.

mshal pointed out this: https://bugzilla.mozilla.org/show_bug.cgi?id=1026870

macosx64 repack issues:

repacks 2, 4, 5, 7, 8 and 10/10 have failed with:

 mv /builds/slave/rel-m-beta-osx64_rpk_8-0000000/mozilla-beta/obj-l10n/dist/l10n-stage/firefox/Firefox.app/Contents/Resources/en.lproj /builds/slave/rel-m-beta-osx64_rpk_8-0000000/mozilla-beta/obj-l10n/dist/l10n-stage/firefox/Firefox.app/Contents/Resources/ru.lproj
mv: rename /builds/slave/rel-m-beta-osx64_rpk_8-0000000/mozilla-beta/obj-l10n/dist/l10n-stage/firefox/Firefox.app/Contents/Resources/en.lproj to /builds/slave/rel-m-beta-osx64_rpk_8-0000000/mozilla-beta/obj-l10n/dist/l10n-stage/firefox/Firefox.app/Contents/Resources/ru.lproj: No such file or directory
make[1]: *** [repackage-zip] Error 1
make: *** [repackage-zip-ru] Error 2
command: ERROR
Traceback (most recent call last):
 File "/builds/slave/rel-m-beta-osx64_rpk_8-0000000/scripts/lib/python/util/commands.py", line 47, in run_cmd
   return subprocess.check_call(cmd, **kwargs)
 File "/tools/python27/lib/python2.7/subprocess.py", line 511, in check_call
   raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['make', 'installers-ru']' returned non-zero exit status 2 command: END (13.39s elapsed)

from irc: <mshal> mgerva: hmm, somehow the Firefox.app.tar.gz is corrupted...

• running stand alone repacks for rm ro ru (chunk 8/10)

win32 repack issues

Repacks failed for 2/10, 4/10, 5/10, and retriggered 5/10

• no locale started, so just restarted the entire builder
• details in bug 1057549
• all affected boxes were "just" returned to service today