User:Ewong/osxbustagereport

OSX Bustage Report

Recently, m-c moved to clang for their linux and osx builds. C-c wasn't ready for this change and when m-c changed build/macosx/common to:

  CC=$topsrcdir/clang/bin/clang
 CXX=$topsrcdir/clang/bin/clang++

it perma-oranged our Linux build due to checks that compares c-c's macosx/common and m-c's build/macosx/common.

and it busted SM's OSX builds due to our slaves not having clang.

Plus m-c also added tooltool to their releng, which we also don't have.

Anyway, taking it a step at a time, I thought I'd get a workaround for our OSX bustage until bug 775539 is fixed.

I firstly traced through the mozconfig calls. (Note: It's a confusing spaghetti code of a mess. ;/)

Taking an example OSX64 nightly bustage (10.6 nightly):

At first nightly gets the mozconfig: (This is from the log.)

  if [ -f "suite/config/mozconfigs/macosx-universal/nightly" ]; then
    echo Using in-tree mozconfig;
    cp suite/config/mozconfigs/macosx-universal/nightly .mozconfig;
 else
    echo Downloading mozconfig;
    wget -O .mozconfig http://hg.mozilla.org/build/buildbot-configs/raw-file/default/seamonkey/macosx64/comm-central-trunk/nightly/mozconfig;
 fi

It gets the in-tree mozconfig: suite/config/mozconfigs/macosx-universal/nightly which is just:

   1  if test -e "$topsrcdir/mozilla/build/macosx/universal/mozconfig"; then
   2   # We need some hackery to deal with the mozilla/ build system calling this
   3   # mozconfig.
   4   oldtopsrcdir=$topsrcdir
   5   export topsrcdir=$topsrcdir/mozilla
   6 else
   7   # Big Hack that unsets CC / CXX so that mozconfig.common doesn't get
   8   # mixed up with host/target CPUs when trying to work out how to do the
   9   # universal build. When we redo the build system (bug 648979) this will
  10   # go away.
  11   unset CC
  12   unset CXX
  13 fi
  14 
  15 . $topsrcdir/build/macosx/universal/mozconfig
  16 
  17 if test -n $oldtopsrcdir; then
  18   export topsrcdir=$oldtopsrcdir
  19 fi
  20 
    (some irrelevant stuff deleted for brevity)

Then it runs $topsrcdir/build/macosx/universal/mozconfig. (Note: $topsrcdir is actually $oldtopsrcdir/mozilla) This file :

    1 # This Source Code Form is subject to the terms of the Mozilla Public
    2 # License, v. 2.0. If a copy of the MPL was not distributed with this
    3 # file, You can obtain one at http://mozilla.org/MPL/2.0/.
    4 
    5 # i386/x86-64 Universal Build mozconfig
    6 
    7 # As used here, arguments in $MOZ_BUILD_PROJECTS are suitable as arguments
    8 # to gcc's -arch parameter.
    9 mk_add_options MOZ_BUILD_PROJECTS="i386 x86_64"
   10 
   11 . $topsrcdir/build/macosx/universal/mozconfig.common

So it runs $topsrcdir/build/macosx/universal/mozconfig.common, which in old terms is c-c's $topsrcdir/mozilla/build/macosx/universal/mozconfig.common, which is:

    1 # This Source Code Form is subject to the terms of the Mozilla Public
    2 # License, v. 2.0. If a copy of the MPL was not distributed with this
    3 # file, You can obtain one at http://mozilla.org/MPL/2.0/.
    4 
    5 mk_add_options MOZ_UNIFY_BDATE=1
    6 
    7 mk_add_options MOZ_POSTFLIGHT_ALL+=build/macosx/universal/flight.mk
    8 
    9 # Note, the version (10) is used by libffi's configure.
   10 ac_add_app_options i386 --target=i386-apple-darwin10
   11 ac_add_app_options x86_64 --target=x86_64-apple-darwin10
   12 
   13 ac_add_options --with-macos-sdk=/Developer/SDKs/MacOSX10.6.sdk
   14 
   15 . $topsrcdir/build/macosx/common
   16 
    (also some irrelevant stuff deleted)

This means it runs $topsrcdir/build/macosx/common ($oldtopsrcdir/mozilla/build/macosx/common) which is:

    1 export CC=$topsrcdir/clang/bin/clang
    2 export CXX=$topsrcdir/clang/bin/clang++
    3 
    4 ac_add_options --enable-stdcxx-compat

Then it goes back to the topmost call from suite/config/mozconfigs/macosx-universal/nightly:

   16
   17 if test -n $oldtopsrcdir; then
   18   export topsrcdir=$oldtopsrcdir
   19 fi
   20 

Which basically changes back topsrcdir to refer to c-c's topsrcdir. (Confusing..I know.)

So at this point, CC = topsrcdir/clang/bin/clang and CXX=topsrcdir/clang/bin/clang++. Which is where we're choking. So, right on line 19 of suite/config/mozconfigs/macosx-universal/nightly, we set:

    19  unset CC
    20  unset CXX
    21 fi

This should get our OSX builds to use gcc/g++ again.

Though I was a bit torn between that and the following:

    19  export CC=/usr/bin/gcc
    20  export CXX=/usr/bin/g++
    21 fi

Anyway, I used the first version in the workaround patch.

---

8X------------------------------------------

I've deliberately kept the above just to show workarounds don't really always work. :(

Long story short: The patch failed. Not because it didn't work. It did, but further along the compile process, the build choked because something within mozilla/ (specifically nspr) got its CC and CXX separately and used the mozilla/ way. All in all, that wasn't much of a waste of time, but a fail nonetheless.

Now I'm doing it the 'proper' way, in that I'm going through the 'tooltool' method. I have two patches (suite/ changes and buildbot-config/ changes). Putting them up for review. Follow the details in bug 775539.