Penelope Developer Page

Building Thunderbird

• Build Thunderbird. If you're a coder, go fetch and build Thunderbird. Start learning how it works.

Building Thunderbird 3.0 on Linux from CVS

These were the steps followed to build Thunderbird 3.0a from CVS on linux. These instructions assume an up to date linux machine. The machine I used is running Gentoo Linux 2006.1. More thunderbird linux build details can be found at [developer.mozilla.org]

• Create a new directory and cd into it

• cvs -d :pserver:anonymous:anonymous@cvs-mirror.mozilla.org:/cvsroot co mozilla/client.mk

• cvs -d :pserver:anonymous:anonymous@cvs-mirror.mozilla.org:/cvsroot co mozilla/mail/config/mozconfig

• cd mozilla

• make -f client.mk checkout MOZ_CO_PROJECT=mail

• Create ~/.mozconfig with:

   . $topsrcdir/mail/config/mozconfig
   ac_add_options --enable-default-toolkit=gtk2
   ac_add_options --enable-xft
   ac_add_options --disable-tests

• make -f client.mk build

• Go get breakfast/lunch/dinner because it took over 2 hours on my machine.

Building Penelope on OS X

The steps to build Penelope from CVS on OS X are almost identical to the instructions for Linux. There are may caveats depending on whether you want a universal binary, which version of OS X you have, which version you want to target, etc. More Thunderbird OS X build details can be found at [developer.mozilla.org].

The only deviation from the Linux directions is the installation of LibIDL and GLib, and contents of the ~/.mozconfig file.

• Rememer to do the following:

   chmod +x ./configure
   chmod +x ./build/autoconf/mozconfig*

• Install LibIDL (via orbit) and GLib using fink:

   $ sudo apt-get update
   $ sudo apt-get install orbit orbit-dev glib

• Create ~/.mozconfig with:

   export CC=gcc-3.3
   export CXX=g++-3.3
   . $topsrcdir/mail/config/mozconfig
   ac_add_options --enable-prebinding
   ac_add_options --disable-tests
   ac_add_options --enable-extensions=default,penelope

Note that it is this last line that includes the building the Penelope extension in the main build. The source code for Penelope can be found in mozilla/penelope.

Building Penelope on Windows

See the steps above under Linux for how to get the source code via CVS.

Penelope uses the version of the Mozilla platform before it switched to using the MozillaBuild environment. Make sure you go through in detail the setup documentation for Cygwin. There's lots of environment variables to create and some tools to download and configure. They recommend having a batch script to set up the environment, but you can permanently set all the environment variables so that you can compile from any command prompt and don't have to remember to run the script.

The Windows build prerequisites result in a setup that will not build. The current version of the "ash" component ("Base" category) is 20040127-3. This version is not compatible with the current Mozilla build - you need to go back to 20040127-1. As it turns out, there is only one file in the "ash" package that seems to make a difference in the build: sh.exe.

Here's the options for your %HOME%\.mozconfig file in order to make a debug build:

   . $topsrcdir/mail/config/mozconfig
   ac_add_options --disable-optimize 
   ac_add_options --enable-debug 
   ac_add_options --disable-static --enable-shared
   ac_add_options --disable-tests
   ac_add_options --enable-extensions=default,penelope

Finally, go to the root of the source tree and do a:

• make -f client.mk build

Visual Studio .NET 2003 and Service Pack 1

You must install Service Pack 1 in order to successfully build. You can find Service Pack 1 here. There's one aspect to installing the Service Pack that can be somewhat tricky. At a point during the install of the Service Pack it will ask you for the vs_setup.msi file of the original Visual Studio installer. You must have the exact original installer that you personally used, otherwise the Service Pack upgrade won't continue on. The Service Pack installer is woefully slow, and has bad UI as well. Know that if you run it there will be places where there are long pauses (I had one that was around 15 minutes). The release notes for the SP detail this.

Windows Vista

Windows Vista provides some roadblocks in compiling Thunderbird. The first is the new security model. By default, administrator accounts under Vista run applications at a reduced privilege level called Standard User. Microsoft found that the most common legitimate reason to need administrator rights was when installing an application, and in order for all the existing installation programs to work under Vista they had to come up with a heuristic workaround. What Vista does is automatically attempt to elevate the privilege of any application that has the words "install" or "setup" in the name of the executable filename. I say "attempt" because there appear to be some situations where the privilege elevation fails, and one of those happens to be when called inside of scripts. I think it has to do with the way the process is created (privilege elevation happens when you call ShellExecute(), but not when CreateProcess() is used), but I'm not sure of that entirely because I don't know how Cygwin shells start up new processes.

And wouldn't you know it, one of the utilities used to compile Thunderbird falls in to this heuristic: nsinstall.exe. All that command-line tool really does is copy some files, which doesn't require Administrator privileges as long as you have have write access to the destination directory, but Vista's automatic privilege elevation thinks it might due to the name. The privilege elevation fails and so the build fails. Happy, happy, joy, joy.

There's two current workarounds. One is that the Command Prompt window that you use to compile Thunderbird can be run as Administrator. You can create a shortcut setting that up (its in the Properties, Compatibility tab), or you can right-click on a link to a Command Prompt (e.g. one that shows up on the Start menu) and select "Run as Administrator". This is an unsatisfying solution because it raises the privilege level unnecessarily, but it does work. It does happen to solve other permission-related issues with files in general, so it is an easy way to fix the problem. It's also no different than the security model of Windows before Vista (well, at least the way that most people set it up, which is to run as Administrator all of the time).

The other way is to tell Vista that the app doesn't really need Administrator privileges. You can do this via a manifest file, which is an XML file which has properties about the application. It has the same name as the executable (including the ".exe"), but with an extra extension of ".mainfest" to it. The manifest file contents should look like this:

   <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
   <assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
     <assemblyIdentity version="1.0.0.0"
        processorArchitecture="X86"
        name="nsinstall"
        type="win32"/>
   
   
     <description>Description of your application</description>
     <trustInfo xmlns="urn:schemas-microsoft-com:asm.v3">
       <security>
         <requestedPrivileges>
           <requestedExecutionLevel
             level="asInvoker"
             uiAccess="false"/>
           </requestedPrivileges>
          </security>
     </trustInfo>
   </assembly>

The pertinent part that makes it work is the level="asInvoker" attribute, which tells Vista that it doesn't need any extra privileges to run. There's one snag to this, which is that if you've already run nsinstall.exe without the .manifest file, then Vista has already put the perceived needed privilege elevation in the fusion cache. In order to clear the cache you need to reboot Vista.

There is a third way to workaround the problem, but I don't like the long-term implications of it. You could rename the nsinstall.exe utility to something that doesn't have "install" or "setup" in the name and then change the build scripts to that effect. However, that means keeping a branch of the build scripts forever and that would probably cause more headaches down the road.

Reducing Build Time

The Thunderbird folks advise building in a subdirectory or subdirectories when making a change to the code to reduce the amount of time needed to build. By a narrowing down the build to at least the component you need to rebuild you can cut the compile time dramatically. You can cut down the build time even further by specifically providing the change logic yourself and rebuilding only those portions of the code that you know you changed.

If you use a separate object directory (as the default config suggests), then you'll need to look for the individual makefiles in the object directory hierarchy (for example in my setup I found them under "thunderbird-static" since that's my object directory name). Otherwise the makefiles will be alongside the source.
Scott MacGregor notes:

I don't use the objdir technique, but some developers really like it.
Most of those folks like to use it because they are building multiple
products like firefox and thunderbird and they only want to have one
source tree directory. They can just point their thunderbird build at
one objdir directory and the fx build at another, while working out of
just one instance of the source tree. But I mostly just build thunderbird,
so moving the objdir out of my source tree doesn't help me much.

Specific Example

A lot of the core Thunderbird functionality is provided in mail.dll, which is built from the mailnews directory.

For example, say I wanted to make a change to a portion of the IMAP code (e.g. I want to modify "nsIMAPNamespace.cpp").

Slow Method

The really slow method to recompile would be to invoke:

make -f client.mk build

in the root level of the mozilla source code. On my laptop, even with no changes to the source code, this takes over 12 minutes to complete.

Faster Method

A faster way to recompile would be to invoke "make" inside the mailnews directory (no arguments are needed for "make" since the mailnews directory contains "Makefile"). Without an object directory specified that would be mozilla/mailnews; with an object directory specified that would be something like mozilla/objectdir/mailnews.
On my laptop, this took less than 2 minutes to recompile.

Fastest Method

The fastest way to recompile would be to first invoke "make" inside the mailnews/imap directory.
Then invoke "make" inside the mailnews/build directory (which just re-links mail.dll).
On my laptop, this took less than 10 seconds to recompile (ignoring time for me to type, change directories, invoke the 2nd command etc.)

Troubleshooting

Here are some problems that have cropped up during development:

• The prefs.js file in the profile directory may have stale settings in it which override the behavior you are trying to test. You must remove them manually.
  • On Linux the profile directory is in ~/.thunderbird/
  • On Mac OSX the profile directory is in ~/Library/Thunderbird/Profiles/
  • On windows the profile directory is C:\Documents and Settings\<Windows login/user name>\Application Data\Thunderbird\Profiles\
• The extensions subdirectory may contain the penelope extension if it was installed for Thunderbird. This will conflict with an install of Eudora because the penelope extension resides in the Eudora install directory. Both extensions will be applied at the same time which will cause problems. If you install Eudora, you must remove any previous install from the profile directory
• Beware cross-volume symlinks (see Discussion page)
• Very bad things happen when you have an unresolved entity (missing a DTD file or entry in a DTD file). This can manifest as the whole application is a window with zero size (which is very hard to find!).
• Multiple key mappings occur when they are scoped in different keysets. When redefining a key mapping, make sure it is done in the correct keyset ID.
• Some key mappings canot be set via the prefs file. They seem to be dynamically updated via a xul overlay at run time. The only way I have found to redefine these keys is to create an overlay that remaps the keys.
• If the resulting build in dist/bin will not run try adding execute permission to dist/bin/run-mozilla.sh
• Some developers have had problems when the root source directory was not named "mozilla" (e.g. not being able to use MOZ_OBJDIR to put generated files in to a separate directory from the source). This is unfortunate because our trunk in Perforce is named "eudora", which means you need to set up a client-spec view to rename the directory locally. Also bad because there already is a directory named "mozilla" at that same level, which contains the original code from Mozilla.org.
• To re-create the Makefiles you have to remove the generated configure files then re-run make:

   rm config.log config.cache config.status mozilla-config.h
   make -f client.mk build