Mobile/Fennec/Android old

Developing Fennec for Android

This page contains developer information about Firefox for Android, part of the Fennec project.

To download Firefox for Android or learn more about it, see the main Fennec for Android page.

Current Status

• Current bugs

Download Source Code

Check out the mozilla-central Mercurial repository.

hg clone http://hg.mozilla.org/mozilla-central

Install Dependencies

• Standard Linux build environment or Mac OS X build environment
• Android NDK - (See below) - This version is based on the original Android NDK, modified to include C++ standard libraries and sys/queue.h.
• Android SDK - (See below)

Set up a build environment

See http://blog.lassey.us/2010/07/09/android-development-vm/ for a preconfigured Ubuntu 10.04 VMWare VM.

Ubuntu 10.04/10.10 Desktop/Server

sudo apt-get install python-software-properties
sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"
sudo apt-get update
sudo apt-get build-dep firefox
sudo apt-get install sun-java6-jre sun-java6-plugin sun-java6-fonts sun-java6-jdk 
sudo update-java-alternatives -s java-6-sun

if Running on amd64 or ia64 (basically 64bit linux), install the ia32-libs package :

sudo apt-get install ia32-libs

Then install Android sdk and ndk :

wget http://dl.google.com/android/android-sdk_r11-linux_x86.tgz
tar xzf android-sdk_r06-linux_86.tgz
wget http://ftp.mozilla.org/pub/mozilla.org/mobile/source/android-ndk-r4c-0moz3.tar.bz2
tar xjf android-ndk-r4c-0moz3.tar.bz2
./android-sdk-linux_x86/tools/android update sdk

Fedora 13

yum install \
 cvs zip gcc-c++ glib2-devel gtk2-devel fontconfig-devel libnotify-devel \
 libIDL-devel alsa-lib-devel libXt-devel freetype-devel pkgconfig \
 dbus-glib-devel curl-devel autoconf213 xorg-x11-proto-devel libX11-devel \
 libXau-devel libXext-devel wireless-tools-devel glibc-static yasm

Install java: http://wporta.wordpress.com/2010/07/06/installing-jdk-in-fedora-13/

Then install Android sdk and ndk :

 wget http://dl.google.com/android/android-sdk_r06-linux_86.tgz
 tar xzf android-sdk_r06-linux_86.tgz
 wget http://ftp.mozilla.org/pub/mozilla.org/mobile/source/android-ndk-r4c-0moz3.tar.bz2
 tar xjf android-ndk-r4c-0moz3.tar.bz2
 ./android-sdk-linux_86/tools/android update sdk

Fedora 14 x64

If you're running Fedora 14 x64, in addition to the Fedora 13 requirements above, you may also need to install some or all of zlib-devel.i686 glibc.i686 glibc-devel.i686 libstdc++.i686

Quickstart (Linux)

XXX it's not clear to me if this section is a redundant with the above? XXX

Download the SDK and NDK:

wget http://dl.google.com/android/android-sdk_r06-linux_86.tgz
wget http://ftp.mozilla.org/pub/mozilla.org/mobile/source/android-ndk-r4c-0moz3.tar.bz2

Then unpack them:

tar xvfj android-ndk-r4c-0moz3.tar.bz2
tar xvfz android-sdk_r06-linux_86.tgz

Install Java runtime and JDK:

sudo apt-get install sun-java6-jre sun-java6-plugin sun-java6-fonts sun-java6-jdk

Ubuntu note: be sure to install the Sun or OpenJDK version of Java, gcj will not work. In Ubuntu 10.04 Sun Java was moved to the Partner repository. Open the Software Center, edit Software Sources and enable the Partner repository.

Android SDK needs to be setup manually.

• In the SDK directory, run tools/android
• Check Settings->Force https://... sources to be fetched using http://...
• Click Save & Apply
• Installed Packages->Update All...
• Select at least the Android SDK, API level 6. Optionally you can install API levels 4 and 8. XXX it wasn't clear to me what's meant here. Do you mean in 'Installed Packages' or 'Available Packages'? there is no 'Android SDK, API level 6' here, anyway. XXX
• Click Install, Accept, etc.

Quickstart (Mac)

There's no preconfigured NDK available for Mac, so the process is a little less straightforward.

• Download Android SDK and unpack it
• from the console, run

 $ /path/to/sdk/tools/android update sdk

and install the recommended packages

• Download Crystax NDK and unpack it
• using the mozconfig from below, add

 ac_add_options --disable-webm
 ac_add_options --disable-crashreporter

• in the console, perform the following

 cd /path/to/android-ndk-r4-crystax
 ln -s build/platforms/android-5/arch-arm/usr/include/sys/queue.h build/platforms/android-8/arch-arm/usr/include/sys/queue.h

You should now be able to type |make -f client.mk build| in your srcdir and watch it build without problems.

Build Fennec for Android

Full Build

Normal Firefox build, with the following mozconfig. (Put this in a file called "mozconfig" in your top-level source directory.)

OBJDIR=objdir-android
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/$OBJDIR

# Add the correct paths here:
ac_add_options --with-android-ndk="ABSOLUTE/PATH/TO/android-ndk-r4c"
ac_add_options --with-android-sdk="ABSOLUTE/PATH/TO/android-sdk-linux_86/platforms/android-8"

# android options
ac_add_options --enable-application=mobile
ac_add_options --target=arm-android-eabi
ac_add_options --with-endian=little

# Uncomment to run on ARMv5 or ARMv6 devices, including the emulator.
# Note that on devices that do *not* need this, applying it may cause
# odd problems (like crashes or hangs).
#ac_add_options --disable-thumb2

ac_add_options --disable-tests

export MOZ_DEBUG_SYMBOLS=1

A standard build should succeed with those paths.

After that build finishes, running make package in the OBJDIR should generate a file in the dist directory that looks like fennec-8.0a1.en-US.android-arm.apk. This file can be installed on an Android device with adb install [apk name here]. To uninstall it via adb, use the package name: adb uninstall org.mozilla.fennec_unofficial.

You'll probably have to add a udev rule to set permissions on the usb device, see this page, and if that doesn't work, try using a very high priority number for your rule, for example here on debian, to get a Samsung phone to work, I created

 /etc/udev/rules.d/98-android.rules

with this contents:

 SUBSYSTEM=="usb", SYSFS{idVendor}=="04e8", MODE="0666"

Using adb

To install the apk as just mentioned, you need to run adb. The first time you do this, you will need to follow the instructions for your device and operating system at Developing on a Device.

Note for VMWare fusion users: Certain devices like the Nexus S USB Driver will not work, you will need to follow:

• http://communities.vmware.com/thread/292049
• In particular shutdown your VM, open your vm package, find the devices's vid/pid in vmware.log, add |usb.quirks.device0 = "0xvid:0xpid skip-reset"| with vid/pid replaced to your .vmx file and reboot the VM.

If the udev rules from that page do not work and you get a permission error, you can run the adb server as root instead:

 sudo ./adb kill-server
 sudo ./adb start-server

JS/NSPR only

In the commands below, replace these with the appropriate paths:

• $NDK is the NDK location
• $moz is the mozilla-droid repo checkout
• $out is some base destination directory

First, regenerate configure in the js dir (the nspr configure is checked in):

cd $moz/js/src && autoconf2.13

Then create a nspr directory, configure nspr, and build it:

cd $out
mkdir nspr
cd nspr
$moz/nsprpub/configure \
  --target=arm-android-eabi \
  --with-android-ndk=$NDK 
make -s

Then do the same for JS, telling it where to find the NSPR you just built:

cd $out
mkdir js
cd js
$moz/js/src/configure \
  --target=arm-android-eabi \
  --with-android-ndk=$NDK \
  --with-nspr-cflags=-I$out/nspr/dist/include/nspr \
  --with-nspr-libs='-L$out/nspr/dist/lib -lnspr4 -lplc4 -lplds4' \
  --enable-threadsafe \
  --with-endian=little \
  --with-arm-kuser
make -s

More information about NSPR is on this separate page.

Run JS/NSPR on Android

You'll need to copy the NSPR libraries and the js shell to the emulator or your device. Whether you're running on a physical device or an emulator, /data/local should be writable by the user.

cd $out/nspr/dist/lib
for f in *.so ; do adb push $f /data/local ; done
cd $out/js/shell
adb push js /data/local

Then, connect to a shell on the device, and run js (adb shell prompts prefixed with "android"):

% adb shell
android% cd /data/local
android% LD_LIBRARY_PATH=. ./js
js> 1+1
2

Debugging with GDB

Note that android writes breakpad-like crash information to logcat, at I/DEBUG. This can be useful for filing bug reports, and doesn't require setting up gdb. The output looks like

I/DEBUG   ( 2196): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
I/DEBUG   ( 2196): Build fingerprint: 'Samsung/SGH-T959/SGH-T959/SGH-T959:2.1- update1/ECLAIR/UVJFD:user/release-keys'
I/DEBUG   ( 2196): pid: 11834, tid: 11834  >>> /data/local/js/js <<<
I/DEBUG   ( 2196): signal 11 (SIGSEGV), fault addr e3c43162
...

Use the correct build

Fennec on android uses its own library loader. This loader and gdb don't get along: gdb doesn't work correctly with loaded through our dlopen.

Our loader is disabled in debug builds. In opt builds, for now you need to manually flip the extractLibs variable in APKOpen.cpp to "1".

Preparation

Debugging with gdbserver requires root access. If you have not rooted your phone, search the web for instructions for your particular device. Or see notes on debugging without root access.

Using gold linker (optional)

The gold linker will let you link faster. If you plan on doing a lot of debugging this is recommended, otherwise you can skip this step.

See BenWa's blog: Faster Linking For Fennec

Using nVidia version (recommended)

Need to document here. See User:Blassey/Notes/Android for how to install.

Note that you'll still need to pull the system libraries as described in the next section -- don't skip it entirely just because you've gotten the NVIDIA gdb.

Using the NDK version

NOTE: Fennec might crash when attaching using the gdb binaries included in the Google NDK. nVidia's gdb binaries are better.

If you do not already have gdbserver on your device (use adb shell gdbserver to check), you can push a pre-built version from the NDK.

adb push /PATH/TO/android-ndk-r4/build/prebuilt/linux-x86/arm-eabi-4.4.0/bin/gdbserver /data/local

Copy the necessary files from your device via adb pull. /system/bin/linker is referred to by absolute path so setting solib-absolute-prefix in gdb will be necessary for it to be found. Other files can be placed anywhere as long as they can be found in solib-search-path.

In short, find a directory to put some libraries, make sure adb is in your path, and paste this into your terminal:

adb pull /system/bin/linker . &&
adb pull /system/bin/app_process . &&
mkdir -p system/bin &&
mv linker system/bin &&
for LIB in libc.so libm.so libstdc++.so liblog.so libz.so libGLESv1_CM.so
do
adb pull /system/lib/$LIB .
done

Attach GDB

• launch in debug mode:

 (Host)$ adb shell am start -a org.mozilla.gecko.DEBUG \
                            -n org.mozilla.fennec/org.mozilla.fennec.App

• Forward a port for gdb between your device and computer using adb. Any port you can open should work. 1234 is used here.

 (Host)$ adb forward tcp:1234 tcp:1234

• Find the pid of your process if you don't know it. You can look at the second column of adb shell ps|grep fennec.
• Attach gdbserver to the process

 (Host)$ adb shell /data/local/gdbserver localhost:1234 --attach YOURPID
 Attached; pid = YOURPID
 Listening on port 1234

• Run arm-eabi-gdb on your binary. (android-ndk-1.6_r1/build/prebuilt/linux-x86/arm-eabi-4.2.1/bin) For debugging gecko, a copy of app_process from your device should be used. (Note that app_process is not a variable name -- it is actually a binary pulled from the device in the previous section.)

 (Host)$ /PATH/TO/NDK/build/prebuilt/linux-x86/arm-eabi-4.2.1/bin/arm-eabi-gdb app_process
 GNU gdb 6.6
 Copyright (C) 2006 Free Software Foundation, Inc.
 GDB is free software, covered by the GNU General Public License, and you are
 welcome to change it and/or distribute copies of it under certain conditions.
 Type "show copying" to see the conditions.
 There is absolutely no warranty for GDB.  Type "show warranty" for details.
 This GDB was configured as "--host=x86_64-unknown-linux-gnu --target=arm-elf-linux"...

• Configure solib-absolute-prefix. This will point to the directory where you unpacked your system image or where you copied the libraries.

 (Host gdb) set solib-absolute-prefix ANDROIDLIBS

• Configure solib-search-path. This needs to point to the directories you need debugging symbols from - /system/lib or where ever you copied the libraries.

 (Host gdb) set solib-search-path ANDROIDLIBS:ANDROIDLIBS/system/lib:OBJDIR/dist/bin

• Connect to gdbserver

 (Host gdb) target remote localhost:1234

• These three Host gdb commands can be placed in a .gdbinit file to automate future gdb runs. .gdbinit should be located where you normally run gdb.

Automating gdbserver setup

You can use the script below to setup gdbserver:

#!/bin/bash
PORT=1234
PID=""

# By default, this will attach to the parent process.
# Use "plugin-container" as argument to attach to child.
if [ -z $1 ]; then
  GREP=org.mozilla.fennec
else
  GREP=$1
fi

adb forward tcp:$PORT tcp:$PORT &&
adb shell am start -a org.mozilla.gecko.DEBUG -n org.mozilla.fennec/org.mozilla.fennec.App &&
while [ -z $PID ]; do
  PID=`adb shell ps | grep $GREP | head -n 1 | cut -c11-16`
done &&
adb shell run-as org.mozilla.fennec /data/local/gdbserver localhost:$PORT --attach $PID

Debugging on Galaxy S devices

These include AT&T Galaxy S, Tmobile Vibrant, Sprint Epic.

Unfortunately, it appears that android v2.1 denies non-root programs the ptrace capability, which means that non-root gdb (and breakpad!) won't work there. So if you have 2.1, you need to root first.

• T-mobile Vibrant
•  ???

To ensure that your device is rooted (at least partially), run the following commands while the home screen is unlocked: you'll need to grant superuser permissions on the device itself.

adb shell
su

If you get a # prompt, you're good to go. If you get 'Permission Denied' you might need to have the 'Superuser' app running on your phone and manually grant access (don't forget to check remember!).

Next find a good directory for your gdb install and set up gdb. The dir where you installed the NDK/SDK works well. Let's call that dir ~/android

cd ~/android
mkdir gdb && cd gdb
wget http://developer.download.nvidia.com/tegra/files/tegra-gdb-20100902.zip
unzip tegra-gdb-20100902.zip
rm -f debug.sh and install.sh

tegra-gdb ships with install.sh and debug.sh scripts, but they don't work with the root above for the T-mobile Vibrant. So we'll create our own ~/android/gdb/install.sh.

chmod +x prebuilt/linux-x86/arm-eabi-gdb
adb push prebuilt/gdbserver /data/local
adb shell chmod 755 /data/local/gdbserver
mkdir lib && cd lib
adb pull /system/bin/linker .
adb pull /system/bin/app_process .
mkdir -p system/bin
mv linker system/bin
for file in $(adb shell ls /system/lib | tr "\n" " " | tr "\r" " "); do
    adb pull /system/lib/$file .
done

Then

bash install.sh

Now you're ready to start gdbserver and gdb. The tegra-gdb has a bug in which it apparently can't find threads created before it attaches to a process. This sounds like an easy bug to fix, but a workaround is to use the DEBUG intent as follows.

adb forward tcp:12345 tcp:12345
adb shell
su
am start -a org.mozilla.gecko.DEBUG -n org.mozilla.fennec/org.mozilla.fennec.App

On device, you should see a button that says "Launch". Fennec is now waiting for you to connect with gdb. Back in the adb shell

ps
# you should see a line like
#   app_95    7549  2204  248356 26264 ffffffff 83844aa8 D org.mozilla.fennec
# the first number is the PID
/data/local/gdbserver localhost:12345 --attach 7549

If that goes well, gdbserver should print something like

Attached; pid = 7549
Listening on port 12345

You're now ready to connect a local gdb to the gdbserver on device. We'll pretend that you installed tegra-gdb and the device libs into /home/cjones/android/gdb, and that your objdir is /home/cjones/mozilla/ma-opt/dist/bin. Then in a new host shell (or emacs gdb), point gdb at the right symbols by running

~/android/gdb/prebuilt/linux-x86/arm-eabi-gdb ~/android/gdb/lib/app_process
# (in gdb)
set solib-absolute-prefix /home/cjones/android/gdb/lib
set solib-search-path /home/cjones/android/gdb/lib:/home/cjones/android/gdb/lib/system/bin:/home/cjones/mozilla/ma-opt/dist/bin

You very much want to add this setup to a .gdbinit script. Finally, connect to the gdbserver

target remote localhost:12345

If all goes well, you should see something like

(gdb) target remote localhost:12345
Remote debugging using localhost:12345
Reading symbols from /home/cjones/android/gdb/lib/system/bin/linker...(no debugging symbols found)...done.
Loaded symbols for /home/cjones/android/gdb/lib/system/bin/linker
Reading symbols from libc.so...(no debugging symbols found)...done.
Loaded symbols for libc.so
[snip]
0xafe0d9c4 in __futex_wait () from libc.so
(gdb) bt
#0  0xafe0d9c4 in __futex_wait () from libc.so
#1  0xafe0f444 in __pthread_cond_timedwait_relative () from libc.so
#2  0xafe0f4d0 in __pthread_cond_timedwait () from libc.so
#3  0xad0402d6 in ?? () from libdvm.so
#4  0xad0402d6 in ?? () from libdvm.so
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

The final warning about a corrupt stack appears to be a quirk of the Dalvik VM's implementation, and isn't worrisome because the Java frames would be meaningless in C++ anyway.

To connect to a content process, the steps are mostly the same, except you want to start fennec with

am start -a org.mozilla.gecko.DEBUG -n org.mozilla.fennec/org.mozilla.fennec.App --es env0 MOZ_DEBUG_CHILD_PROCESS=1

attach to a process that looks like

app_95    16029 16005 20488  5852  c03b08dc afe0d0ac S /data/data/org.mozilla.fennec/plugin-container

and start android-gdb with

~/android/gdb/prebuilt/linux-x86/arm-eabi-gdb $objdir/dist/bin/plugin-container

Running tests on device

It's probably going to be painful, prepare thyself.

JS trace-tests

A rooted device isn't required to run trace-tests. One may or may not be required to debug test failures, see debugging instructions bove.

Install the "android scripting environment" (sl4a_r*.apk) and the python interpreter (python_for_android_r*.apk) from here.

update from blassey -- run the python app that gets installed and then click the install button from there

Set up a local staging directory for files to be pushed to device. We'll push all the junk to /data/local/js/.

mkdir testjs
cd testjs
wget https://bugzilla.mozilla.org/attachment.cgi?id=480264 -O - > testjs.py
mkdir js
cp $objdir/js/src/js \
   $objdir/memory/mozalloc/libmozalloc.so \
   $objdir/nsprpub/pr/src/libnspr4.so \
   $objdir/nsprpub/lib/libc/src/libplc4.so \
   $objdir/nsprpub/lib/ds/libplds4.so \
   js/
cp -R $srcdir/js/src/trace-test js/trace-test
adb shell mkdir /data/local/js
adb shell chmod 777 /data/local/js
adb push js /data/local/js
adb push testjs.py /sdcard/sl4a/scripts/

update from blassey -- you also need libmozutils.so

adb push ../dist/bin/libmozutils.so /data/local/js

Time for a sanity check. Try running

adb shell LD_LIBRARY_PATH="/data/local/js" /data/local/js/js

You may or may not get a "> " prompt (and if you do, it's broken anyway), but if you don't see any error messages you're probably good to go.

testjs.py is a wrapper script for trace_test.py (and soon other tests). We'll run testjs.py using the python interpreter. If the scripting environment wasn't broken, you could run the tests with

$ am start -a com.googlecode.android_scripting.action.LAUNCH_TERMINAL -n com.googlecode.android_scripting/.activity.ScriptingLayerServiceLauncher -e com.googlecode.android_scripting.extra.SCRIPT_NAME testjs.py

(lol! java). But it is broken. So instead, you need to launch the "SL4A" application and then click "testjs.py". Choose the "terminal" icon to run the script in the built-in terminal. You should then see trace-test's usual progress meter.

When all the tests finish, the failed tests and their output will be in the file /data/local/js/trace-test-failures.txt (it gets stomped on every test run, beware). You may need to mark the /data/local/js directory as writable by all in order for the script to sae its output. See the results with

$ adb shell cat /data/local/js/trace-test-failures.txt
basic/testComparisons.js
Exit code: -11  # oops, segfault
...

To debug a particular test, use the newly-added js -g option, which has jsshell sleep for 30 seconds on startup, and then attach with gdb using the instructions above.

adb shell LD_LIBRARY_PATH="/data/local/js" /data/local/js/js -g 30 -j -e "const platform='linux2'; const libdir='/data/local/js/trace-test/lib/';" -f /data/local/js/trace-test/lib/prolog.js -f /data/local/js/trace-test/tests/basic/testComparisons.js
# attach using gdb instructions for your device

You run trace-test with custom options by editing testjs.py. For example, to run with -mj, add

 trace_test.main([
     JS,
     '--avoid-stdio',
+    '--jitflags', 'mj',
     '--write-failure-log', failures_file
 ])

then

adb push testjs.py /sdcard/sl4a/scripts/.

and re-run using the instructions above.

Running Tests The Way Tinderbox Does

To run the tests standard battery of tests, there are several options. You can run them using adb, the on-device test agent, from a build environment, or using only packaged builds. We have dedicated an entire page to it. Go check it out.

Profiling

There are ways to compile and run oprofile and PowerTOP

Android Development Tips

• You can build a full Android OS from source following the instructions on Cyanogen's site. This helps tremendously with debugging.
• Running in the Android SDK emulator requires a build with --disable-thumb2. You might also need to change the emulator options to give it enough space to install and run Fennec: emulator -partition-size 256 -memory 512
• Pass command line arguments to Fennec as "args" extras using the --es argument: adb shell am start -n org.mozilla.fennec/org.mozilla.fennec.App --es args -safe-mode
• You can reinstall fennec by using adb install -r ...
• CheckJNI can help by warning about JNI errors.

Signing non-local builds with your local key

Nightly builds are available unsigned, so that you can sign them with your local debug key and install them on top of your own debug builds (without uninstalling and losing your profile). To sign and install the unsigned nightly build:

wget http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/latest-mozilla-central-android-r7/gecko-unsigned-unaligned.apk
jarsigner -keystore ~/.android/debug.keystore -storepass android -keypass android gecko-unsigned-unaligned.apk androiddebugkey
zipalign -f -v 4 gecko-unsigned-unaligned.apk gecko-signed-aligned.apk
adb install -r gecko-signed-aligned.apk

Or you can also re-sign a signed build. If "fennec.apk" is signed already, this will remove the signature and replace it with your own. The result will be saved as "fennec-resigned.apk":

zip fennec.apk -d META-INF/*
jarsigner -keystore ~/.android/debug.keystore -storepass android -keypass android fennec.apk androiddebugkey
zipalign -f -v 4 fennec.apk fennec-resigned.apk

There's a handy script to do this at: http://people.mozilla.com/~mbrubeck/re-sign-apk.sh

Passing in environment variables to Fennec

adb shell am start -a org.mozilla.fennec -n org.mozilla.fennec/org.mozilla.fennec.App --es env0 KEY=VALUE

Updates

• Updates check implementation

Other Resources

• Joel's Notes
• Vlad's Notes
• Vlad's Test App Hg Repo