Mobile/Fennec/Android

From MozillaWiki
< Mobile‎ | Fennec(Redirected from Android)
Jump to: navigation, search

Contents

Building Fennec

Preparing your build environment

Run either of the following in a terminal. If you have curl installed (note that Mac OS X ships curl by default):

curl -O https://hg.mozilla.org/mozilla-central/raw-file/default/python/mozboot/bin/bootstrap.py && python bootstrap.py

If you have wget installed:

wget -q https://hg.mozilla.org/mozilla-central/raw-file/default/python/mozboot/bin/bootstrap.py && python bootstrap.py

(Alternatively, if you already have a clone of the source code, you can also run

mach bootstrap

to run the setup script.)

Choose to build "Firefox for Android" at the prompts. This will help you install all the build prerequisites required for building Fennec on the most popular host operating systems (including OS X and Debian-flavoured Linuxes, including Ubuntu; but sadly not including Windows).

You may need an hour or more to download and install prerequisites (Java, the Android SDK and NDK, and the Android platform itself, are very large). You may be prompted to enter your password or to accept license agreements.

NOTE: If that doesn't work, or you need more details: Follow the detailed instructions to set up a build environment on your machine. Once you have done that, follow the steps to get the source, set up your mozconfig, and build Fennec.

Getting the source

First, grab a clone of the Fennec source code repository:

hg clone http://hg.mozilla.org/mozilla-central/

The source files will be in the mozilla-central directory, which we refer to as your source directory.

Before doing anything else, it's a good idea to install and configure Mercurial as described here.

Preparing a Fennec mozconfig

Next, navigate to your source directory and create a text file called "mozconfig". The mozconfig file is what tells the Mozilla build system what to build and how your build environment is prepared. Here is the minimal mozconfig file for building Fennec; it should be named mozconfig in your source directory, as the build scripts will read from the mozconfig file in your source directory by default, although you can configure this if you need to.

# Build Firefox for Android:
ac_add_options --enable-application=mobile/android
ac_add_options --target=arm-linux-androideabi

# With the following Android SDK and NDK:
ac_add_options --with-android-sdk="/path/to/android-sdk/platforms/android-21"
ac_add_options --with-android-ndk="/path/to/android-ndk-r8e"

This assumes you have followed the instructions above and are using Android platform 21 and NDK version r8e. You must use $HOME instead of ~ in your mozconfig because ~ does not get expanded.

Mozilla's build system writes all build intermediate files and output artifacts into a separate directory called the object directory. The build system will choose a reasonable default name for the object directory if you don't specify one, but it's common to specify an object directory starting with "obj" and to put it inside your source directory. Add a line like the following to your mozconfig:

mk_add_options MOZ_OBJDIR=./objdir-droid

Building and deploying the Fennec Android package

Before you start building, connect your Android device and enable USB debugging.

The following steps will build Fennec (compile and link all C/C++, Java, and JavaScript source code), prepare a fresh Fennec APK, and deploy Fennec to your Android device:

./mach build
./mach package
./mach install

The APK file can be found in your objdir-droid/dist folder, and will be called something like fennec-28.0a1.en-US.android-arm.apk. You can install this APK to your device manually using adb install or ./mach install.

The name of the app that shows up on your phone will be "Fennec $USER" (where $USER is the username under which you built the code).

To speed up subsequent builds you should build only part of the mobile/android directory using ./mach build mobile/android or ./mach build mobile/android/base, depending on where your changes are.

Creating commits and submitting patches

Mercurial (hg) is the the main version control system that Mozilla uses, and to submit code for review, you should be able to generate patches or commits. Be sure to run mach mercurial-setup when cloning the source code repository -- this will ensure you have good default settings and extensions.

Historically, Mozilla has used Mercurial patch queues to submit code changes for review. However, Mozilla developers are now encouraged to use a Mercurial bookmark-based workflow.

If you're a contributor and new to Mercurial, patch queues are conceptually simple and fine for getting your first few patches up; see the instructions on Mercurial patch queues.

If you're interested in using bookmarks, take a look at bookmarks-based development.

After writing the patch, make sure the commit message is of the correct format:
Bug <bug#> - <bug title>. r=<reviewer>

So for example, for bug 1128431, the commit message should be:
Bug 1128431 - 'Start browsing' link from onboarding v1.5 is not visible on small screen devices. r=liuche

After uploading a patch, set the r? flag on the bug (to the mentor or a relevant reviewer) to get review on the patch. If the patch isn't quite ready, the f? flag can be set to request feedback from someone.

Once the patch gets an r+ on review, if you don't have tree privileges, ask someone to land it for you; or, set checkin-needed in the keywords field AND ask someone to push it to the try server.

(If you want more information about bookmarks-based workflow, take a look at gps's slides; gps's blog is another good place to get information about version control at Mozilla.)

Android NDK and SDK version notes

Members of the Fennec community are successfully building Fennec with:

  • Android SDK Platform Android 5.0 (API 21)
  • SDK Build Tools 21
  • SDK Platform Tools 21
  • Android SDK Tools 23.0.5

You will also need the "Android Support Library" and "Google Play Services". Both are listed under Extras by the android package manager.

You should ensure that the Support Library and SDK versions match. If you update the Support Library but don't switch to a later SDK, you'll get errors that mention a newer version than your SDK, like this:

  Warning: android.support.v4.view.accessibility
  .AccessibilityNodeInfoCompatApi22: can't find referenced method
  'void setTraversalAfter(android.view.View,int)' in class 
  android.view.accessibility.AccessibilityNodeInfo

In the event that the NDK and SDK versions listed here are out of date, you can always find the canonical version numbers that Mozilla uses to build Fennec in automation. Look for the following variables:

  • ANDROID_NDK_VERSION
  • ANDROID_NDK_VERSION_32BIT
  • ANDROID_SDK_VERSION

bug 1108782 tracks listing and enforcing these version requirements in the source code.

Advanced topics

ccache

You can optionally install ccache, which can make rebuilding Firefox faster after you have built it once and don't change (much) of the C++ source code. In general, ccache is not a performance improvement if you use only one object directory, or edit only Java and/or JavaScript source code.

If you want to use ccache, add a line the following to your mozconfig:

ac_add_options --with-ccache[=/optional/path/to/ccache]

The default cache size of 1GB is not large enough for a Fennec build. To fully utilize ccache (generally meaning you can expect full builds to complete in <5 minutes from a populated cache), you'll need to change the cache size to at least 4GB. Do so by running:

ccache --max-size 4G

On Mac OS X, install ccache like:

brew install ccache

On Debian-like Linuxes, install ccache using:

sudo apt-get install mercurial ccache

Enabling C++ debugging

If you want to create a build suitable for debugging the C++ code, add:

ac_add_options --enable-debug-symbols

Building for the x86 architecture

If you want to build for x86, substitute:

ac_add_options --target=i386-linux-android

for the target specified above. Note: you cannot use ac_add_options --disable-optimize when building for x86. See bug 965870.

Building for the ARMv6 architecture

ARMv6 is no longer a supported target as of Firefox 33.

If you want to build for ARMv6 instead of ARMv7 (only required for very old, very low end phones), add:

ac_add_options --with-arch=armv6

Unsupported build flags

Do not specify these flags because the resulting build will cause out-of-memory crashes:

ac_add_options --disable-install-strip # Do not use!

Troubleshooting

Don't set CC / CXX environmental variables

If you've set the environmental variables CC and CXX (e.g. via .bash_aliases or via your mozconfig), then you probably need to unset them before building for Android, or else your build may fail with something like:

checking whether the C compiler (gcc -mandroid -fno-short-enums (etc etc)) works... no

followed by errors about "C compiler cannot create executables", "Relocations in generic ELF (EM: 40)", and "crtbegin_dynamic.o: error adding symbols: File in wrong format". This is a sign that you're compiling with your platform's native compiler (due to having CC / CXX set), instead of the android-specific GCC version that ships with the NDK. See bug 977817 for more details; as noted there, the build system may trust your custom CC & CXX variables, when you probably don't want it to.

Hacking

Developing Fennec with IDEs

Fennec has support for developing using IntelliJ (Ultimate and Community Edition) and Android Studio. (We also have some legacy support for Eclipse, but you should prefer using IntelliJ or Android Studio.) Get started Developing Fennec with IDEs.

Finding relevant code

Almost all of the Fennec-specific code is in the mozilla-central source tree under:

mobile/android/...

Of particular interest to most new contributors will be:

mobile/android/base/GeckoApp.java # the main Android activity that starts when you open Fennec
mobile/android/chrome/content/browser.js # the main JavaScript file that controls Gecko to make it do what we want

If you are looking for something specific, use the code-search tool at http://dxr.mozilla.org/ to search for relevant pieces of code.

Debugging

For the JavaScript parts of the code base, remote debugging with a desktop version of Firefox is the most effective way to see what is going on, add breakpoints, etc.

For Android Java code, if you have IntelliJ set up, the debugger works well and allows you to set breakpoints, inspect variables and objects, etc. Debugging without an IDE is much more primitive, and you can try using Android Log statements and inspecting logcat.

Coding Style

Java

Follow the Mozilla Coding Style.

XML files (layout, resources, styles, etc)

  • Each child tag should be indented with 4 spaces.
  • The properties should be aligned with the first property of a tag.
  • Each element should have "android:id" first, "style" (if exists) second, and the other properties follow. The root will have "xmlns:android" as first property.
  • A Line break after every tag.

Common nits

Check for these nits before asking for review for a patch:

  • Remove trailing whitespace
  • Spacing
    • Four spaces for Java indent (for main Fennec code - android-sync github project has different spacing)
    • Around operators (+, -, etc.) and :
    • Between comment "//" and text
  • Braces for Java if statements, even if they are one line
  • Comments should be full sentences (capitalization, punctuation)
    • Good comments are useful and clear even for someone reading a particular area of code for the first time
  • Avoid using single-letter variables in almost all cases - it makes code harder to read

Closing resources

When handling resources (like Cursors), a try/finally block should be used to ensure these are closed properly. For example:

final Cursor c = getCursor();
try {
    useCursorWhichMightThrowException(c);
} catch (SomeSpecificException sse) {
    log(sse);
} finally {
    c.close();
}

Once the try block is entered, the finally block will *always* get executed upon exit of the try block. The one exception is if there is a System.exit call inside the try block, which immediately exits the program and makes everything moot anyway. The finally block will get executed on caught and uncaught exceptions, as well as normal returns.

If you are casting the resource to something, make sure that you do the cast inside the try block, like so:

// GOOD!
InputStream is = getInputStream();
try {
    FileInputStream fis = (FileInputStream) is;
    ...
} finally {
    ...
}

rather than doing this:

// BAD!
FileInputStream fis = (FileInputStream) getInputStream();
try {
    ...
} finally {
    ...
}

This is so that in case of ClassCastExceptions you don't get a dangling open resource left behind.

Caveats for Timing

TLDR: Google recommends using SystemClock.uptimeMillis() for general purpose interval timing of user interface events or performance measurements. If you're adding stuff for timing, use SystemClock.uptimeMillis(), rather than something like new Date().getTime().

Normally in Java the default time-getter is System.currentTimeMillis() since it avoids the overhead of creating a new Date object. This is also what new Date() does under the hood. However, currentTimeMillis() and the Date object are both subject to change in unexpected ways if the user changes the time on their device, or if daylight savings comes into effect, or there's a network time update, or whatever. So Android has generously provided android.os.SystemClock which has various functions that you can use to get a better timestamp. Refer to the class javadoc and pick whichever function is most suitable for what you're trying to measure.

http://developer.android.com/reference/android/os/SystemClock.html

Multilocale builds

  • Create a directory, clone mozharness, copy the config file for easy editing/usage:
mkdir multilocale
cd multilocale
hg clone http://hg.mozilla.org/build/mozharness
cp mozharness/configs/multi_locale/standalone_mozilla-central.py myconfig.py
  • Edit myconfig.py
    • currently will check out m-c into a directory named 'mozilla-central' in this directory
    • currently assumes your mozconfig is in this directory and named 'mozconfig'
    • currently assumes your mozconfig sets your objdir name to 'objdir-droid'
  • pull mozilla-central
mozharness/scripts/multil10n.py --cfg myconfig.py --pull-build-source
# Alternately, you can hg clone http://hg.mozilla.org/mozilla-central
  • Run the script, which will create a multilocale apk
mozharness/scripts/multil10n.py --cfg myconfig.py

And you're done.

  • If you want to recompile or re-run the script, restore your objdir to en-US first!
mozharness/scripts/multil10n.py --cfg myconfig.py --restore-objdir

Also see this blog post for more information.

Single-locale language repacks

There is a script in mozharness for this (scripts/mobile_l10n.py) but it relies on buildbot information so it's not suitable for local repacks.

This assumes that $(AB_CD) is the locale you want to repack with; I tested with "ar" and "en-GB".

  • clone l10n-central/$(AB_CD) so that it is a sibling of your mozilla-central directory
  • I assume your object directory is "objdir-droid" and that you have built and packaged already
make -f client.mk && make -C objdir-droid package
  • copy your .mozconfig to .mozconfig.l10n and add the following lines
# L10n
ac_add_options --with-l10n-base=../../l10n-central

# Global options
ac_add_options --disable-tests

mk_add_options MOZ_OBJDIR=./objdir-l10n
  • cd to mozilla-central
  • configure and prepare objdir-l10n
MOZCONFIG=.mozconfig.l10n make -f client.mk configure
make -C objdir-l10n/config
  • copy your built package into objdir-l10n
cp ./objdir-droid/dist/fennec-*en-US*.apk ./objdir-l10n/dist
  • unpack. This files objdir-l10n/dist with the bits of the APK, ready for re-assembling.
make -C objdir-l10n/mobile/android/locales unpack
  • compare locales (you may need to install the compare-locales tool first). This writes locale differences into objdir-l10n/merged.
compare-locales -m objdir-l10n/merged mobile/android/locales/l10n.ini ../l10n-central $(AB_CD)
  • finally, re-assemble with the locale differences
LOCALE_MERGEDIR=objdir-l10n/merged make -C objdir-l10n/mobile/android/locales installers-$(AB_CD)

You should find an APK at "objdir-l10n/dist/fennec-*$(AB_CD)*.apk".

Testing

This section has instructions for running tests locally on a device of your choice. In theory, it is easy to run tests locally, but the test infrastructure is optimized and maintained primarily for the tegra boards and the panda boards used for continuous integration: It is not uncommon for device-specific bugs to creep in to the test infrastructure, causing problems when tests are run locally.

Running tests locally on non-rooted devices is particularly troublesome and sometimes impossible -- use a rooted device if at all possible.

Having trouble? Ping :gbrown on #mobile, or ask for help on #ateam.

Device Managers

Most test suites - mochitests, reftests, xpcshell tests, and others - use a "device manager" module to communicate with the remote device. There are two device manager implementations: ADB and SUT.

The ADB device manager uses the adb command from the Android SDK to communicate with the remote device. To use the ADB device manager:

  • ensure the adb command is in your shell's PATH
  • set environment variable DM_TRANS=adb
  • do not set environment variable TEST_DEVICE, or set TEST_DEVICE= (unless you really need it)

The SUT device manager uses TCP to communicate with a remote agent, which must be installed on the device. To use the SUT device manager:

  • ensure TCP connectivity between the local host and the remote device: check that they are on the same network and you can ping each from the other
  • ensure the SUT agent is installed and started on the remote device
    • the SUT agent APK is built alongside Fennec; just install <objdir-droid>/build/mobile/sutagent/android/sutAgentAndroid.apk
    • The agent should be configured to start automatically with your phone when it boots. To start it immediately from an adb shell do: `am start -n com.mozilla.SUTAgentAndroid/.SUTAgentAndroid -a android.intent.action.MAIN`
  • set environment variable DM_TRANS=sut
  • set environment variable TEST_DEVICE=<ip address of remote device -- displayed by SUT agent>

Note: to run the tests you should ensure that your host machine and device are on the same network so that the device can contact the server that will run on the host.

Host Builds (MOZ_HOST_BIN)

Android mochitests and reftests are driven by test suites on a host machine running xpcshell. The Android device being driven is referred to as the target device. The test suite locates xpcshell on the host machine via the environment variable MOZ_HOST_BIN, which must point to the directory that contains the xpcshell binary (executable on the host machine), its associated executables (certutil, pk12util, ssltunnel, etc), and its shared libraries.

Quick setup

You can download prebuilt host utilities:

Host architecture Size Download link
Mac OS X 76MB http://people.mozilla.org/~nalexander/host-utils/host-utils-37.0a2.en-US.mac.tar.xz
Linux (64-bit) 46MB http://people.mozilla.org/~nalexander/host-utils/host-utils-37.0a2.en-US.linux-x86_64.tar.xz
Linux (32-bit) 46MB http://people.mozilla.org/~nalexander/host-utils/host-utils-37.0a2.en-US.linux-i686.tar.xz

Extract the archives using tar (or tar and xzip), like:

 cd ~/.mozbuild
 wget http://people.mozilla.org/~nalexander/host-utils/host-utils-37.0a2.en-US.mac.tar.xz
 tar Jxf host-utils-37.0a2.en-US.mac.tar.xz
 export MOZ_HOST_BIN=~/.mozbuild/host-utils-37.0a2.en-US.mac

Packaging the host utility archives listed above is easy: follow these instructions.

Advanced setup

Alternatively, if one of the prebuilt host utilities packages is not appropriate for you or does not work, you can fetch the host utils by using the getxre utility that comes with JimDB. This utility will download the correct binaries for your host machine, and set the appropriate permissions. You should prefer the prebuilt host utilities because they are more likely to be tested and are significantly smaller downloads than the intermediate packages downloaded by getxre. (getxre does what the packaging instructions describe, except it has not been updated for current Firefox package structure on Mac OS X.) In the following stanza, replace $DIR with an output directory, such as ~/.mozbuild/host-utils.

 wget https://github.com/darchons/android-gdbutils/raw/master/python/getxre.py
 python getxre.py -d $DIR
 export MOZ_HOST_BIN=$DIR/bin

Alternatively, you can build desktop Firefox with a mozconfig which might be as simple as:

 ac_add_options --enable-application=browser
 mk_add_options MOZ_OBJDIR=./objdir-desktop

Then execute:

 MOZCONFIG=mozconfig.desktop ./mach build
 export MOZ_HOST_BIN=objdir-desktop/dist/bin

On Linux, the path to that build may also need to be in your LD_LIBRARY_PATH, unless your LD_LIBRARY_PATH contains ".":

 LD_LIBRARY_PATH=$LD_LIBRARY_PATH:.

Test Directory

All Android tests require a remote test directory: A place to store pre-configured profiles, support binaries and libraries, test files, etc. Most tests use a directory in /mnt/sdcard by default; xpcshell and cppunittests use /data/local by default (because it is usually not possible to set execute permission on files on /mnt/sdcard).

The default remote test directory is usually correct and sufficient, but sometimes the default is not appropriate for a device:

  • the device may not contain an SD card, or the SD card may not be mounted
  • there may not be enough free space on the default location's partition
  • the default location may not be writable by the ADB shell and/or SUT agent

(If you are using a Nexus S, the trick to making your device mountable is to not allow USB Storage between your computer and your device. When you plug in your device to your computer, simply don't click the button to allow this on your device and you should be able to run your tests.)

If necessary, the default remote test directory may be changed with:

 export EXTRA_TEST_ARGS=--remoteTestRoot=<remote-directory>

Reftests

First, ensure that MOZ_HOST_BIN has been set according to the directions above.

 TEST_PATH=<path> make -C <objdir-droid> reftest-remote

Example TEST_PATH:

 TEST_PATH=layout/reftests/reftest-sanity/reftest.list

Notes:

  • If TEST_PATH is not specified, *all* reftests will be attempted: This usually fails/hangs and is not recommended. Specify a TEST_PATH or use the --total-chunks and --this-chunk arguments to reduce the number of tests executed.
  • sut recommended. Test results displayed and saved to reftest.log and reftest-remote.log.
  • adb works also. Test results displayed and saved to reftest.log (additional diagnostics in reftest-remote.log).
  • Ensure that the device and host machine are on the same network.

mochitest-plain

Pre-requisites:

  • Ensure that MOZ_HOST_BIN has been set according to the directions above.
  • Ensure your device is connected and visible with "adb devices".
  • Ensure that the device and host machine are on the same network <-- This is important. The device and the host need to communicate over the network to run the tests. Having the device connected to the host via USB is not sufficient.
  • Ensure that Fennec has been built and installed: mach build && mach package && mach install

Running tests:

 mach mochitest-plain
 OR mach mochitest-plain <test-dir>
 OR mach mochitest-plain <test-dir>/<test-name>

Use "find -name mochitest.ini" to find valid test directories.

Notes:

  • Currently the mach command only supports the adb device manager.
  • You can also run tests using make -C $OBJDIR mochitest-remote, but this is deprecated. Use TEST_PATH=<test-dir>/<test-name> or TEST_PATH=<test-dir> to run a subset of tests.
    • A rooted device is required to run the unit tests using the SUT Agent. Use ADB for unrooted devices (see [[1]]).
    • Currently using ADB and using /data/local/tests on a non-rooted phone causes setting TEST_PATH to a directory rather than an individual HTML file to fail, and SpecialPowers tests to fail too - filed as bug 822652

mochitest-chrome

Pre-requisites:

  • Ensure that MOZ_HOST_BIN has been set according to the directions above.
  • Ensure your device is connected and visible with "adb devices".
  • Ensure that the device and host machine are on the same network <-- This is important. The device and the host need to communicate over the network to run the tests. Having the device connected to the host via USB is not sufficient.
  • Ensure that Fennec has been built and installed: mach build && mach package && mach install

Running tests:

 mach mochitest-chrome
 OR mach mochitest-chrome <test-dir>
 OR mach mochitest-chrome <test-dir>/<test-name>

Use "find -name chrome.ini" to find valid test directories.

Notes:

  • Currently the mach command only supports the adb device manager.
  • mochitest-chrome on Android is not run in continuous integration on treeherder; many tests may fail.

Robocop

The Robocop test suite verifies UI behavior in native Fennec. It is built on the Robotium testing framework. To run tests locally, a separate Robocop test APK also needs to be installed.

To get tests running locally, first build and install native Fennec,

 mach build
 mach package
 mach install

Next, execute mach robocop which installs the Robocop APK to the device and starts the entire testing the entire test suite. Prior to running this, ensure that MOZ_HOST_BIN has been set according to the directions above.

 mach robocop

More info at Robocop and http://mxr.mozilla.org/mozilla-central/source/mobile/android/base/tests/README.rst.

If you make changes to the tests and want to see them run on a device, you need to build the tests again and reinstall.

 mach build build/mobile/robocop
 mach robocop

This builds the tests in mobile/android/base/tests and then installs the debug-signed Robocop APK onto the device. (It's unintuitive that building mobile/android doesn't build the tests within mobile/android/base/tests.)

Notes:

  • Mach is self documenting! For help, try mach help robocop.
  • To run one test at a time, find the test name (like "testLoad") in mobile/android/base/tests/robocop.ini and pass it as an argument, like: mach robocop testLoad.
  • A rooted device is NOT required.
  • SUT and ADB device managers are both supported.
  • MOZ_HOST_BIN is used to launch xpcshell on the desktop to provide a web server.
  • You can also run tests using make -C $OBJDIR mochitest-robocop, but this is deprecated. Use TEST_PATH=<test-name> to run just one test at a time, like: TEST_PATH=testAwesomebar make -C $OBJDIR mochitest-robocop.

Troubleshooting:

  • Ensure the device's screen is on
  • Ensure that the device and host machine are on the same network.
    • Are the phone and the desktop both using wifi? (wifi vs ethernet??)
    • Are the phone and the desktop both using the same wifi network? (Mozilla vs Mozilla Guest??)
    • Is the desktop environment running in a VM? If so, you likely want a "Bridged" connection -- not NAT or Host-only.
  • Ensure Robocop has the correct IP address for your machine
    • Make sure _SERVER_ADDR in the Robocop output is the same as your machine's IP address.
    • If not, set it to your IP by running export DM_FLAGS="--remote-webserver=xxx.xxx.xxx.xxx"
  • Ensure the binaries in your MOZ_HOST_BIN folder are executable, in case you pull them down from the FTP site. You might see "OSError: [Errno 13] Permission denied" if they are not executable. Use chmod to fix them. Check certutil, pk12util and ssltunnel.
  • Additional tips at Auto-tools/Projects/Robocop#Frequently_found_errors

xpcshell

Be sure you have successfully built Fennec and generated an APK, as described above.

To run all tests referenced by the master xpcshell manifest:

 mach xpcshell-test

(Currently the master manifest is restricted: not all tests are run.)

To run a subset of tests in the specified directory:

 mach xpcshell-test <test-directory>

Once either of the xpcshell-tests-remote commands has completed successfully, all test files have been copied to device, and it is then possible to run a single test quickly, without setup:

 mach xpcshell-test <test.js> --no-setup

Notes:

  • A rooted device IS required.
  • Both ADB and SUT device managers are supported.
  • Setup can take several minutes! When using ADB, setup is faster if unzip is available on the remote device; if your device does not have unzip, try installing busybox.

cppunittests

To run a single compiled code test:

 cd <objdir-droid>
 export TEST_PATH=<test>
 make cppunittests-remote

For example, TEST_PATH=xpcom/tests/TestTimers, or TEST_PATH="xpcom/tests/TestTimers xpcom/tests/TestFile".

To run all the compiled code tests in a directory:

 cd <objdir-droid>
 make -C <dir> cppunittests-remote

For example:

 make -C xpcom/tests cppunittests-remote

Advanced features:

As with xpcshell tests, you can skip setup if needed:

 export EXTRA_TEST_ARGS="--noSetup"

This will avoid copying any files to the device -- be sure that all necessary files are on the device before using this!

You can change the environment variables seen by the unit test:

 export EXTRA_TEST_ARGS="--addEnv MYVAR=value"
 export EXTRA_TEST_ARGS="--addEnv MYVAR1=value1 --addEnv MYVAR2=value2"
 export EXTRA_TEST_ARGS="--addEnv HOME="

Notes:

  • Either adb or sut device manager may be used
  • All files are copied to /data/local/tests by default. On some devices, you may need to create /data/local/tests and make it world writable.

browser-chrome

Before you run tests, you will need to make sure you have packaged the tests in your object dir:

 make -C <objdir-droid> package-tests

There is currently no special make command to build and run browser-chrome tests, but it should be possible to make them run by calling:

 cd <objdir-droid>/_tests/testing/mochitest
 python runtestsremote.py --dm_trans=adb --test-path=mobile --browser-chrome --deviceIP=1.2.3.4 
                          --app=org.mozilla.fennec_$USER --xre-path=<objdir_x86>/dist/bin/

Running the Android 2.3 ARM emulator

The "Android 2.3 Opt" tests on tbpl run in an Android ARM emulator. For best results investigating and reproducing Android 2.3 test failures, try server is recommended: Running the same tests on the same emulator on different host hardware may very well produce different results. You can also borrow a slave: https://wiki.mozilla.org/ReleaseEngineering/How_To/Request_a_slave.

But if you really want to try running the emulator locally...

1. Obtain the AVD definitions from tooltool. To determine the url, open http://hg.mozilla.org/mozilla-central/raw-file/default/testing/config/tooltool-manifests/androidarm/releng.manifest and you'll see something like:

 [{
 "size": 261782928,
 "digest":
 "7140e026b7b747236545dc30e377a959b0bdf91bb4d70efd7f97f92fce12a9196042503124b8df8d30c2d97b7eb5f9df9556afdffa0b5d9625008aead305c32b",
 "algorithm": "sha512",
 "filename": "AVDs-armv7a-gingerbread-build-2014-01-23-ubuntu.tar.gz",
 "unpack": "True"
 }]

Now browse https://secure.pub.build.mozilla.org/tooltool/pvt/build and download '7140e0...' -- this is AVDs-armv7a-gingerbread-build-2014-01-23-ubuntu.tar.

2. Extract the AVD definition archive in your .android directory:

tar xvf AVDs-armv7a-gingerbread-build-2014-01-23-ubuntu.tar -C ~/.android

3. Use the emulator executable from any recent Android SDK. In production, the emulator from Android 18 is used.

4. Launch the emulator with:

emulator -avd test-1 -debug init,console,gles,memcheck,adbserver,adbclient,adb,avd_config,socket -port 5554 -qemu -m 1024 -cpu cortex-a9

(expect lots of output).

5. Wait for sutagent to launch.

6. If you want to use sutagent, redirect ports for sutagent and set the device manager environment variables:

adb forward tcp:20700 tcp:20700

adb forward tcp:20701 tcp:20701

DM_TRANS=sut

TEST_DEVICE=127.0.0.1

7. Run tests as described above.

(Or, if you want to run tests very much like on tbpl, see https://wiki.mozilla.org/ReleaseEngineering/Mozharness/How_to_run_tests_as_a_developer.)

Running the Android 4.2 x86 emulator

The "Android 4.2 x86 Opt" tests on tbpl run in an Android x86 emulator. For best results investigating and reproducing Android 4.2 x86 test failures, try server is recommended: Running the same tests on the same emulator on different host hardware may very well produce different results. You can also borrow a slave: https://wiki.mozilla.org/ReleaseEngineering/How_To/Request_a_slave.

But if you really want to try running the emulator locally...

1. Obtain the AVD definitions from tooltool. To determine the url, open http://hg.mozilla.org/mozilla-central/raw-file/default/testing/config/tooltool-manifests/androidx86/releng.manifest and you'll see something like:

 [{
 "size": 561274118,
 "digest":   "3b2d18eb0194d82c70c5ee17487ccbac309f9b2e9839fe7ca4a27a9a06f6338bb24394476da78559685d99151fccc85fdde03297aa73ee2f7fb3183e11925c4d",
 "algorithm": "sha512",
 "filename": "AVDs-x86-android-4.2_r1-build-2013-11-13-ubuntu.tar.gz",
 "unpack": "True"
 }]

Now browse https://secure.pub.build.mozilla.org/tooltool/pvt/build and download '3b2d18...' -- this is AVDs-x86-android-4.2_r1-build-2013-11-13-ubuntu.tar.

2. Extract the AVD definition archive in your .android directory:

tar xvf AVDs-x86-android-4.2_r1-build-2013-11-13-ubuntu.tar -C ~/.android

3. Install KVM, if necessary:

sudo apt-get install cpu-checker qemu-kvm bridge-utils

See bug 929048 for more details, if you have trouble with KVM on your host.

You may be able to use the emulator executable from any recent Android SDK. In production, a patched version of "tools-r22" SDK is used. See bug 933918 for details.

4. Launch the emulator with:

emulator -avd test-1 -debug init,console,gles,memcheck,adbserver,adbclient,adb,avd_config,socket -port 5554 -qemu -m 1024 -enable-kvm

(expect lots of output).

5. Wait for sutagent to launch.

6. If you want to use sutagent, redirect ports for sutagent and set the device manager environment variables:

adb forward tcp:20700 tcp:20700

adb forward tcp:20701 tcp:20701

DM_TRANS=sut

TEST_DEVICE=127.0.0.1

7. Run tests as described above.

(Or, if you want to run tests very much like on tbpl, see https://wiki.mozilla.org/ReleaseEngineering/Mozharness/How_to_run_tests_as_a_developer.)

talos

  • NOTE: this requires a fix for bug 1122701
  • NOTE: this requires python 2.7

See also https://wiki.mozilla.org/Buildbot/Talos

 hg clone http://hg.mozilla.org/build/talos talos
 cd talos
 python INSTALL.py
 . bin/activate
 cd talos
 <obtain your fennec apk and put it in the current directory>
 <obtain your robocop apk and put it in the current directory>
 <obtain your fennec_ids.txt and put it in the current directory>
 <install your fennec apk>
 <if running a robocop-based test, install robocop.apk>
 <start sutagent on the device>
 python remotePerfConfigurator.py --apkPath=fennec-38.0a1.en-US.android-arm.apk 
    -v -e org.mozilla.fennec 
    --webServer=<your local desktop IP>:8080 
    --noChrome 
    --remoteDevice=<sutagent-IP> 
    --sampleConfig=remote.config --output=local.yml --browserWait=60 
    --activeTests=tcheck2 
    --fennecIDs=fennec_ids.txt
 python run_tests.py --apkPath=fennec-38.0a1.en-US.android-arm.apk --noisy local.yml

For Robocop based tests (tcheck2, tprovider, etc...), we need to use the --fennecIDs flag to pass in the generated fennec_ids.txt file from the build you are testing. This file is generated during build time and has to match the fennec.apk and robocop.apk file. When this flag is used, we copy fennec_ids.txt and robotium.config (generated during configuration time) to the device and use those to run Robocop. If you are running ts, tp4, tsvg, or other traditional talos tests, there is no need for the --fennecIDs flag.

Aside: For a quick-and-dirty hacky way to run robocop-talos tests locally, see Mobile/Fennec/Android/LocalRoboTalos

S1/S2 Automation

These tests start Fennec with a URL and measure the time to throbber start, time to throbber stop, and drawing end times.

S1/S2 graphs can be viewed at: http://phonedash.mozilla.org/

Manual run instructions can be found at: https://etherpad.mozilla.org/fennec-perf-ts-take2

See also: https://wiki.mozilla.org/Mobile/Performance/S1S2-Tests

Eideticker

Eideticker measures perceived Firefox performance by video capturing automated browser interactions.

Eideticker graphs can be viewed at: http://eideticker.mozilla.org/

See also:

Trouble-shooting testing problems

  • Does your mozconfig contain "ac_add_options --disable-tests"?
    • If so, you will see something like:
make: *** No rule to make target <your-test-target>.  Stop.
  • Is adb in your $PATH?
  • Is your device connected? Does it appear in the output from "adb devices"?
  • Can you run adb shell?

Debugging

Using logcat

logcat is a tool that is going to show you some logs prompted by the device. It might be a good help if you don't want to or can't run gdb. You can use it by running this command:

 adb logcat -v time

You can make things appear in logcat using printf_stderr. With debug builds, NS_WARNING, NS_ERROR and NS_ASSERTIONS will show up in logcat. If you're trying to debug something, you may wish to pipe the logcat output through grep to filter out irrelevant things (most Fennec-related output will be viewable by

 adb logcat -v time | grep Gecko

but remember that when attaching log output to a bug you should include unfiltered output as there may be relevant log entries under other tags.

Using other logcat apps

ICS (4.1) and below
  • Install the aLogCat app. Use it to capture logs and attach the logs to bugs.

Once you have alogcat installed, just use Fennec as you would normally. Upon encountering a bug or issue, start the aLogcat app (as soon as possible after seeing the Fennec issue) and select "Share" or "Save" from the menu to send it via email or save it to the SD card. The log can then be attached to a bug or sent to a developer. As with adb logcat, it is better to have a log with timestamps than without timestamps. To enable timestamps in the log, select "Preferences" from the aLogcat menu, and change the "Format?" option to "Time".

If you need to, you can search for some kinds of Fennec-related output by using the "Filter" menu item and entering "Gecko". However, when submitting logs for bug reports, please make sure you clear the filter and include all of the available log data.

Jelly Bean (4.2) and above

JavaScript dump()

To use the dump() function in JavaScript to write to the log:

  1. Go to about:config and set browser.dom.window.dump.enabled to "true"
  2. Run the following ADB commands:
adb shell stop
adb shell setprop log.redirect-stdio true
adb shell start

Using JimDB

See Mobile/Fennec/Android/GDB

Using Debug Intent

Note: this is not useful with JimDB. If you want to use JimDB, just start it.

In order to attach before things get running, launch with:

  adb shell am start -a org.mozilla.gecko.DEBUG -n org.mozilla.fennec_foobar/.App

(Replace foobar by your username)

and just click launch once gdb is attached. If you need to debug a crash that happens before XRE_Main is called, the patch on bug 572247 may be useful.

this script [2] will attach gdbserver for you

Getting dalvik java stack dumps using gdb

(gdb) call dvmDumpAllThreads(true)

this will dump a stack trace to logcat

Note: this will only work if you have symbols for dalvik.

Debugging with jdb

JimDB can now launch JDB integration

You can also use eclipse for debugging in a similar way by setting up for debugging "Remote Java application".

Debugging with eclipse

You need to find the PID of your fennec process. Forward it to a local TCP socket as in "Debugging with jdb."

In Eclipse switch to the debug perspective. Go to Run > Debug configurations... Remote Java Application. Change the port to the TCP port you specified in your adb command. Under Source, navigate to your checkout, then into mobile/android.

Eclipse looks for source code in a specific location. You need to create the directory hierarchy:

mobile
 /android
   /org
     mozilla/
       gecko -> ../../base

That is, in mozilla-central/mobile/android, create org/mozilla, and put the symlink gecko pointing to mozilla-central/mobile/android/base.

You may also want to add more debugging information and can do that like this:

diff --git a/config/android-common.mk b/config/android-common.mk
index 4591239..a47726a 100644
--- a/config/android-common.mk
+++ b/config/android-common.mk
@@ -70,6 +70,6 @@ JAVAC_FLAGS = \
   -classpath $(JAVA_CLASSPATH) \
   -bootclasspath $(JAVA_BOOTCLASSPATH) \
   -encoding UTF8 \
-  -g:source,lines \
+  -g:source,lines,vars \
   -Werror \
   $(NULL)

Arguments and Environment Variables

If you need to set an environment variable at run time, append --es env# VAR=VAL to your activity manager command where # is the ordered number of variables for example:

 adb shell am start -a android.activity.MAIN -n org.mozilla.fennec_$USER/.App --es env0 VAR=val --es env1 FOO=bar

If you need to pass arguments at run time, append --es args "<your-args>" to your activity manager command. For example, to launch with a specific profile:

 adb shell am start -a android.activity.MAIN -n org.mozilla.fennec_$USER/.App --es args "--profile /mnt/sdcard/myprofile"

To launch with a specific URL, use the am -d option to set the intent's data URI:

 adb shell am start -a android.activity.MAIN -n org.mozilla.fennec_$USER/.App -d 'http://www.mozilla.org'

PR Logging

You can use the env vars as described above to enable NSPR logging:

 adb shell am start -a android.activity.MAIN -n org.mozilla.fennec_$USER/.App --es env0 NSPR_LOG_MODULES=all:5 --es env1 NSPR_LOG_FILE=/mnt/sdcard/log.txt

If no file is specified, logging is directed to the android logs:

 adb shell am start -a android.activity.MAIN -n org.mozilla.fennec_$USER/.App --es env0 NSPR_LOG_MODULES=all:5

Look for lines marked "PRLog" in the adb logcat output.

Using legacy GDB (non-JimDB)

See Fennec/Android/GDBNoRoot

Reading back the framebuffer

If you need to verify what is in the back buffer at a particular time, you can cleverly call functions from gdb to allocate memory, read back, and write that to disk.

You need to know the size of your framebuffer a priori; in this case, it's 480x699.

You also need to know what the GL enum values are, because unless you compile with -ggdb, you don't have #defines available to you in the debugger. Very helpful information: GL_RGBA = 0x1908, GL_UNSIGNED_BYTE = 0x1401. The rest you can find in gfx/gl/GLDefs.h.

You should be able to call glReadPixels directly, but in my experience that causes Fennec to crash. However, if you have a GLContext* lying around, as you often do, you can work around that problem.

(gdb) set $m = (int*)malloc(480*699*4)
(gdb) call aManager->mGLContext.mRawPtr->fReadPixels(0, 0, 480, 699, 0x1908, 0x1401, (void*)$m)
(gdb) set $f = fopen("/sdcard/outputfile", "wb+")
(gdb) call fwrite($m, 1, 480*699*4, $f)
$7 = 1342080
(gdb) call fclose($f)
$8 = 0

Now there is a file called /sdcard/outputfile that you can adb pull. But since it's just raw RGBA values, you need to be able to wrap that in PNG headers to display it. Jeff Muizelaar wrote a header called minpng.h that you can use to do so.

Get Jeff's minpng.h, and put it in a directory along with a driver c program:

#include "minpng.h"

int main(int argc, char* argv[])
{
	FILE* f = fopen(argv[1], "rb");
	int w = atoi(argv[2]);
	int h = atoi(argv[3]);
	char* d = (char*) malloc(w * h * 4);
	fread(d, w * h * 4, 1, f);
	fclose(f);
	write_png(argv[4], d, w, h);
}

Compile and run:

$ gcc -o minpng minpng.c
$ ./minpng outputfile 480 699 output.png

Using Rendertrace (Maple)

Rendertrace is a utility that will dump layer position and timing information (such as drawing, upload) to the console. This information can pasted into the rendertrace web front end to visualize the layer position and event timeline. This will let you understand where you're gecko is spending its time and why were checkerboarding.

To enable go in 'gfx/layers/RenderTrace.h' and uncomment '#define MOZ_RENDERTRACE'. Rebuild and run 'adb logcat | grep RENDERTRACE', paste the result in http://people.mozilla.org/~bgirard/rendertrace.html and hit 'reload'. For details talk to BenWa.

Using apitrace

Apitrace is a tool for tracing GL/EGL calls for debugging purposes. It basically uses an interim shared library called libapitrace that contains shadow gl* and egl* functions, which then get logged and then passed through to the real driver.


Use apitrace from https://github.com/apitrace/apitrace to build for desktop and android.

apt-get install libegl1-mesa-dev libgles1-mesa-dev libgles2-mesa-dev libqt4-dev cmake
git clone https://github.com/gw280/apitrace.git
cd apitrace

# Build for Android
cmake -DANDROID_NDK=/path/to/your/ndk -DCMAKE_TOOLCHAIN_FILE=android/android.toolchain.cmake -DANDROID_API_LEVEL=9 -Bbuild-android -H.
make -C build-android -j8

# Build for desktop
cmake -H. -Bbuild
make -C build -j8

export EGL_SOFTWARE=true
./build/eglretrace -v /path/to/your/apitrace_log.trace

The Android build will create egltrace.so in build-android/wrappers, which you can then push to your device to /data/local:

adb push build-android/wrappers/egltrace.so /data/local/tmp

Restarting Fennec will cause it to load the apitrace library and the apitrace log will be saved to /data/data/org.mozilla.fennec_username/firefox.trace

You can then adb pull /data/data/org.mozilla.fennec_username/firefox.trace and analyse it on your desktop.

You can also use qapitrace as a GUI to inspect your trace files. (be sure to switch qapitrace to the EGL api using the options dialog)

These instructions provide a trace that does not include the Java GL code. To get traces including java code is more complicated. You need to use the patch from this bug https://bugzilla.mozilla.org/show_bug.cgi?id=749859 and this version of https://github.com/ideak/apitrace/tree/dev. Further, you'll need to build your own image/modify the current one to replace /init.rc. You also need to disable hardware acceleration of the UI (https://bug746703.bugzilla.mozilla.org/attachment.cgi?id=619009) Ask jrmuizel for more information if you want to do this.

about:memory

about:memory provides heaps (ha!) of useful memory information.

You can obtain a snapshot of memory info from a running Fennec instance using:

 adb shell am broadcast -a org.mozilla.gecko.MEMORY_DUMP

This dumps a json file to the SD card and prints out the exact filename to logcat. You can pull the json file to desktop using

 adb pull <absolute-path-to-file>

and view it in firefox's about:memory: use the "Read reports from a file" option at the bottom of the about:memory page.

Profiling

See https://wiki.mozilla.org/Mobile/Fennec/Android/Profiling.

Debugging Java code with DDMS

See http://developer.android.com/tools/debugging/ddms.html

Other useful tips and tricks

Addons

There are some addons which may be useful for development purposes. See Mobile/Fennec/Android/Development/Addons.

Tweaking UI prefs

By default, all of these prefs are set to "-1" in Fennec, meaning they take the values listed below, which are maintained in Axis.java.

Fractional values are specified in 1/1000th of a value; to specify a value of 0.3, write 300.

Note: You need to restart Fennec after changing these values.

Pref Default value Description
ui.scrolling.friction_slow 850 This fraction in 1000ths of velocity remains after every animation frame when the velocity is low.
ui.scrolling.friction_fast 970 This fraction in 1000ths of velocity remains after every animation frame when the velocity is high.
ui.scrolling.velocity_threshold 10 Below this velocity (in pixels per frame), the friction changes from friction_fast to friction_slow.
ui.scrolling.max_event_acceleration 12 The maximum velocity change factor between events, per ms, in 1000ths.
ui.scrolling.overscroll_decel_rate 40 The rate of deceleration when the surface has overscrolled, in 1000ths.
ui.scrolling.overscroll_snap_limit 300 The fraction of the surface which can be overscrolled before it must snap back, in 1000ths.
ui.scrolling.min_scrollable_distance 500 The minimum amount of space that must be present for an axis to be considered scrollable, in 1/1000ths of pixels.
gfx.displayport.strategy 1 The strategy we use to determine how display ports are calculated. 0 = fixed margin, 1 = velocity bias, 2 = dynamic resolution, 3 = no margins
gfx.displayport.strategy_fm.multiplier 1500 When gfx.displayport.strategy = 0 (fixed margin), the 1000th of each dimension of the viewport the displayport is sized to.
gfx.displayport.strategy_fm.danger_x 100 When gfx.displayport.strategy = 0 (fixed margin), the 1000th of the width of the viewport the horizontal danger zone is set to.


Danger zone is defined as the space at the edge of the viewport at which the viewport (and hence displayport) starts being changed.

gfx.displayport.strategy_fm.danger_y 200 When gfx.displayport.strategy = 0 (fixed margin), the 1000th of the height of the viewport the vertical danger zone is set to.
gfx.displayport.strategy_vb.multiplier 1500 When gfx.displayport.strategy = 1 (velocity bias), the 1000th of each dimension of the viewport the displayport is sized to.
gfx.displayport.strategy_vb.threshold 32 When gfx.displayport.strategy = 1 (velocity bias), the threshold for velocity, in pixels/frame, when multiplied by the screen DPI.
gfx.displayport.strategy_vb.reverse_buffer 200 When gfx.displayport.strategy = 1 (velocity bias), the fraction of the buffer (in 1000ths) to be kept in the direction opposite the direction of the scroll.

Invalidate the JavaScript startup cache

To make life easier for developers, in local development builds only, bug 976216 invalidates the JavaScript startup cache every time Firefox for Android starts. On non-local development builds (including TBPL builds and try builds), the JavaScript startup cache is not invalidated at startup.

Background: JavaScript files and modules (like browser.js) are cached for fast startup. The cache is invalidated only when an internal build ID is updated, which only happens when certain C++ code is rebuilt. That doesn't happen for most patches that only touch Java and JavaScript within mobile/android. See bug 695145 for the details of this build ID handling. If you happen to be building a non-local development build in some way, you might need to invalidate the JavaScript startup cache by touching toolkit/xre/nsAndroidStartup.cpp and rebuilding libxul.

killer script

#!/bin/sh
if [ $# -ne 1 ]
then
    echo "usage: $0  packagename.of.your.activity"
    echo "for example:"
    echo "  $0 org.mozilla.fennec"
    exit
fi

p=`adb shell ps | grep $1 | awk '{print $2}'`
if [ "$p" = "" ];
then
    echo "ERROR: That doesn't seem to be a running process. Please make sure your"
    echo "application has been started and that you are using the correct"
    echo "namespace argument."
    exit
fi

adb shell run-as $1 kill $p

.gdbinit

This is an example .gdbinit that uses the symbols from a locally built rom and automatically attaches to gdbserver. Note that putting a .gdbinit file inside a directory will make gdb load it thus you will not pollute your regular gdb init with those configurations.

set solib-search-path /home/blassey/android/system/out/target/product/passion/symbols/system/bin:/home/blassey/android/system/out/target/product/passion/symbols/system/lib/:/home/blassey/src/ndk5-m-c/objdir-droid-dbg/dist/bin
set solib-absolute-prefix /home/blassey/android/system/out/target/product/passion/symbols/system/lib/
target remote localhost:12345

Rooting Android devices

See Rooting Android Devices.

Sign a Fennec build

Nightly builds are available unsigned, so that you can sign them with your local debug key and install them on top of your own debug builds (without uninstalling and losing your profile). To sign and install the unsigned nightly build:

 wget http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/latest-mozilla-central-android-r7/gecko-unsigned-unaligned.apk
 jarsigner -sigalg SHA1withRSA -digestalg SHA1 -keystore ~/.android/debug.keystore -storepass android -keypass android gecko-unsigned-unaligned.apk androiddebugkey
 zipalign -f -v 4 gecko-unsigned-unaligned.apk gecko-signed-aligned.apk
 adb install -r gecko-signed-aligned.apk

Or you can also re-sign a signed build. If "fennec.apk" is signed already, this will remove the signature and replace it with your own. The result will be saved as "fennec-resigned.apk":

 zip fennec.apk -d META-INF/*
 jarsigner -sigalg SHA1withRSA -digestalg SHA1 -keystore ~/.android/debug.keystore -storepass android -keypass android fennec.apk androiddebugkey
 zipalign -f -v 4 fennec.apk fennec-resigned.apk

If you get this error when you try to sign a package:

 jarsigner: unable to sign jar: java.util.zip.ZipException: invalid entry compressed size (expected 16716 but got 16964 bytes)
    

You should to follow some steps to complete your task:

* rename the .apk to .zip
* unzip the file in some folder
* remove the METAINF folder
* zip the remaining files
* change the name .zip to .apk
* sign again

To verify if everything is alright use the command

 jarsigner -verbose -verify