Mobile/Fennec/Android/GDB

Deprecated

This page is the information of deprecated debugging tools. We recommend to use Android studio (https://mozilla.github.io/geckoview/tutorials/native-debugging.html).

What JimDB?

JimDB is a fork of GDB used for simplifying Fennec development. It consists of two parts:

• Binaries - the GDB/GDBServer binaries that include Android-specific patches (https://github.com/darchons/android-gdb)
• Scripts - the Python scripts that run under GDB to simplify common tasks (https://github.com/darchons/android-gdbutils)

Use #mobile on IRC for support.

Getting started

Using mach

You can use mach to fetch, install, configure, and run JimDB with one simple command. Note that if you are on OS X, mach may install JimDB correctly, but then try to debug using lldb. If that happens, you can still run JimDB manually from your ~/.mozbuild folder (you can find the full path in the output below).

 $ ./mach run --debug
 JimDB (arm) not found: /home/gbrown/.mozbuild/android-device/jimdb-arm does not exist
 Download and setup JimDB (arm)? (Y/n) y
 Installing JimDB (linux64/arm). This may take a while...
 From https://github.com/darchons/android-gdbutils
  * [new branch]      master     -> origin/master
  * [new tag]         gdbutils-2 -> gdbutils-2
  * [new tag]         initial-release -> initial-release
  1:45.57 /home/gbrown/.mozbuild/android-device/jimdb-arm/bin/gdb -q --args 
 Fennec GDB utilities
   (see utils/gdbinit and utils/gdbinit.local on how to configure settings)
 1. Debug Fennec (default)
 2. Debug Fennec with env vars and args
 3. Debug using jdb
 4. Debug content Mochitest
 5. Debug compiled-code unit test
 6. Debug Fennec with pid
 Enter option from above: 1
 New ADB device is "emulator-5554"
 Using device emulator-5554
 Using object directory: /home/gbrown/objdirs/droid
 Set sysroot to "/home/gbrown/.mozbuild/android-device/jimdb-arm/lib/emulator-5554".
 Updated solib-search-path.
 Ignoring BHM signal.
 Using package org.mozilla.fennec_gbrown.
 Launching org.mozilla.fennec_gbrown... Done
 Attaching to pid 674... Done
 Setting up remote debugging... Done
 Ready. Use "continue" to resume execution.
 : No such file or directory.
 (gdb)

Using pre-built binaries

Instructions

1. Download the latest pre-built jimdb binaries for your host and device platforms from this directory (use 'jimdb-arm' for ARM devices and 'jimdb-x86' for x86 devices)
2. Download the latest pre-built gdbserver binary for your device platform from the same directory
3. Extract to a user-writable directory
4. Replace the gdbserver binary in the jimdb-*/bin/ directory with the one you downloaded separately (it's newer)
5. Run 'git pull' from inside the utils/ directory to get the latest scripts
6. Launch bin/gdb to start using JimDB!
7. OPTIONAL: For easier access, you can create a symlink to jimdb under, e.g., /usr/bin

Example

For ARM devices on Linux,

cd $HOME
wget https://people.mozilla.org/~nchen/jimdb/jimdb-arm-linux_x64.tar.bz2 # step 1
tar -xf jimdb-arm-linux_x64.tar.bz2 # step 3
cd jimdb-arm/utils && git pull # step 5
../bin/gdb # step 6
sudo ln -s $HOME/jimdb-arm/bin/gdb /usr/bin/jimdb # step 7

If you download the wrong x86 or x64 build, you might get an error like:

$ ./bin/gdb
./bin/gdb: error while loading shared libraries: libpython2.7.so.1.0: cannot open shared object file: No such file or directory

Custom-building

Step 1. Compiling the binaries

There are two things to build here: gdb and gdbserver

Linux

GDB

1. Clone the GitHub repo (do not use the Zip archive, as the build system will try to obtain a git changeset id)
2. You may need to install additional packages in order to build GDB. For example,

   sudo apt-get build-dep gdb

3. Run configure inside the source directory

   # ARM:
   ./configure --target=arm-linux-android --with-python=yes --prefix=/nonexistent \
     --with-gdb-datadir=/nonexistent/utils --with-system-gdbinit=/nonexistent/utils/gdbinit
   # x86:
   ./configure --target=i686-pc-linux-android --with-python=yes --prefix=/nonexistent \
     --with-gdb-datadir=/nonexistent/utils --with-system-gdbinit=/nonexistent/utils/gdbinit

4. Run make

   make -j8

5. The compiled binary will be at gdb/gdb

If you see an error like:

../../../../bfd/doc/bfd.texinfo:325: unknown command `colophon'
../../../../bfd/doc/bfd.texinfo:336: unknown command `cygnus'
Makefile:421: recipe for target 'bfd.info' failed

you can fix this by running:

sed -i -e 's/@colophon/@@colophon/' -e 's/doc@cygnus.com/doc@@cygnus.com/' bfd/doc/bfd.texinfo

or by checking out a more recent branch (this bug is in the 7_5 branch).

GDBServer

Consider using the pre-built gdbserver binary from the "Using pre-built binaries" section above; gdbserver is not specific to your platform. Otherwise,

1. Use the source directory from above
2. Make sure you have a copy of the Android NDK
3. Run configure inside the gdb/gdbserver directory (following is for NDK r8d)

   export NDK=/PATH/TO/NDK
   # ARM:
   export PATH=$PATH:$NDK/toolchains/arm-linux-androideabi-4.6/prebuilt/linux-x86/bin
   ./configure --host=arm-linux-androideabi --with-sysroot=$NDK/platforms/android-9/arch-arm
   # x86:
   export PATH=$PATH:$NDK/toolchains/x86-4.6/prebuilt/linux-x86/bin
   ./configure --host=i686-linux-android --with-sysroot=$NDK/platforms/android-9/arch-x86

4. Run make

   make -j8

5. The compiled binary will be at gdbserver

Mac

GDB

1. Get a zip of the source from the GitHub repo (Get the zip because cloning takes a lot longer)
2. Extract the zip
3. You may need to install additional packages in order to build GDB. For example,

   port install bison flex ncurses texinfo python27

4. Run configure inside the source directory

   # ARM:
   ./configure --target=arm-linux-android --with-python=yes --prefix=/nonexistent \
     --with-gdb-datadir=/nonexistent/utils --with-system-gdbinit=/nonexistent/utils/gdbinit
   # x86:
   ./configure --target=i686-pc-linux-android --with-python=yes --prefix=/nonexistent \
     --with-gdb-datadir=/nonexistent/utils --with-system-gdbinit=/nonexistent/utils/gdbinit

5. Run make

   make -j8

6. The compiled binary will be at gdb/gdb

GDBServer

Consider using the pre-built gdbserver binary from the "Using pre-built binaries" section above; gdbserver is not specific to your platform. Otherwise,

1. Use the source directory from above
2. Make sure you have a copy of the Android NDK
3. Run configure inside the gdb/gdbserver directory (following is for NDK r8d)

   export NDK=/PATH/TO/NDK
   # ARM:
   export PATH=$PATH:$NDK/toolchains/arm-linux-androideabi-4.6/prebuilt/darwin-x86/bin
   ./configure --host=arm-linux-androideabi --with-sysroot=$NDK/platforms/android-9/arch-arm
   # x86:
   export PATH=$PATH:$NDK/toolchains/x86-4.6/prebuilt/darwin-x86/bin
   ./configure --host=i686-linux-android --with-sysroot=$NDK/platforms/android-9/arch-x86

4. Run make

   make -j8

5. The compiled binary will be at gdb/gdbserver/gdbserver

Step 2. Creating JimDB directory

To get the most out of JimDB, in addition to the binaries, you need the Python scripts. But first, the scripts assume you have the following JimDB directory structure,

./bin/
    gdb
    gdbserver
./utils/
    gdbinit
    python/

Therefore, you'd need to,

1. Make a JimDB directory
2. Copy gdb and gdbserver from steps above to bin/
3. Clone the Python scripts to utils/
4. Run bin/gdb

For example,

mkdir -p jimdb/bin # step 1
cp /PATH/TO/SOURCE/gdb/gdb jimdb/bin # step 2
cp /PATH/TO/SOURCE/gdb/gdbserver/gdbserver jimdb/bin
git clone git://github.com/darchons/android-gdbutils.git jimdb/utils # step 3
jimdb/bin/gdb # step 4

Using JimDB

First run

In order to debug Fennec, JimDB needs a copy of your system libraries. When you try to debug Fennec on a particular device for the first time, JimDB will download the libraries from your device. This may take some time but will only need to run once. You may need to restart JimDB in order for debug symbols to work.

Configuring JimDB

utils/gdbinit is the GDB command file that JimDB runs at launch. Open it in your favorite text editor to see configurable settings for JimDB.

utils/gdbinit also loads utils/gdbinit.local. Because utils/gdbinit can change frequently across updates, it's recommended to copy your custom settings to utils/gdbinit.local, so that updates do not override your settings.

For example, to configure the feninit.default.objdir option, paste the following line anywhere inside utils/gdbinit.local,

python feninit.default.objdir = '/home/user/mozilla/central/objdir-android'

FenInit

FenInit is the Python script that simplifies launching Fennec on your device and setting up a debug environment. When JimDB starts, FenInit runs automatically and presents several options,

1. Debug Fennec (default)
2. Debug Fennec with env vars and args
3. Debug using jdb
4. Debug content Mochitest
5. Debug compiled-code unit test

Debug Fennec

This is the default option. Once you choose an object directory to use, the script will automatically launch Fennec, and run GDBServer to attach to Fennec.

The script tries to scan your user directory for object directories. For more control, see the Configuration section on how to specify default object directories or where to scan for object directories.

Debug Fennec with env vars and args

This option gives you the additional ability to specify environment variables and arguments when launching Fennec. See the Configuration section on how to specify additional default environment variables and arguments.

Debug using jdb

This option will let you use the Java debugger (jdb) to debug the native Java portion of Fennec. This is the default option if you already have a previous JimDB debugging session running.

JDB support is experimental. It is able to at least set breakpoints, show source code, and examine objects.

Debug content Mochitest

This option will let you debug a regular Mochitest. Make sure Fennec is compiled with the '--enable-tests' option in the mozconfig file. Because running Mochitests require a copy of the XUL Runtime Environment (XRE), the script will offer to automatically download a copy of XRE for you, as well as launching the test harness. You can specify a file or a directory to test, or set TEST_PATH. Environment variables and arguments to the test harness are also supported.

Debug compiled-code unit test

This option will let you debug a C++ test. Environment variables and arguments are supported.

Configuration

feninit.default.objdir

• Set feninit.default.objdir to the default object directory. Note that once it is set, the script will not prompt you to choose an object directory.
• Alternatively, if your object directories are called 'obj*' under your source directory, you can set feninit.default.objdir to the source directory itself. The script will scan the source directory and list all the object directories that it found.
• Note that there have been problems in the past with using ~ and $HOME in the setting, so use absolute path for best results.
• PRO-TIP: Put the following code block inside gdbinit.local to always use the current directory as the object directory,

python
import os
feninit.default.objdir = os.getcwd()
end

feninit.default.srcroot

• Unlike the previous setting, feninit.default.srcroot specifies the directory containing all the source directories. This is convenient if you have 'src/mozilla-central', 'src/mozilla-inbound', 'src/mozilla-aurora', etc. In that case, you can set feninit.default.srcroot to 'src' and all the source directories will be scanned for object directories.
• Again, use absolute path for best results.

feninit.default.no_launch

• This settings is meant for B2G, and may not be useful for Fennec debugging.

feninit.default.gdbserver_port

• To debug Fennec on your device, GDB on your computer needs to talk to GDBServer on your device through a TCP port forwarded to your device through ADB.
• Normally, JimDB will use a random port. However, you can pick a specific port to use. This is useful in certain situations such as inside an SSH session. See this blog post.

feninit.default.jdwp_port

• Similar to feninit.default.gdbserver_port, except for using JDB for debugging.

feninit.default.env

• Specify the default environment variables to use when launching Fennec. Only used for the "Debug Fennec" options.

feninit.default.args

• Specify the default arguments to use when launching Fennec. Only used for the "Debug Fennec" options.

feninit.default.cpp_env

• Similar to feninit.default.env, except only used for the "Debug compiled-code unit test" option.

feninit.default.mochi_env

• Similar to feninit.default.env, except only used for the "Debug content Mochitest" option.

feninit.default.mochi_args

• Specify the default arguments to use when debugging Mochitest. Only used for the "Debug content Mochitest" option.
• Note that Mochitest arguments are arguments to the test harness (e.g. --remote-webserver), and not arguments to Fennec itself.

feninit.default.mochi_xre

• Specify the XRE directory to use, instead of letting the script download and manage an XRE copy.

feninit.default.mochi_harness

• Specify the directory containing the harness itself, i.e. "runtestsremote.py". Not needed normally.

feninit.default.mochi_xre_url

• Specify the directory on ftp.mozilla.org to use for downloading XRE. Not used if feninit.default.mochi_xre is specified.

feninit.default.mochi_xre_update

• Specify the XRE check for updates interval. Only used if the script is managing the XRE copy, and not used if feninit.default.mochi_xre_update is specified.

GDB commands specific to JimDB

set delay-add-remote-solibs

gdb> set delay-add-remote-solibs <0|1>

Default setting is 0. Set delay-add-remote-solibs to 1 to delay loading symbols when shared libraries are loaded. This may improve the set up time. However, you may need to use the "sharedlibrary" command to manually load symbols before using breakpoints.

monitor set ignore-ondemand

gdb> monitor set ignore-ondemand <0|1>

Default setting is 1. Set ignore-ondemand to 0 to not ignore segmentation faults generated by on-demand decompression. This setting may be needed to debug specific segmentation faults that are inadvertently being ignored. This command can only be run after attaching to the target, so it cannot be specified in gdbinit.local.

FAQ

What is moz-gdb?

• moz-gdb was a synonym for JimDB, but it's no longer used, to avoid confusion

"configure: error: failure running python-config" when compiling GDB

• python may point to python3 on your machine. You need to specify where python2 is by using, for example, --with-python=python2 in your configure line

"error while loading shared libraries: libtinfo.so.5" on Linux

• You may need to create a libtinfo.so.5 symlink to libncurses.so.5. For example,

sudo ln -s libncurses.so.5 /usr/lib/libtinfo.so.5

"Segmentation fault: 11" or "Illegal instruction: 4" on Mac OS X

• This may be due to incompatible OS X SDK versions. You should build your own JimDB binary to avoid the issue.

"cannot locate symbol "__exidx_end"" when starting to debug

• If you're using pre-built binaries, you need to update to a newer version of JimDB
• If you're using custom-built binaries, you need to recompile gdbserver using a newer version of the NDK

"No module named printing" when debugging Fennec with JimDB

• The JimDB Python scripts lack several scripts included in regular GDB packages. To fix that, simply copy your local copy of these scripts to the JimDB utils/python directory. For example, on a typical Linux system, run

cp -r /usr/share/gdb/python/gdb $jimdb/utils/python/

• If you don't have these files, they are available here; just extract it under $jimdb/utils/python/

Random segmentation faults when debugging Fennec

• GDB can catch segmentation faults generated by on-demand decompression; just use "continue" to continue execution.
• Alternatively, specify the "MOZ_LINKER_ONDEMAND=0" environment variable when launching Fennec using Option 2. You can also put the option inside jimdb/utils/gdbinit.local.

GDB crashed and Fennec is stuck. Halp!!

• GDBServer may still be attached to Fennec but is unable to communicate with GDB; try killing GDBServer

adb shell run-as org.mozilla.fennec_$USER kill `adb shell ps | grep gdb | awk '{print $2}'`

"make: `[path]/mochitest-remote' is up to date." while trying to run Mochitests

• Delete the mochitest-remote file.

"failed to run gdbserver"

   "gdbserver" output:
    /system/bin/sh: /data/local/tmp/gdbserver: not executable: magic 7F45
   "run-as" output:
    /data/local/tmp/gdbserver[1]: syntax error: '(' unexpected
   "su -c" output:
    /system/bin/sh: su: not found
   [path]/jimdb-x86/bin/../utils/gdbinit:136: Error in sourced command file:
   failed to run gdbserver

You are probably trying to run the x86 version on an ARM device. Download the ARM version of JimDB instead.

Alternatively, you may just have an out-of-date version of gdbserver. Check for an updated version in the link posted above.

   "gdbserver" output:
   error: only position independent executables (PIE) are supported.

This means your build of gdbserver is too old for the Android version you're debugging on. You may need a newer platform sysroot and/or to add flags to gdb/gdbserver/Makefile to build a position-independent executable.

   CFLAGS += -fvisibility=default -fPIE
   LDFLAGS += -rdynamic -fPIE -pie

JimDB sucks! Halp!!

• Come on #mobile on IRC
• Or file a bug under Firefox for Android, JimDB component

JimDB is awesome now!! Can I buy you a $beverage?

Yes! See also bug 800000

Using `next` to traverse a function sometimes gets stuck with an error like "Unable to find end of function"

Try using the env-var MOZ_LINKER_ONDEMAND=0.

Execution pauses often for "Program received signal SIG33, Real-time event 33."

This should be caught automatically, but try:

   handle SIG33 pass nostop noprint

"Symbol not found: __PyErr_ReplaceException"

You should not use python 2.7.11.

See : https://github.com/conda/conda/issues/1367