Mobile/Fennec/Android/Development/AdvancedTopics: Difference between revisions

From MozillaWiki
< Mobile‎ | Fennec‎ | Android
Jump to navigation Jump to search
(Lifting Advanced topics out of the main "Setup" page so it's less intimidating.)
 
(Wrong page title)
 
Line 1: Line 1:
== Advanced Topics ==


=== Testing ===
See [[Mobile/Fennec/Android/Testing]].
=== Building ===
==== ccache ====
You can optionally install [https://developer.mozilla.org/en-US/docs/Ccache ccache], which can make rebuilding Firefox faster after you have built it once and don't change (much) of the C++ source code.  In general, ccache is not a performance improvement if you use only one object directory, or edit only Java and/or JavaScript source code.
If you want to use ccache, add a line the following to your mozconfig:
ac_add_options --with-ccache[=/optional/path/to/ccache]
The default cache size of 1GB is not large enough for a Fennec build. To fully utilize ccache (generally meaning you can expect full builds to complete in <5 minutes from a populated cache), you'll need to change the cache size to at least 4GB.  Do so by running:
ccache --max-size 4G
On Mac OS X, install ccache like:
brew install ccache
On Debian-like Linuxes, install ccache using:
sudo apt-get install mercurial ccache
=== Multilocale builds ===
* Create a directory, clone mozharness, copy the config file for easy editing/usage:
mkdir multilocale
cd multilocale
hg clone http://hg.mozilla.org/build/mozharness
cp mozharness/configs/multi_locale/standalone_mozilla-central.py myconfig.py
* Edit myconfig.py
** currently will check out m-c into a directory named 'mozilla-central' in this directory
** currently assumes your mozconfig is in this directory and named 'mozconfig'
** currently assumes your mozconfig sets your objdir name to 'objdir-droid'
* pull mozilla-central
mozharness/scripts/multil10n.py --cfg myconfig.py --pull-build-source
# Alternately, you can hg clone http://hg.mozilla.org/mozilla-central
* Run the script, which will create a multilocale apk
mozharness/scripts/multil10n.py --cfg myconfig.py
And you're done.
* If you want to recompile or re-run the script, restore your objdir to en-US first!
mozharness/scripts/multil10n.py --cfg myconfig.py --restore-objdir
Also see [http://escapewindow.dreamwidth.org/234671.html this blog post] for more information.
=== Single-locale language repacks ===
There is a script in mozharness for this (scripts/mobile_l10n.py) but it relies on buildbot information so it's not suitable for local repacks.
This assumes that $(AB_CD) is the locale you want to repack with; I tested with "ar" and "en-GB".
* clone l10n-central/$(AB_CD) so that it is a sibling of your mozilla-central directory
* I assume your object directory is "objdir-droid" and that you have built and packaged already
make -f client.mk && make -C objdir-droid package
* copy your .mozconfig to .mozconfig.l10n and add the following lines
# L10n
ac_add_options --with-l10n-base=../../l10n-central
# Global options
ac_add_options --disable-tests
mk_add_options MOZ_OBJDIR=./objdir-l10n
* cd to mozilla-central
* configure and prepare objdir-l10n
MOZCONFIG=.mozconfig.l10n make -f client.mk configure
make -C objdir-l10n/config
* copy your built package into objdir-l10n
cp ./objdir-droid/dist/fennec-*en-US*.apk ./objdir-l10n/dist
* unpack.  This files objdir-l10n/dist with the bits of the APK, ready for re-assembling.
make -C objdir-l10n/mobile/android/locales unpack
* compare locales (you may need to install the compare-locales tool first).  This writes locale differences into objdir-l10n/merged.
compare-locales -m objdir-l10n/merged mobile/android/locales/l10n.ini ../l10n-central $(AB_CD)
* finally, re-assemble with the locale differences
LOCALE_MERGEDIR=objdir-l10n/merged make -C objdir-l10n/mobile/android/locales installers-$(AB_CD)
You should find an APK at "objdir-l10n/dist/fennec-*$(AB_CD)*.apk".
==== Enabling C++ debugging ====
If you want to create a build suitable for debugging the C++ code, add:
ac_add_options --enable-debug-symbols
==== Building for the x86 architecture ====
If you want to build for x86, substitute:
ac_add_options --target=i386-linux-android
for the target specified above.  Note: you cannot use <tt>ac_add_options --disable-optimize</tt> when building for x86.  See {{bug|965870}}.
==== Building for the ARMv6 architecture ====
'''ARMv6 is no longer a supported target as of Firefox 33.'''
If you want to build for ARMv6 instead of ARMv7 (only required for very old, very low end phones), add:
ac_add_options --with-arch=armv6
==== Unsupported build flags ====
Do '''not''' specify these flags because the resulting build will cause out-of-memory crashes:
ac_add_options --disable-install-strip # Do not use!
==== Updating the builders` SDK ====
i.e. Nightly builds & treeherder. See [[Mobile/Fennec/Android/Updating_SDK_on_builders]].
=== Troubleshooting ===
===== Don't set CC / CXX environmental variables =====
If you've set the environmental variables CC and CXX (e.g. via .bash_aliases or via your mozconfig), then you probably need to unset them before building for Android, or else your build may fail with something like:
checking whether the C compiler (gcc -mandroid -fno-short-enums (etc etc)) works... no
followed by errors about "C compiler cannot create executables", "Relocations in generic ELF (EM: 40)", and "crtbegin_dynamic.o: error adding symbols: File in wrong format". This is a sign that you're compiling with your platform's native compiler (due to having CC / CXX set), instead of the android-specific GCC version that ships with the NDK.  See {{bug|977817}} for more details; as noted there, the build system may trust your custom CC & CXX variables, when you probably don't want it to.
=== Coding Gotchas ===
==== Closing resources ====
When handling resources (like Cursors), a try/finally block should be used to ensure these are closed properly. For example:
final Cursor c = getCursor();
try {
    useCursorWhichMightThrowException(c);
} catch (SomeSpecificException sse) {
    log(sse);
} finally {
    c.close();
}
Once the try block is entered, the finally block will *always* get executed upon exit of the try block. The one exception is if there is a System.exit call inside the try block, which immediately exits the program and makes everything moot anyway. The finally block will get executed on caught and uncaught exceptions, as well as normal returns.
If you are casting the resource to something, make sure that you do the cast inside the try block, like so:
// GOOD!
InputStream is = getInputStream();
try {
    FileInputStream fis = (FileInputStream) is;
    ...
} finally {
    ...
}
rather than doing this:
// BAD!
FileInputStream fis = (FileInputStream) getInputStream();
try {
    ...
} finally {
    ...
}
This is so that in case of ClassCastExceptions you don't get a dangling open resource left behind.
==== Caveats for Timing ====
TLDR: Google recommends using SystemClock.uptimeMillis() for general purpose interval timing of user interface events or performance measurements. If you're adding stuff for timing, use SystemClock.uptimeMillis(), rather than something like new Date().getTime().
Normally in Java the default time-getter is System.currentTimeMillis() since it avoids the overhead of creating a new Date object. This is also what new Date() does under the hood. However, currentTimeMillis() and the Date object are both subject to change in unexpected ways if the user changes the time on their device, or if daylight savings comes into effect, or there's a network time update, or whatever. So Android has generously provided android.os.SystemClock which has various functions that you can use to get a better timestamp. Refer to the class javadoc and pick whichever function is most suitable for what you're trying to measure.
http://developer.android.com/reference/android/os/SystemClock.html
== Advanced Debugging ==
=== Using logcat ===
[http://developer.android.com/guide/developing/tools/logcat.html logcat] is a tool that is going to show you some logs prompted by the device. It might be a good help if you don't want to or can't run gdb. You can use it by running this command:
  adb logcat -v time
You can make things appear in logcat using printf_stderr. With debug builds, NS_WARNING, NS_ERROR and NS_ASSERTIONS will show up in logcat. If you're trying to debug something, you may wish to pipe the logcat output through grep to filter out irrelevant things (most Fennec-related output will be viewable by
  adb logcat -v time | grep Gecko
but remember that when attaching log output to a bug you should include unfiltered output as there may be relevant log entries under other tags.
==== Using other logcat apps ====
===== ICS (4.1) and below =====
* Install the [https://market.android.com/details?id=org.jtb.alogcat&hl=en aLogCat] app. Use it to capture logs and attach the logs to bugs.
Once you have alogcat installed, just use Fennec as you would normally. Upon encountering a bug or issue, start the aLogcat app (as soon as possible after seeing the Fennec issue) and select "Share" or "Save" from the menu to send it via email or save it to the SD card. The log can then be attached to a bug or sent to a developer. As with adb logcat, it is better to have a log with timestamps than without timestamps. To enable timestamps in the log, select "Preferences" from the aLogcat menu, and change the "Format?" option to "Time".
If you need to, you can search for some kinds of Fennec-related output by using the "Filter" menu item and entering "Gecko". However, when submitting logs for bug reports, please make sure you clear the filter and include all of the available log data.
===== Jelly Bean (4.2) and above =====
* Install the [https://addons.mozilla.org/en-US/android/addon/logview/ LogView add-on]
* Root the device and use the [https://market.android.com/details?id=org.jtb.alogcat&hl=en aLogCat] app as above.
==== JavaScript dump() ====
To use the dump() function in JavaScript to write to the log:
# Go to about:config and set browser.dom.window.dump.enabled to "true"
# Run the following ADB commands:
adb shell stop
adb shell setprop log.redirect-stdio true
adb shell start
=== Using JimDB ===
See [[Mobile/Fennec/Android/GDB]]
=== Using Debug Intent ===
'''Note: this is not useful with JimDB. If you want to use JimDB, just start it.'''
In order to attach before things get running, launch with:
  adb shell am start -a org.mozilla.gecko.DEBUG -n org.mozilla.fennec_foobar/.App
(Replace foobar by your username)
and just click launch once gdb is attached. If you need to debug a crash that happens before XRE_Main is called, the patch on {{bug|572247}} may be useful.
this script [http://dump.lassey.us/debug.sh] will attach gdbserver for you
=== Getting dalvik java stack dumps using gdb ===
(gdb) call dvmDumpAllThreads(true)
this will dump a stack trace to logcat
Note: this will only work if you have symbols for dalvik.
=== Debugging with jdb ===
JimDB can now launch JDB integration
You can also use eclipse for debugging in a similar way by setting up for debugging "Remote Java application".
=== Debugging with eclipse ===
You need to find the PID of your fennec process. Forward it to a local TCP socket as in "Debugging with jdb."
In Eclipse switch to the debug perspective. Go to Run > Debug configurations... Remote Java Application. Change the port to the TCP port you specified in your adb command. Under Source, navigate to your checkout, then into mobile/android.
Eclipse looks for source code in a specific location. You need to create the directory hierarchy:
<code>
mobile
  /android
    /org
      mozilla/
        gecko -> ../../base</code>
That is, in mozilla-central/mobile/android, create org/mozilla, and put the symlink gecko pointing to mozilla-central/mobile/android/base.
You may also want to add more debugging information and can do that like this:
diff --git a/config/android-common.mk b/config/android-common.mk
index 4591239..a47726a 100644
--- a/config/android-common.mk
+++ b/config/android-common.mk
@@ -70,6 +70,6 @@ JAVAC_FLAGS = \
    -classpath $(JAVA_CLASSPATH) \
    -bootclasspath $(JAVA_BOOTCLASSPATH) \
    -encoding UTF8 \
-  -g:source,lines \
+  -g:source,lines,vars \
    -Werror \
    $(NULL)
=== Arguments and Environment Variables ===
If you need to set an environment variable at run time, append '''--es env# VAR=VAL''' to your activity manager command where # is the ordered number of variables for example:
  adb shell am start -a android.activity.MAIN -n org.mozilla.fennec_$USER/.App --es env0 VAR=val --es env1 FOO=bar
If you need to pass arguments at run time, append '''--es args "<your-args>"''' to your activity manager command. For example, to launch with a specific profile:
  adb shell am start -a android.activity.MAIN -n org.mozilla.fennec_$USER/.App --es args "--profile /mnt/sdcard/myprofile"
To launch with a specific URL, use the am -d option to set the intent's data URI:
  adb shell am start -a android.activity.MAIN -n org.mozilla.fennec_$USER/.App -d 'http://www.mozilla.org'
=== PR Logging ===
You can use the env vars as described above to enable NSPR logging:
  adb shell am start -a android.activity.MAIN -n org.mozilla.fennec_$USER/.App --es env0 NSPR_LOG_MODULES=all:5 --es env1 NSPR_LOG_FILE=/mnt/sdcard/log.txt
If no file is specified, logging is directed to the android logs:
  adb shell am start -a android.activity.MAIN -n org.mozilla.fennec_$USER/.App --es env0 NSPR_LOG_MODULES=all:5
Look for lines marked "PRLog" in the adb logcat output.
=== Using legacy GDB (non-JimDB)  ===
See [[Mobile/Fennec/Android/GDBNoRoot|Fennec/Android/GDBNoRoot]]
=== Reading back the framebuffer ===
If you need to verify what is in the back buffer at a particular time, you can cleverly call functions from gdb to allocate memory, read back, and write that to disk.
You need to know the size of your framebuffer a priori; in this case, it's 480x699.
You also need to know what the GL enum values are, because unless you compile with -ggdb, you don't have #defines available to you in the debugger. Very helpful information: GL_RGBA = 0x1908, GL_UNSIGNED_BYTE = 0x1401. The rest you can find in gfx/gl/GLDefs.h.
You '''should''' be able to call glReadPixels directly, but in my experience that causes Fennec to crash. However, if you have a GLContext* lying around, as you often do, you can work around that problem.
<pre>(gdb) set $m = (int*)malloc(480*699*4)
(gdb) call aManager->mGLContext.mRawPtr->fReadPixels(0, 0, 480, 699, 0x1908, 0x1401, (void*)$m)
(gdb) set $f = fopen("/sdcard/outputfile", "wb+")
(gdb) call fwrite($m, 1, 480*699*4, $f)
$7 = 1342080
(gdb) call fclose($f)
$8 = 0</pre>
Now there is a file called /sdcard/outputfile that you can adb pull. But since it's just raw RGBA values, you need to be able to wrap that in PNG headers to display it. [https://github.com/jrmuizel/minpng/blob/master/minpng.h Jeff Muizelaar wrote a header called minpng.h that you can use to do so.]
Get Jeff's minpng.h, and put it in a directory along with a driver c program:
<pre>#include "minpng.h"
int main(int argc, char* argv[])
{
FILE* f = fopen(argv[1], "rb");
int w = atoi(argv[2]);
int h = atoi(argv[3]);
char* d = (char*) malloc(w * h * 4);
fread(d, w * h * 4, 1, f);
fclose(f);
write_png(argv[4], d, w, h);
}</pre>
Compile and run:
<pre>$ gcc -o minpng minpng.c
$ ./minpng outputfile 480 699 output.png
</pre>
=== Using Rendertrace (Maple) ===
Rendertrace is a utility that will dump layer position and timing information (such as drawing, upload) to the console. This information can pasted into the rendertrace web front end to visualize the layer position and event timeline. This will let you understand where you're gecko is spending its time and why were checkerboarding.
To enable go in 'gfx/layers/RenderTrace.h' and uncomment '#define MOZ_RENDERTRACE'. Rebuild and run 'adb logcat | grep RENDERTRACE', paste the result in http://people.mozilla.org/~bgirard/rendertrace.html and hit 'reload'. For details talk to BenWa.
=== Using apitrace ===
Apitrace is a tool for tracing GL/EGL calls for debugging purposes. It basically uses an interim shared library called libapitrace that contains shadow gl* and egl* functions, which then get logged and then passed through to the real driver.
Use apitrace from https://github.com/apitrace/apitrace to build for desktop and android.
<pre>
apt-get install libegl1-mesa-dev libgles1-mesa-dev libgles2-mesa-dev libqt4-dev cmake
git clone https://github.com/gw280/apitrace.git
cd apitrace
# Build for Android
cmake -DANDROID_NDK=/path/to/your/ndk -DCMAKE_TOOLCHAIN_FILE=android/android.toolchain.cmake -DANDROID_API_LEVEL=9 -Bbuild-android -H.
make -C build-android -j8
# Build for desktop
cmake -H. -Bbuild
make -C build -j8
export EGL_SOFTWARE=true
./build/eglretrace -v /path/to/your/apitrace_log.trace
</pre>
The Android build will create egltrace.so in build-android/wrappers, which you can then push to your device to /data/local:
<pre>
adb push build-android/wrappers/egltrace.so /data/local/tmp
</pre>
Restarting Fennec will cause it to load the apitrace library and the apitrace log will be saved to /data/data/org.mozilla.fennec_username/firefox.trace
You can then adb pull /data/data/org.mozilla.fennec_username/firefox.trace and analyse it on your desktop.
You can also use qapitrace as a GUI to inspect your trace files. (be sure to switch qapitrace to the EGL api using the options dialog)
These instructions provide a trace that does not include the Java GL code. To get traces including java code is more complicated. You need to use
the patch from this bug https://bugzilla.mozilla.org/show_bug.cgi?id=749859 and this version of https://github.com/ideak/apitrace/tree/dev. Further, you'll need to build your own image/modify the current one to replace /init.rc. You also need to disable hardware acceleration of the UI (https://bug746703.bugzilla.mozilla.org/attachment.cgi?id=619009) Ask jrmuizel for more information if you want to do this.
=== about:memory ===
about:memory provides heaps (ha!) of useful memory information.
You can obtain a snapshot of memory info from a running Fennec instance using:
  adb shell am broadcast -a org.mozilla.gecko.MEMORY_DUMP
This dumps a json file to the SD card and prints out the exact filename to logcat. You can pull the json file to desktop using
  adb pull <absolute-path-to-file>
and view it in firefox's about:memory: use the "Read reports from a file" option at the bottom of the about:memory page.
=== Profiling ===
See https://wiki.mozilla.org/Mobile/Fennec/Android/Profiling.
=== Tweaking UI prefs ===
By default, all of these prefs are set to "-1" in Fennec, meaning they take the values listed below, which are maintained in Axis.java.
Fractional values are specified in 1/1000th of a value; to specify a value of 0.3, write 300.
Note: You need to restart Fennec after changing these values.
{|
! Pref !! Default value !! Description !!
|-
| ui.scrolling.friction_slow || 850 || This fraction in 1000ths of velocity remains after every animation frame when the velocity is low.||
|-
| ui.scrolling.friction_fast || 970 || This fraction in 1000ths of velocity remains after every animation frame when the velocity is high.||
|-
| ui.scrolling.velocity_threshold || 10 || Below this velocity (in pixels per frame), the friction changes from friction_fast to friction_slow.||
|-
| ui.scrolling.max_event_acceleration || 12 || The maximum velocity change factor between events, per ms, in 1000ths. ||
|-
| ui.scrolling.overscroll_decel_rate || 40 || The rate of deceleration when the surface has overscrolled, in 1000ths. ||
|-
| ui.scrolling.overscroll_snap_limit || 300 || The fraction of the surface which can be overscrolled before it must snap back, in 1000ths. ||
|-
| ui.scrolling.min_scrollable_distance || 500 || The minimum amount of space that must be present for an axis to be considered scrollable, in 1/1000ths of pixels. ||
|-
| gfx.displayport.strategy || 1 || The strategy we use to determine how display ports are calculated. 0 = fixed margin, 1 = velocity bias, 2 = dynamic resolution, 3 = no margins ||
|-
| gfx.displayport.strategy_fm.multiplier || 1500 || When gfx.displayport.strategy = 0 (fixed margin), the 1000th of each dimension of the viewport the displayport is sized to. ||
|-
| gfx.displayport.strategy_fm.danger_x || 100 || When gfx.displayport.strategy = 0 (fixed margin), the 1000th of the width of the viewport the horizontal danger zone is set to.
Danger zone is defined as the space at the edge of the viewport at which the viewport (and hence displayport) starts being changed.
|-
| gfx.displayport.strategy_fm.danger_y || 200 || When gfx.displayport.strategy = 0 (fixed margin), the 1000th of the height of the viewport the vertical danger zone is set to. ||
|-
| gfx.displayport.strategy_vb.multiplier || 1500 || When gfx.displayport.strategy = 1 (velocity bias), the 1000th of each dimension of the viewport the displayport is sized to. ||
|-
| gfx.displayport.strategy_vb.threshold || 32 || When gfx.displayport.strategy = 1 (velocity bias), the threshold for velocity, in pixels/frame, when multiplied by the screen DPI. ||
|-
| gfx.displayport.strategy_vb.reverse_buffer || 200 || When gfx.displayport.strategy = 1 (velocity bias), the fraction of the buffer (in 1000ths) to be kept in the direction opposite the direction of the scroll. ||
|-
|}
=== Invalidate the JavaScript startup cache ===
To make life easier for developers, '''in local development builds only''', {{bug|976216}} invalidates the JavaScript startup cache every time Firefox for Android starts.  On non-local development builds (including TBPL builds and try builds), the JavaScript startup cache is not invalidated at startup.
Background: JavaScript files and modules (like <code>browser.js</code>) are cached for fast startup.  The cache is invalidated only when an internal build ID is updated, which only happens when certain C++ code is rebuilt.  That doesn't happen for most patches that only touch Java and JavaScript within <code>mobile/android</code>.  See {{bug|695145}} for the details of this build ID handling.  If you happen to be building a non-local development build in some way, you might need to invalidate the JavaScript startup cache by touching <code>toolkit/xre/nsAndroidStartup.cpp</code> and rebuilding libxul.
=== killer script ===
#!/bin/sh
if [ $# -ne 1 ]
then
    echo "usage: $0  packagename.of.your.activity"
    echo "for example:"
    echo "  $0 org.mozilla.fennec"
    exit
fi
p=`adb shell ps | grep $1 | awk '{print $2}'`
if [ "$p" = "" ];
then
    echo "ERROR: That doesn't seem to be a running process. Please make sure your"
    echo "application has been started and that you are using the correct"
    echo "namespace argument."
    exit
fi
adb shell run-as $1 kill $p
=== .gdbinit ===
This is an example .gdbinit that uses the symbols from a locally built rom and automatically attaches to gdbserver. Note that putting a ''.gdbinit'' file inside a directory will make gdb load it thus you will not pollute your regular gdb init with those configurations.
set solib-search-path /home/blassey/android/system/out/target/product/passion/symbols/system/bin:/home/blassey/android/system/out/target/product/passion/symbols/system/lib/:/home/blassey/src/ndk5-m-c/objdir-droid-dbg/dist/bin
set solib-absolute-prefix /home/blassey/android/system/out/target/product/passion/symbols/system/lib/
target remote localhost:12345
=== Rooting Android devices ===
See [[Mobile/Fennec/Android/Rooting|Rooting Android Devices]].
=== Sign a Fennec build ===
Nightly builds are available unsigned, so that you can sign them with your local debug key and install them on top of your own debug builds (without uninstalling and losing your profile). To sign and install the unsigned nightly build:
   
  wget http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/latest-mozilla-central-android-r7/gecko-unsigned-unaligned.apk
  jarsigner -sigalg SHA1withRSA -digestalg SHA1 -keystore ~/.android/debug.keystore -storepass android -keypass android gecko-unsigned-unaligned.apk androiddebugkey
  zipalign -f -v 4 gecko-unsigned-unaligned.apk gecko-signed-aligned.apk
  adb install -r gecko-signed-aligned.apk
Or you can also re-sign a signed build. If "fennec.apk" is signed already, this will remove the signature and replace it with your own. The result will be saved as "fennec-resigned.apk":
   
  zip fennec.apk -d META-INF/*
  jarsigner -sigalg SHA1withRSA -digestalg SHA1 -keystore ~/.android/debug.keystore -storepass android -keypass android fennec.apk androiddebugkey
  zipalign -f -v 4 fennec.apk fennec-resigned.apk
If you get this error when you try to sign a package:
   
  jarsigner: unable to sign jar: java.util.zip.ZipException: invalid entry compressed size (expected 16716 but got 16964 bytes)
   
You should to follow some steps to complete your task:
   
* rename the .apk to .zip
* unzip the file in some folder
* remove the METAINF folder
* zip the remaining files
* change the name .zip to .apk
* sign again
To verify if everything is alright use the command
   
  jarsigner -verbose -verify
=== Useful Addons ===
There are some addons which may be useful for development purposes. See [[Mobile/Fennec/Android/Development/Addons]].

Latest revision as of 01:15, 11 September 2015

Retrieved from "https://wiki.mozilla.org/index.php?title=Mobile/Fennec/Android/Development/AdvancedTopics&oldid=1094764"