CloudServices/NativeSync

Setup

note: installing *all* the platform tools (all Android from 1 to 3.3) takes a while, if you only need a specific SDK version, just install from the GUI tool:

$ANDROID_HOME/tools/android

Emulators

to create:

Use the command line $ANDROID_HOME/tools/android, or use the graphical Android AVD Manager (from within Eclipse: Window > AVD Manager).

$ANDROID_HOME/tools/android create avd -t android-10 -n NAME -c 2047M

produces the output

Auto-selecting single ABI armeabi
Android 2.3.3 is a basic Android platform.
Do you wish to create a custom hardware profile [no]
Created AVD 'NAME' based on Android 2.3.3, ARM (armeabi) processor,
with the following hardware config:
hw.lcd.density=240
vm.heapSize=24
hw.ramSize=256

Note: Target (in the example above, android-10) probably needs to match build specified in .mozconfig (see Building Fennec Native below).

to launch:

$ANDROID_HOME/tools/emulator -wipe-data -avd NAME -partition-size 2047

Note: The partition size is set large so that you don't run into an insufficient storage error.

to delete:

$ANDROID_HOME/tools/android delete avd -n NAME

After installation, fennec would crash on installation (unable to locate mozutils library, although it's in the apk). Possibly related to emulator failing to run NDK applications?

Running everything

The main android-sync source repository is hosted on github:

https://github.com/mozilla-services/android-sync

Try

git clone git@github.com:mozilla-services/android-sync.git

android-sync dependencies

android-sync has the following dependencies:

httpclientandroidlib, which provides a working modern version of the Apache HTTPClient under a different Java package.
json-simple 1.1 and commons-codec 1.2, which ship with Android.
Android 2.3.3.

For testing it requires:

JUnit 4.1.
org.simpleframework.simple, for HTTP testing.
Tiny “un-stub” packages for android.util.Log, android.util.Base64, and android.content.SharedPreferences.

These live on github, and *must be built and installed*.

What you need to do

git clone https://github.com/rnewman/base64-unstub
git clone https://github.com/rnewman/log-unstub
git clone https://github.com/rnewman/sharedpreferences-stub
git clone https://github.com/mozilla-services/android-sync
pushd base64-unstub;          mvn install; popd
pushd log-unstub;             mvn install; popd
pushd sharedpreferences-stub; mvn install; popd
pushd android-sync
git checkout develop
./preprocess.sh
mvn test

Note: mvn assembly:assembly may fail, however, it is still ok to proceed.

To do real Android development and testing, you need to import android-sync as a project in Eclipse with the ADT installed.

Note that android-sync has JUnit 4 tests that run in both Eclipse and Maven. The subdirectory android-sync/test includes Android JUnit 3 tests for activity and store testing. This is to avoid the mammoth annoyance of testing in a VM without introducing robolectric.

Running Fennec

The directions at Building Native Fennec should be correct, specifically How To Build.

After building Fennec, make the apk.

make -sj8 -C objdir-droid/ package

The apk will be something like objdir-droid/dist/fennec-12.0a1.en-US.android-arm.apk.

Running Fennec on an Android device

Ensure USB debugging is enabled on your Android device: fom the home screen, select Settings > Applications > Development > USB debugging.
Ensure your Android device is connected to your computer's USB port.
Run

$ANDROID_HOME/platform-tools/adb install -r objdir-droid/dist/fennec*apk

Running Fennec on an emulator

Ensure the emulator is running.
Run

$ANDROID_HOME/platform-tools/adb install -r objdir-droid/dist/fennec*apk

Note: You may need to run adb kill-server; adb start-server if adb doesn't recognize the emulator.

Accessing Fennec/Android Sync debug statements

$ANDROID_HOME/platform-tools/adb logcat

Setting up Eclipse

If you want to use Eclipse:

Install Eclipse and the SDK plugin, as described in the Android SDK page.
Correct formatting: Preferences > Java > Code style > Formatter > Edit..., set profile name to Mozilla, set Tab policy to Spaces only change both Indentation size and Tab size to 2, and check Align fields in columns.
Run these commands in your android-sync git repository directory (don't forget to checkout the develop branch):

mvn -Declipse.workspace=<path-to-eclipse-workspace> eclipse:add-maven-repo
mvn -DdownloadJavadocs=true -DdownloadSources=true eclipse:eclipse

Install the project and classpath files (again in your git repository directory), making sure to update the paths where necessary in both files:

cp example.project .project
cp example.classpath .classpath

Open Eclipse, choose File > Import... > General > Existing Projects into Workspace, and specify your git repository directory.

Note: You may need to set M2_REPO (for example, as described at [1]) to something like $HOME/.m2/repository.

To hide warnings in external code

You'll need Eclipse 4.2, also called "Juno". Right click an external folder (for example, external/httpclientandroidlib/httpclientandroidlib/src), select Properties > Java Compiler, and check Ignore optional compile problems. This prevents warnings from appearing in the Problems pane, and also means that your Eclipse project icon won't always be the yellow warning exclamation mark. This makes it easier to see if you've introduced Java warnings to the tree.

To run the unit test suite under Eclipse

First configure the test suite launcher, under Preferences > Run/Debug > Launching > Default Launchers. Set the Debug and Run launchers to Android JUnit Test Launcher.
Select the android-sync project and execute Run > Run As ... > JUnit Test.

To run the integration test suite under Eclipse

Add the test subdirectory as a sub-project using File > Import > Existing project.

Refresh and clean everything.

Select the test project and execute Run > Run As ... > Android JUnit Test.

To add JUnit 4 import support

From http://stackoverflow.com/questions/288861/eclipse-optimize-imports-to-include-static-imports:

From Preferences > Java > Editor > Content Assist > Favorites, choose Add Type and enter "org.junit.Assert". Then Shift + Ctrl + O and Alt + 1 should automatically find assertEquals, fail, and friends.

Development

Let's stick to some fairly sane Java conventions: 2-space indenting (Java is wide enough as it is), Maven2 for dependencies and build, JUnit4 for unit tests. We can wire in Hudson later if we have time. "Simple" for HTTP test server.

Let's figure out bleeding edge stuff ("how do I get a SyncAdapter to work?") in throwaway projects. There will be a lot of these :)

We don't quite follow the Mobile/Fennec/Android#Coding_Style. See also the Android coding style.

Formatting Javadoc comments

Document all parameters and any return value; always start parameter and return value documentation lower case. Please put one blank line after summary and before documenting parameters and return types. Java is verbose enough already; please don't include type information unless it is truly useful. Please don't include author lines; we have git blame and big sticks for that already. Use paragraph, code, and list HTML tags when appropriate.

Eclipse will format Javadoc comments for you: select the entire comment and (by default) type Command-Shift-F. This introduces trailing blanks; remove them by hand. You can preview your formatted Javadoc by hovering over the function name in Eclipse. Example:

  /**
   * Generate a "plain" auth token.
   * <p>
   * Android caches only the value of the key
   * <code>AccountManager.KEY_AUTHTOKEN</code>, so if a caller needs the other
   * keys in this bundle, it needs to invalidate the token (so that the bundle
   * is re-generated).
   *
   * @param context
   *          Android context.
   * @param account
   *          Android account.
   * @return a <code>Bundle</code> instance containing a subset of the following
   *         keys: (caller's must check for missing keys)
   *         <ul>
   *         <li><code>AccountManager.KEY_ACCOUNT_TYPE</code>: the Android
   *         Account's type</li>
   *
   *         <li><code>AccountManager.KEY_ACCOUNT_NAME</code>: the Android
   *         Account's name</li>
   *
   *         <li><code>AccountManager.KEY_AUTHTOKEN</code>: the Sync account's
   *         password </li>
   *
   *         <li><code> Constants.OPTION_USERNAME</code>: the Sync account's
   *         hashed username</li>
   *
   *         <li><code>Constants.OPTION_SERVER</code>: the Sync account's
   *         server</li>
   *
   *         <li><code> Constants.OPTION_SYNCKEY</code>: the Sync account's
   *         sync key</li>
   *
   *         </ul>
   * @throws NetworkErrorException
   */
  public static Bundle getPlainAuthToken(final Context context, final Account account)
      throws NetworkErrorException {
    ...
  }

Running unit test code coverage

To see the current unit test code coverage (using the Maven plugin integrating Cobertura), from the android-sync directory run

mvn cobertura:cobertura

browse

./android-sync-app/target/site/cobertura/index.html

and click on org.mozilla.gecko.

Note: At this time, you can't see the current integration test code coverage.

Signing Fennec (for testing multiple Fennecs)

Follow the instructions at http://developer.android.com/intl/zh-CN/tools/publishing/app-signing.html.

The signing happens in place; try:

cp $OBJDIR/dist/gecko.ap_ $OBJDIR/dist/custom.signed.apk
jarsigner -verbose -sigalg MD5withRSA -digestalg SHA1 -keystore ~/Mozilla/nalexander-keystore $OBJDIR/dist/custom.signed.apk nalexander-keystore

Security

Goals: no less secure than currently, at most Fennec, Service, UI have access to credentials.

Android Docs on security

Sandboxing by using UIDs, specifically the section on "User IDs and File Access".

Permissions

   <uses-permission android:name="android.permission.GET_ACCOUNTS" />
   <uses-permission android:name="android.permission.MANAGE_CREDENTIALS" />
   <uses-permission android:name="android.permission.USE_CREDENTIALS" />
   <uses-permission android:name="android.permission.AUTHENTICATE_ACCOUNTS" />
   <uses-permission android:name="android.permission.WRITE_SYNC_SETTINGS" />
   <uses-permission android:name="android.permission.WRITE_SETTINGS" />
   <uses-permission android:name="android.permission.READ_SYNC_STATS" />
   <uses-permission android:name="android.permission.READ_SYNC_SETTINGS" />
   <uses-permission android:name="android.permission.INTERNET"/>
   <uses-permission android:name="com.android.browser.permission.READ_HISTORY_BOOKMARKS"/>
   <uses-permission android:name="com.android.browser.permission.WRITE_HISTORY_BOOKMARKS"/>

Adjusting sync frequency

If API > 8, use ContentResolver.addPeriodicSync()

If API = 7, create a service with a periodic timer callback, and call ContentResolver.requestSync().

API < 6 does not support syncAdapter, so no worries.

Triggering a sync:

 android:supportsUploading defaults to true and if true an upload-only sync will be requested for all syncadapters associated with an   
 authority whenever that authority's content provider does a notifyChange(android.net.Uri, android.database.ContentObserver, boolean) with 
 syncToNetwork set to true.

Altering Sync settings

 android:syncAdapterSettingsAction defaults to null and if supplied it specifies an Intent action of an activity that can be used to adjust the sync adapter's sync settings. The activity must live in the same package as the sync adapter.

Localization

Strings used in Firefox need to be localized into ALL THE LANGUAGES. Here's how to add a string.

Add the string as an entity to ./strings/sync_strings.dtd.in. These are the actual strings that get localized, and should never be changed after landing. If you do need to make a change, make a new string entity, and do not change or delete any previous ones. For example:

<!ENTITY sync.title.success.label 'Setup Complete'>

Add the entity to ./strings/strings.xml.in. For example:

<string name="sync_title_success">&sync.title.success.label;</string>

Run the preprocess script to generate the strings for use:

$ ./preprocess.sh 
Using ANDROID_PACKAGE_NAME org.mozilla.fennec_ncalexan.
Using MOZ_APP_DISPLAYNAME FxSync.
Using MOZ_APP_VERSION 0.
Using MOZ_ANDROID_SHARED_ID org.mozilla.fennec_ncalexan.sharedID.
Using MOZ_ANDROID_SHARED_ACCOUNT_TYPE org.mozilla.fennec_ncalexan_sync.

Preprocessing makes the string available in /res/values/strings.xml. Once preprocessed, strings can be accessed from resource XML files:

<TextView android:text="@string/sync_title_success" />

and Java code:

String string = context.getString(R.string.sync_title_sucess);

It's definitely a good idea to look over the Android Localization guide.

Schemas

TODO

Gotchas

Android's sqlite cursors are limited. Oh, and the heap is limited. We have to be very careful about how much data we store and process at one time.

“You are out of heap space. With a 16MB non-compacting heap, and the fact that a Cursor holds the entire result set in the heap, that is not out of the question. CursorWindow only supports 1MB of data, which is what the error message suggests more directly.
If there is a logical way to divide your queries into discrete chunks, you could do incremental queries and use CursorJoiner to stitch them together, and see if that helps.”

jvoll's notes

Having trouble with the emulator when loading Fennec and Sync onto it? Use: emulator -avd android-14 -partition-size 2047

Lines to remove from mobile/android/base/AndroidManifest.xml.in if you don't feel like working around the permissions stuff: -android:permission="org.mozilla.gecko.permissions.BROWSER_PROVIDER"/> from both providers -<permission android:name="org.mozilla.gecko.permissions.BROWSER_PROVIDER" android:protectionLevel="signature"/>

Tests failing and seeing a ProfileDatabaseException logged in logcat? This usually happens when you did a fresh install of fennec and haven't opened it yet. So, just open Fennec. If that fails (and this works for other weird stuff too), go to apps and stop Fennec, clear app data and/or in extreme cases uninstall and reinstall it. This stuff *should* all be fixed shortly, but for now these are good work arounds.

Accessing android db on emulator: adb shell cd /data/data/org.mozilla.fennec_jason/files/mozilla/yvzvl036.default sqlite3 browser.db (or w/e database is wanted)

Troubleshooting

"R cannot be resolved"

Something is wrong with xml resolution. Check your Eclipse console errors in xml files, which will prevent Eclipse from building resource (R) files - resolve them first.

Things to try:

Update string resources: ./preprocess
Project > Clean, then rebuild the project. Alternatively, you can delete the gen/ directory, and Eclipse will rebuild automatically (Project > Build Automatically checked)

Loading apk: "Insufficient space" error on emulator

Start an emulator with "-partition-size 2047" as a flag, to set emulator size.