CloudServices/NativeSync: Difference between revisions

<h1> Native Sync </h1>

<h2> Setup </h2>

<p><a href="Mobile/Fennec/Android OtherBuildEnvs#Mac_OSX"> Setting up an Android dev environment</a>

<ul><li> note: installing *all* the platform tools (all Android from 1 to 3.3) takes a while, if you only need a specific SDK version, just install from the GUI tool:

<pre class="_fck_mw_lspace">$ANDROID_HOME/tools/android

<h3> Emulators </h3>

<ul><li> to create:

<p>Use the command line <tt>$ANDROID_HOME/tools/android</tt>, or use the graphical Android AVD Manager (from within Eclipse: <tt>Window &gt; AVD Manager</tt>).

<pre class="_fck_mw_lspace">$ANDROID_HOME/tools/android create avd -t android-10 -n NAME -c 2047M

<p>produces the output

<pre class="_fck_mw_lspace">Auto-selecting single ABI armeabi

Android 2.3.3 is a basic Android platform.

Do you wish to create a custom hardware profile [no]

Created AVD 'NAME' based on Android 2.3.3, ARM (armeabi) processor,

with the following hardware config:

hw.lcd.density=240

vm.heapSize=24

hw.ramSize=256

<p><span class="fck_mw_template">{{note|Target (in the example above, &lt;tt&gt;android-10&lt;/tt&gt;) probably needs to match build specified in .mozconfig (see Building Fennec Native below).}}</span>

<ul><li> to launch:

<pre class="_fck_mw_lspace">$ANDROID_HOME/tools/emulator -wipe-data -avd NAME -partition-size 2047

<p><span class="fck_mw_template">{{note|The partition size is set large so that you don't run into an insufficient storage error.}}</span>

<ul><li> to delete:

<pre class="_fck_mw_lspace">$ANDROID_HOME/tools/android delete avd -n NAME

<p>After installation, fennec would crash on installation (unable to locate mozutils library, although it's in the apk). Possibly related to emulator failing to run NDK applications?

<h2> Running everything </h2>

<p>The main <tt>android-sync</tt> source repository is hosted on github:

<pre class="_fck_mw_lspace">https://github.com/mozilla-services/android-sync

<p>Try

<pre class="_fck_mw_lspace">git clone git@github.com:mozilla-services/android-sync.git

<h3> android-sync dependencies </h3>

<p>android-sync has the following dependencies:

<ul><li> sync-crypto, described below.

</li><li> httpclientandroidlib, which provides a working modern version of the Apache HTTPClient under a different Java package.

</li><li> json-simple 1.1 and commons-codec 1.2, which ship with Android.

</li><li> Android 2.3.3.

<p>For testing it requires:

<ul><li> JUnit 4.1.

</li><li> org.simpleframework.simple, for HTTP testing.

</li><li> Tiny “un-stub” packages for android.util.Log, android.util.Base64, and android.content.SharedPreferences.

<p>These live on github, and must be built and installed.

<ul><li><ul><li> https://github.com/rnewman/base64-unstub

</li><li> https://github.com/rnewman/log-unstub

</li><li> https://github.com/rnewman/sharedpreferences-stub

<h3> sync-crypto dependencies </h3>

<p>sync-crypto relies on Apache commons-codec 1.5, which includes Base32. The Android system libraries include the 8-year-old version 1.2. As a result, sync-crypto is packaged as an assembly jar (aka über jar).

<h3> What you need to do </h3>

<pre class="_fck_mw_lspace">git clone https://github.com/rnewman/base64-unstub

git clone https://github.com/rnewman/log-unstub

git clone https://github.com/rnewman/sharedpreferences-stub

git clone https://github.com/mozilla-services/android-sync

pushd base64-unstub;          mvn install; popd

pushd log-unstub;            mvn install; popd

pushd sharedpreferences-stub; mvn install; popd

pushd android-sync

git checkout develop

./preprocess.sh

mvn test

<p><span class="fck_mw_template">{{note|&lt;tt&gt;mvn assembly:assembly&lt;/tt&gt; may fail, however, it is still ok to proceed.}}</span>

</p><p>To do real Android development and testing, you need to import android-sync as a project in Eclipse with the ADT installed.

</p><p>Note that both android-sync and sync-crypto have JUnit 4 tests that run in both Eclipse and Maven.  The subdirectory android-sync/test includes Android JUnit 3 tests for activity and store testing.  This is to avoid the mammoth annoyance of testing in a VM without introducing robolectric.

<h3> Running Fennec </h3>

<p>The directions at <a href="Fennec/NativeUI"> Building Native Fennec</a> should be correct, specifically <a href="Fennec/NativeUI#How_To_Build"> How To Build</a>.

</p><p>After building Fennec, make the apk.

<pre class="_fck_mw_lspace">make -sj8 -C objdir-droid/ package

<p>The apk will be something like <tt>objdir-droid/dist/fennec-12.0a1.en-US.android-arm.apk</tt>.

<h4> Running Fennec on an Android device </h4>

<ul><li> Ensure USB debugging is enabled on your Android device: fom the home screen, select <tt>Settings &gt; Applications &gt; Development &gt; USB debugging</tt>.

</li><li> Ensure your Android device is connected to your computer's USB port.

</li><li> Run

<pre class="_fck_mw_lspace">$ANDROID_HOME/platform-tools/adb install -r objdir-droid/dist/fennec*apk

<h4> Running Fennec on an emulator </h4>

<ul><li> Ensure the emulator is running.

</li><li> Run

</li></ul>

<pre class="_fck_mw_lspace">$ANDROID_HOME/platform-tools/adb install -r objdir-droid/dist/fennec*apk

<p><span class="fck_mw_template">{{note|You may need to run &lt;tt&gt;adb kill-server; adb start-server&lt;/tt&gt; if adb doesn't recognize the emulator.}}</span>

<h4> Accessing Fennec/Android Sync debug statements </h4>

<pre class="_fck_mw_lspace">$ANDROID_HOME/platform-tools/adb logcat

<h3> Setting up Eclipse </h3>

<p>If you want to use Eclipse:

<ul><li> Install Eclipse and the SDK plugin, as described in the Android SDK page.

</li><li> Correct formatting: <tt>Preferences &gt; Java &gt; Code style &gt; Formatter &gt; Edit...</tt>, set profile name to <tt>Mozilla</tt>, set <tt>Tab policy</tt> to <tt>Spaces only</tt> change both <tt>Indentation size</tt> and <tt>Tab size</tt> to 2, and check <tt>Align fields in columns</tt>.

</li><li> Run these commands in your <tt>android-sync</tt> git repository directory (don't forget to checkout the <tt>develop</tt> branch):

<pre class="_fck_mw_lspace">mvn -Declipse.workspace=&lt;path-to-eclipse-workspace&gt; eclipse:add-maven-repo

mvn -DdownloadJavadocs=true -DdownloadSources=true eclipse:eclipse

<ul><li> Install the project and classpath files (again in your git repository directory), making sure to update the paths where necessary in both files:

<pre class="_fck_mw_lspace">cp example.project .project

cp example.classpath .classpath

<ul><li> Open Eclipse, choose <tt>File &gt; Import... &gt; General &gt; Existing Projects into Workspace</tt>, and specify your git repository directory.

<p><span class="fck_mw_template">{{note|You may need to set &lt;tt&gt;M2_REPO&lt;/tt&gt; (for example, as described at [http://www.mkyong.com/maven/how-to-configure-m2_repo-variable-in-eclipse-ide/]) to something like &lt;tt&gt;$HOME/.m2/repository&lt;/tt&gt;.}}</span>

<h4> To run the unit test suite under Eclipse </h4>

<ul><li> First configure the test suite launcher, under <tt>Preferences &gt; Run/Debug &gt; Launching &gt; Default Launchers</tt>.  Set the Debug and Run launchers to <tt>Android JUnit Test Launcher</tt>.

</li><li> Select the <tt>android-sync</tt> project and execute <tt>Run &gt; Run As ... &gt; JUnit Test</tt>.

<h4> To run the integration test suite under Eclipse </h4>

<ul><li> Add the <tt>test</tt> subdirectory as a sub-project using <tt>File &gt; Import &gt; Existing project</tt>.

<ul><li> Refresh and clean everything.

<ul><li> Select the <tt>test</tt> project and execute <tt>Run &gt; Run As ... &gt; Android JUnit Test</tt>.

<h2> Development </h2>

<p>Let's stick to some fairly sane Java conventions: 2-space indenting (Java is wide enough as it is), Maven2 for dependencies and build, JUnit4 for unit tests. We can wire in Hudson later if we have time. "Simple" for HTTP test server.

</p><p>Let's figure out bleeding edge stuff ("how do I get a SyncAdapter to work?") in throwaway projects. There will be a lot of these&#160;:)

</p><p>We don't quite follow the <a _fcknotitle="true" href="Fennec/NativeUI/CodingStyle">Fennec/NativeUI/CodingStyle</a>. See also <a href="http://source.android.com/source/code-style.html">the Android coding style</a>.

<h3> Running unit test code coverage </h3>

<p>To see the current unit test code coverage (using the Maven plugin integrating Cobertura), from the <tt>android-sync</tt> directory run

<pre class="_fck_mw_lspace">mvn cobertura:cobertura

<p>browse

<pre class="_fck_mw_lspace">./target/site/cobertura/index.html

<p>and click on <tt>org.mozilla.gecko</tt>.

</p><p><span class="fck_mw_template">{{note|At this time, you can't see the current integration test code coverage.}}</span>

<h2> Security </h2>

<p>Goals: no less secure than currently, at most Fennec, Service, UI have access to credentials.

</p><p><a href="http://developer.android.com/guide/topics/security/security.html">Android Docs on security </a>

</p><p>Sandboxing by using UIDs, specifically the section on "User IDs and File Access".

</p><p>See also <a href="http://developer.android.com/guide/topics/manifest/manifest-element.html">manifest entries</a>:

<h2> Permissions </h2>

<pre class="_fck_mw_lspace">  &lt;uses-permission android:name="android.permission.GET_ACCOUNTS" /&gt;

  &lt;uses-permission android:name="android.permission.MANAGE_CREDENTIALS" /&gt;

  &lt;uses-permission android:name="android.permission.USE_CREDENTIALS" /&gt;

  &lt;uses-permission android:name="android.permission.AUTHENTICATE_ACCOUNTS" /&gt;

  &lt;uses-permission android:name="android.permission.WRITE_SYNC_SETTINGS" /&gt;

  &lt;uses-permission android:name="android.permission.WRITE_SETTINGS" /&gt;

  &lt;uses-permission android:name="android.permission.READ_SYNC_STATS" /&gt;

  &lt;uses-permission android:name="android.permission.READ_SYNC_SETTINGS" /&gt;

  &lt;uses-permission android:name="android.permission.INTERNET"/&gt;

  &lt;uses-permission android:name="com.android.browser.permission.READ_HISTORY_BOOKMARKS"/&gt;

  &lt;uses-permission android:name="com.android.browser.permission.WRITE_HISTORY_BOOKMARKS"/&gt;

</pre>

<h2> Adjusting sync frequency </h2>

<p>If API &gt; 8, use ContentResolver.addPeriodicSync()

</p><p>If API = 7, create a service with a periodic timer callback, and call ContentResolver.requestSync().

</p><p>API &lt; 6 does not support syncAdapter, so no worries.

</p><p>Triggering a sync:

<pre class="_fck_mw_lspace"> android:supportsUploading defaults to true and if true an upload-only sync will be requested for all syncadapters associated with an   

authority whenever that authority's content provider does a notifyChange(android.net.Uri, android.database.ContentObserver, boolean) with  

syncToNetwork set to true.  

<h2> Altering Sync settings </h2>

<pre class="_fck_mw_lspace"> android:syncAdapterSettingsAction defaults to null and if supplied it specifies an Intent action of an activity that can be used to adjust the sync adapter's sync settings. The activity must live in the same package as the sync adapter.  

<h2> Schemas </h2>

<p>TODO

</p>

<h2> Gotchas </h2>

<p>Android's sqlite cursors are limited. Oh, and the heap is limited. We have to be very careful about how much data we store and process at one time.

<blockquote cite="http&#58;//stackoverflow.com/questions/1407442/android-sqlite-and-huge-data-sets"><p>“You are out of heap space. With a 16MB non-compacting heap, and the fact that a Cursor holds the entire result set in the heap, that is not out of the question. CursorWindow only supports 1MB of data, which is what the error message suggests more directly.</p><p>If there is a logical way to divide your queries into discrete chunks, you could do incremental queries and use CursorJoiner to stitch them together, and see if that helps.”</p></blockquote>

<h2> jvoll's notes </h2>

<p>Having trouble with the emulator when loading Fennec and Sync onto it? Use: emulator -avd android-14 -partition-size 2047

</p><p>Lines to remove from mobile/android/base/AndroidManifest.xml.in if you don't feel like working around the permissions stuff:

-android:permission="org.mozilla.gecko.permissions.BROWSER_PROVIDER"/&gt; from both providers

-&lt;permission android:name="org.mozilla.gecko.permissions.BROWSER_PROVIDER" android:protectionLevel="signature"/&gt;

</p><p>Tests failing and seeing a ProfileDatabaseException logged in logcat? This usually happens when you did a fresh install of fennec and haven't opened it yet. So, just open Fennec. If that fails (and this works for other weird stuff too), go to apps and stop Fennec, clear app data and/or in extreme cases uninstall and reinstall it. This stuff *should* all be fixed shortly, but for now these are good work arounds.

</p><p>Accessing android db on emulator:

<h2> Troubleshooting </h2>

<h3> "R cannot be resolved" </h3>

<p>Something is wrong with xml resolution. Check your Eclipse console errors in xml files, which will prevent Eclipse from building resource (R) files - resolve them first.

</p><p>Things to try:

<ul><li>Update string resources: ./preprocess

</li><li>Project &gt; Clean, then rebuild the project. Alternatively, you can delete the gen/ directory, and Eclipse will rebuild automatically (Project &gt; Build Automatically checked)

<h3> Loading apk: "Insufficient space" error on emulator </h3>

<p>Start an emulator with "-partition-size 2047" as a flag, to set emulator size.