User:Clouserw/APKFactory

C&C APK Factory

We're building an APK Factory! In short, this is a black box which will accept a manifest URL as input and return a .apk file. This project includes some UI changes on the Marketplace but the vast majority of the project is backend APIs.

The primary use case is:

• An end user, using Fennec, clicks an Install button on any web page
• Fennec retrieves the manifest linked to from the Install button and sends it to the APK Factory
• The APK Factory processes it and turns it into an APK (~seconds)
• The APK Factory returns the APK in response
• Fennec installs the APK as a normal android app

• What are we doing with existing developers? Are we opting them in to doing this conversion?
  • We are opting them in. We'll notify them first. -- David Almstrom
• Question from David Bialer - How do we handle people who have apps already installed?
  • It's possible we have something in Fx29 for this already which will do a one time check. There is a bug already filed. -- David Almstrom
  • It's unclear how many there actually are; we may be willing to have people with existing implementations reinstall their apps. We have no ADIs to check what percentage of installs are in active use. -- myk
  • This is not a blocker.
  • Bill Walker / Vishy is investigating our options: http://bugzil.la/938667 (wrong bug?)

User Stories

Use case: install

As a user, I can tap Install on a web page and an APK is downloaded to my device so that my phone's native package installer can install the app.

Use case: manual update

As a user, I can have my phone check for updates and have my phone download them so that I may keep my webapps safely up-to-date.

Implementation Note: Fennec will poll the APK factory (or, perhaps a different domain?) with a JSON blob to see if there are updates. Push notifications are not available across all our platforms so we're back to polling here.

Use case: update

As a user, I can trust that my phone will automatically keep my webapps up to date.

Not v1 Use case: developer's own keys

As a developer, I can use my own key to sign my Android APK, so I can use the key elsewhere.

Implementation Notes:

• Changing keys for an already published APK is impossible
• For a large proportion of developers, a randomly generated APK signing key will be sufficient.
• We'll need this early enough in the submission process that the first copy of their app will have the right key
• [uiwanted]: We need to make sure developers with keys actually see this as otherwise their userbase is forked.

Not v1 Use case: developer opt-out

As a Developer, I can chose not to have my app auto-generated as an Android APK: This story is obsolete since anyone could have their own marketplace and hit the APK builder and generate a package.

Implementation Note:

• [uiwanted] We need to make sure the developer realizes what is happening so their user base isn't forked. Original concern:
  • dvd: Issue: what if the developer doesn't know that his app is going to be auto-generated and installed as an APK on Android. If the OWA has not been submitted through the marketplace, the developer is not aware of this. The developer may already have the same app developed as an Android native app or an Android Webapp. If Fennec displays any webapp with an install button as the webapp thinks it will be deployed within the browser, there is no way for the developer to opt out. I am not arguing against the openness as an argument but rather for the developer to be in control.

Use case: pre-submission development

As a developer I would like to be able to test my APK without submitting it to the marketplace.

IDEAL: This should be possible without requiring a network on each edit-compile-run cycle.

Implementation Notes:

• NO use of APK factory service. Two implementation paths:
  • BEST: I register my app with the App Manager and connect my device. The manager sees the device and offers to install the app on it. I install the app on it and can launch it and use it. The manager sees that the app is running and offers to enable debugging. I enable debugging and can use the Developer Tools to debug the app.
  • ACCEPTABLE: I install the Android SDK and the APK Factory CLI. I use the Factory CLI to build an APK for the app and the Android SDK to install the APK on my device. I can then launch and use the app. And the app is configured to enable Firefox remote debugging via a known host and port, so I click Web Developer > Connect… in Firefox and connect to the app at that host/port. I can then use the Developer Tools to debug the app.
    • This is currently the solution we're aiming for, but Austin would like to have the Factory CLI be a wrapper which would communicate with the APK Factory online. Myk sounds concerned.
      • We're going to move forward with both a separate CLI Factory tool and a CLI Factory wrapper and see which is better. -- Austin

Use case: post-submission review

• As a reviewer, I would like to follow the same review process as I do for the normal app.
• As a reviewer I can run the APK on my Android phone.
• As a developer I can run the same APK as the reviewer.

Implementation Notes: The APK should not be publishable on an APK app-store, e.g. signed with a debug key

• The reviewer (or developer) will set a path on their phone to sign apps with the debug cert server, similar to how reviewers today install a reviewer cert

Use case: post-review acceptance

As a developer, I can download the APK that will be installed on user's devices, so I can distribute it myself.

Implementation Notes:

• This is a download link in the Developer control panel

Implementation

  +------------+    +-------------+    +------------+                         +-------------+
  | User Agent +--->| APK Factory +--->| APK Signer |                         |    Debug    |
  |  (Fennec)  |<---+             |<---+            |                         |     APK     |
  |            |    |             |    |            |                         |   Factory   |
  +------------+    +-------+-----+    +------------+                         +-------+-----+
                          ^ |                                                         ^ |
                          | |                                                         | |
  +-------------+         | |                  +----------------------------+         | |
  | Marketplace +---------+ |                  | Developers' and Reviewers' +---------+ |
  |             |<----------+                  |    User Agents (Fennec)    |<----------+
  |             |                              |                            |
  +-------------+                              +----------------------------+

User Agent / Fennec Modifications

• mozApps.install() and mozApps.installPackage() to send the manifest to the APK Factory
• Manual updating UI and JSON request
• Periodic updating check

Not v1 Marketplace Modifications

None of these are v1 and, once we actually get it implemented, may not be something we need to do at all.

• [uiwanted] Adjust developer flow to opt-out of automatic APK generation
• Adjust developer tools to add APK download link to versions page of the app page
• [uiwanted] Adjust developer tools to accept a user specified signing key (associated with app)
• Adjust reviewer tools to ask for Reviewer APKs (make sure the reviewer is using the debug server somehow)

APK Factory Modifications (this is a new service)

• Requires Java Development Kit (JDK) not just an environment (JRE). Requires new hardware?
• API to accept manifest as input
  • Differentiate between hosted & packaged apps
  • Do the actual packaging
    • duplicate/enhance the prototype
    • APK Factory builds the android project from a template and data in the manifest including icons, L10n, permissions, and web activities
    • For packaged webapps, the package.zip and mini-manifest files will be included in the .apk
• API to accept signing key from the Marketplace if specified
• Ability to sign using an android debug key
• Ability to sign using production app keys if signing for final packages
• API to send and receive keys from the APK Signer
• GC to clean up old packages on disk
• API to accept a JSON list of manifests and return which have updates
  • App freshness is a function of:
    • the manifest
    • changes to the android library code
    • If it's a packaged app, the package.zip (we can hash this)
  • Android Apps have a hard requirement that the versionCode, a non user facing integer representation of the version string, be positively increasing.
• Have the ability to regenerate all apps (including bumping versionCode) (useful for fixing a security hole in apk-factory-library)

• An assumption is that developers will only specify their own key if they are uploading via the Marketplace. Perhaps that isn't true? ozten brings up a good point that we'd need auth for this somewhere, marketplace makes sense...
  • dvd: wouldn't the APK Factory need to store manifest URL, version code, keys (or link to key). Would also need to check if the manifest fits the app, no? Like what we do in the marketplace if the app has significantly changed from the manifest, then it will not run.
  • Not a v1 concern

(todo:dbialer) What kind of scalability are we building this for? thousands per day? millions per day?

APK Signer Modifications (this is a new service)

• Requires new hardware
• Based heavily on solitude
• A secure storage area for production signing keys
• No outside network traffic (see above diagram)