User:Clouserw/APKFactory
C&C APK Factory
We're building an APK Factory! In short, this is a black box which will accept a manifest URL as input and return a URL to where an APK of the app at the manifest URL can be downloaded. This project includes some UI changes on the Marketplace but the vast majority of the project is backend and APIs.
The primary use case is:
- Developer uploads new app to the Marketplace
- Marketplace validates the app
- Marketplace puts app in queue for reviewers
- Marketplace dispatches information to the APK Factory and requests a reviewer copy of the APK (that means signed with a debug key)
- APK Factory returns a URL which will hold an APK when ready (maybe return 202 until it's there?)
- Marketplace retrieves APK from URL and caches locally
- When app is approved, Marketplace dispatches a second request to APK factory asking for a production copy of the APK (signed with a real per-app key)
- APK Factory returns a URL which will hold an APK when ready
- Marketplace retrieves APK from URL, caches locally, and gives to end users when requested
Secondary use case (public factory):
- Developer sends URL directly to the APK Factory
- APK Factory returns a reviewer copy of the APK (signed with a debug key) and a production copy of the APK (signed with a real per-app key)
- (this is a secondary function. Some product decisions need to happen around whether this is a requirement)
Changes needed
+------------------+ +---------------------+ +------------------+
| | | | | |
| Marketplace |+----->| APK Factory |+------> | APK Signer |
| (modify) |<-----+| (create) |<------+ | (create) |
| | | | | |
| | | | | |
+------------------+ +---------------------+ +------------------+
Marketplace
- [uiwanted] Adjust developer flow to opt-out of automatic APK generation
- Adjust developer flow to send manifest (or mini-manifest) URL to APK Factory
- Adjust developer tools to add APK download link to versions page of the app page
- [uiwanted] Adjust developer tools to accept a user specified signing key (associated with app)
- Adjust reviewer tools to add APK download link to review detail page
- Adjust mini-manifests to include APK download URL (should automatically make all install buttons work)
APK Factory
- Requires Java Development Kit (JDK) not just an environment (JRE). Requires new hardware?
- API to accept manifest as input
- Differentiate between hosted & packaged apps
- (...) the actual packaging here. Need to duplicate/enhance prototype for functionality and scale
- API to accept signing key if specified
- Ability to sign using an android debug key for all non-reviewed manifests.
- Ability to sign using production app keys if signing for final packages
- API to send and receive keys from the APK Signer
- GC to clean up old packages on disk
APK Signer
- Requires new hardware
- Based heavily on solitude
- A secure storage area for production signing keys
- No outside network traffic (see above diagram)
User Stories
Use case: install
As a user, I can tap install on a marketplace page and an APK is downloaded to my device so that my phone's native package installer can install the app.
We will be modifying the implementation of the mozApps.install() and mozApps.installPackage() APIs on Android. On webapp install, Fennec will download an APK which is generated from the manifest URL which is passed to the apk-builder.
In this manner, we are able to avoid changing any marketplace code, and support third-party webapp marketplaces.[at the very least, with this method we avoid the marketplace trying to detect whether the phone is android or not -BillW]
Implementation of the apk-builder
It is intended that apk-builder use Android's standard toolchain.
The apk-builder accepts a URL to a JSON document, that could be a webapp manifest or mini-manifest. It should detect the difference between a packaged app and hosted app and act accordingly.
The apk-builder constructs an Android project from a template and data in the webapp manifest, including icons, string translations, permissions, and web activities.
Both webapps and Android have very similar concepts. There is some impedance mismatch, though this is not particularly relevant for the scope of this project.
The exception to this is around APK signing. Android apps are expected to be signed by the developer, regardless of whether they seek any special permissions. The same signature is used to sign successive versions of the same APK, otherwise the update will not be allowed to continue.
For packaged webapps, the package.zip and mini-manifest files will be included in the APK.
Once the apk-builder has created an Android project, it can build it with ant, javac and the rest of the Android toolchain.
By design, the Android project's template contains no Java code. All code is contained in a library-project. Both, especially the library project, are being actively developed in parallel to the apk-builder.
It is desirable that the project template and library code be taken live from a code repository so that changes are easy to incorporate in to the workflow.
Use case: update
As a user, I can have my phone check for updates and have my phone download them so that I may keep my webapps safely up-to-date.
The synthesized APK is relatively lightweight. For hosted apps, Gecko can manage updating appcache and resources generated at runtime. Updating anything used to build the APK should trigger a rebuild of the APK itself. In practice, this means the APK should be rebuilt anytime the webapp manifest changes. From a computation point of view, the detection of change in the manifest and the rebuilding of the APK would be sensibly done on the server.
For packaged apps, any change of the app should trigger a rebuild, however, we are working with the assumption that a release of the app is needed, and a change in the mini-manifest will be triggered.
Android apps have a hard requirement that the versionCode, a non user facing integer representation of the version string, be positively increasing.
The same android signing key should be used to sign the updated APK as was used to sign the previous version of the APK.
Use case: developer's own keys
As a developer, I can use my own key to sign my Android APK, so I can use the key elsewhere.
Changing keys for an already published APK is impossible.
For a large proportion of developers, a randomly generated APK signing key will be sufficient.
However, a few will want to be able to use their own. During app submission, if the app is to be on Android, they should have the opportunity to upload their own key.
Use case: pre-submission development
As a developer I would like to be able to test my APK without submitting it to the marketplace.
It is assumed that the Android toolchain is rather cumbersome and not interesting enough for the webapp developer to actually set it up successfully.
The apk-builder should accept a POST of the webapp manifest and resources needed for the webapp, and return an APK. A command line or option in the app manager will be used to send the POST.
The APK should not be publishable on an APK app-store, e.g. signed with a debug key, icon marked with "IN TEST".
This use of the apk-builder should be invoked by the developer from the command line or App Manager.
In coming releases, we would like the developer to be able to construct an APK without calling out to a server, either from the command line or App Manager.
Use case: post-submission review
- As a reviewer I can run the APK on my Android phone.
- As a developer I can run the same APK as the reviewer.
The APK should not be publishable on an APK app-store, e.g. signed with a debug key, icon marked with "IN REVIEW".
Use case: post-review acceptance
As a developer, I can download the APK that will be installed on user's devices, so I can distribute it myself.
Once the webapp is ready for publishing, the developer should receive (as part of an existing email?), a link to the APK.