User:Clouserw/APKFactory: Difference between revisions

m
 
(35 intermediate revisions by 5 users not shown)
Line 2: Line 2:
= C&C APK Factory =
= C&C APK Factory =


We're building an APK Factory!  In short, this is a black box which will accept a manifest URL as input and return a .apk file.  This project includes some UI changes on the Marketplace but the vast majority of the project is backend and APIs.
We're building an APK Factory!  In short, this is a black box which will accept a manifest URL as input and return a .apk file.  This project includes some UI changes on the Marketplace but the vast majority of the project is backend APIs.


The primary use case is:
The primary use case is:
Line 8: Line 8:
* Fennec retrieves the manifest linked to from the ''Install'' button and sends it to the APK Factory
* Fennec retrieves the manifest linked to from the ''Install'' button and sends it to the APK Factory
* The APK Factory processes it and turns it into an APK (~seconds)
* The APK Factory processes it and turns it into an APK (~seconds)
* The APK Factory returns an APK in response to the manifest
* The APK Factory returns the APK in response
* Fennec installs the APK as a normal android app
* Fennec installs the APK as a normal android app
* What are we doing with existing developers?  Are we opting them in to doing this conversion?
** We are opting them in.  We'll notify them first.  -- David Almstrom
* Question from David Bialer - How do we handle people who have apps already installed?
** It's possible we have something in Fx29 for this already which will do a one time check. There is a bug already filed.  -- David Almstrom
** It's unclear how many there actually are; we may be willing to have people with existing implementations reinstall their apps.  We have no ADIs to check what percentage of installs are in active use.  -- myk
** This is not a blocker.
** Bill Walker / Vishy is investigating our options: http://bugzil.la/938667 (wrong bug?)


= User Stories =
= User Stories =
Line 18: Line 27:
'''As a user, I can have my phone check for updates and have my phone download them so that I may keep my webapps safely up-to-date.'''
'''As a user, I can have my phone check for updates and have my phone download them so that I may keep my webapps safely up-to-date.'''


''Implementation Note:'' Fennec will ping the APK factory (or, perhaps a different domain?) with a JSON blob to see if there are updates.  '''(TODO: Are push notifications an option instead of polling?)'''
''Implementation Note:'' Fennec will poll the APK factory (or, perhaps a different domain?) with a JSON blob to see if there are updates.  Push notifications are not available across all our platforms so we're back to polling here.
 
''dvd: would push be better as the APK Factory would then need to push out updates notifications to all installs (not via marketplace)? ''


=== Use case: update ===
=== Use case: update ===
'''As a user, I can trust that my phone will automatically keep my webapps up to date.'''
'''As a user, I can trust that my phone will automatically keep my webapps up to date.'''


=== Use case: developer's own keys ===
=== <span style="font-size:1.2em; color: #f33">Not v1</span> Use case: developer's own keys ===
'''As a developer, I can use my own key to sign my Android APK, so I can use the key elsewhere.'''
'''As a developer, I can use my own key to sign my Android APK, so I can use the key elsewhere.'''


''Implementation Notes:''   
''Implementation Notes:''   
Line 32: Line 39:
* For a large proportion of developers, a randomly generated APK signing key will be sufficient.
* For a large proportion of developers, a randomly generated APK signing key will be sufficient.
* We'll need this early enough in the submission process that the first copy of their app will have the right key
* We'll need this early enough in the submission process that the first copy of their app will have the right key
* [''uiwanted'']: We need to make sure developers with keys actually see this as otherwise their userbase is forked.


''See my note below. what about the developer actually has keys but is not aware of that the app is being pushed as an APK''
=== <span style="font-size:1.2em; color: #f33">Not v1</span> Use case: developer opt-out ===
'''As a Developer, I can chose not to have my app auto-generated as an Android APK''': This story is obsolete since anyone could have their own marketplace and hit the APK builder and generate a package.


=== Use case: developer opt-out ===
''Implementation Note:''
'''As a Developer, I can chose not to have my app auto-generated as an Android APK'''
* [''uiwanted''] We need to make sure the developer realizes what is happening so their user base isn't forked.  Original concern:
 
** ''dvd: Issue: what if the developer doesn't know that his app is going to be auto-generated and installed as an APK on Android. If the OWA has not been submitted through the marketplace, the developer is not aware of this. The developer may already have the same app developed as an Android native app or an Android Webapp. If Fennec displays any webapp with an install button as the webapp thinks it will be deployed within the browser, there is no way for the developer to opt out. I am not arguing against the openness as an argument but rather for the developer to be in control.''
''dvd: Issue: what if the developer doesn't know that his app is going to be auto-generated and installed as an APK on Android. If the OWA has not been submitted through the marketplace, the developer is not aware of this. The developer may already have the same app developed as an Android native app or an Android Webapp. If Fennec displays any webapp with an install button as the webapp thinks it will be deployed within the browser, there is no way for the developer to opt out. I am not arguing against the openness as an argument but rather for the developer to be in control.''


=== Use case: pre-submission development ===
=== Use case: pre-submission development ===
'''As a developer I would like to be able to test my APK without submitting it to the marketplace.'''
'''As a developer I would like to be able to test my APK without submitting it to the marketplace.'''


'''TODO: Is this use case out of date?  If you want to test your APK without submitting to the marketplace, just do it.  That's what the install button will do, right?'''
IDEAL: This should be possible without requiring a network on each edit-compile-run cycle.
It is assumed that the Android toolchain is rather cumbersome and not interesting enough for the webapp developer to actually set it
up  successfully.


The APK should not be publishable on an APK app-store, e.g. signed with a [https://developer.android.com/tools/publishing/app-signing.html#debugmode debug key], icon marked with "IN TEST".
''Implementation Notes:''
 
* NO use of APK factory service. Two implementation paths:
This use of the apk-factory should be invoked by the developer from the command line or App Manager.
** BEST: I register my app with the App Manager and connect my device. The manager sees the device and offers to install the app on it. I install the app on it and can launch it and use it. The manager sees that the app is running and offers to enable debugging. I enable debugging and can use the Developer Tools to debug the app.
 
** ACCEPTABLE: I install the Android SDK and the APK Factory CLI. I use the Factory CLI to build an APK for the app and the Android SDK to install the APK on my device. I can then launch and use the app. And the app is configured to enable Firefox remote debugging via a known host and port, so I click Web Developer > Connect… in Firefox and connect to the app at that host/port. I can then use the Developer Tools to debug the app.
In coming releases, we would like the developer to be able to construct an APK without calling out to a server, either from the
*** This is currently the solution we're aiming for, but Austin would like to have the Factory CLI be a wrapper which would communicate with the APK Factory online.  Myk sounds concerned.
command line or App Manager.
**** '''We're going to move forward with both a separate CLI Factory tool and a CLI Factory wrapper and see which is better. -- Austin'''


=== Use case: post-submission review ===
=== Use case: post-submission review ===
Line 59: Line 65:
* '''As a developer I can run the same APK as the reviewer.'''
* '''As a developer I can run the same APK as the reviewer.'''


'''TODO: the third story here says "same APK" - does that mean an APK with a debug cert, or the actual same one.  I prefer the former.'''


''Implementation Notes:'' The APK should not be publishable on an APK app-store, e.g. signed with a [https://developer.android.com/tools/publishing/app-signing.html#debugmode debug key], icon marked with "IN REVIEW".
''Implementation Notes:'' The APK should not be publishable on an APK app-store, e.g. signed with a [https://developer.android.com/tools/publishing/app-signing.html#debugmode debug key]
* The reviewer (or developer) will set a path on their phone to sign apps with the debug cert server, similar to how reviewers today install a reviewer cert


=== Use case: post-review acceptance ===
=== Use case: post-review acceptance ===
'''As a developer, I can download the APK that will be installed on user's devices, so I can distribute it myself.'''
'''As a developer, I can download the APK that will be installed on user's devices, so I can distribute it myself.'''


Once the webapp is ready for publishing, the developer should receive (as part of an existing email?), a link to the APK.
''Implementation Notes:''
'''TODO: Why wouldn't we just put a download link in the developer control panel in the Marketplace?  Or is this for 3rd party install buttons too (in which case, how do we get their email address?)'''
* This is a download link in the Developer control panel
 


= Implementation =
= Implementation =
   +------------+    +-------------+    +------------+
   +------------+    +-------------+    +------------+                        +-------------+
   | User Agent +--->| APK Factory +--->| APK Signer |
   | User Agent +--->| APK Factory +--->| APK Signer |                        |    Debug    |
   |  (Fennec)  |<---+            |<---+            |
   |  (Fennec)  |<---+            |<---+            |                        |    APK    |
   |            |    |            |    |            |
   |            |    |            |    |            |                        |  Factory  |
   +------------+    +-------+-----+    +------------+
   +------------+    +-------+-----+    +------------+                        +-------+-----+
                           ^ |
                           ^ |                                                        ^ |
                           | |
                           | |                                                        | |
   +-------------+        | |
   +-------------+        | |                  +----------------------------+        | |
   | Marketplace +---------+ |
   | Marketplace +---------+ |                  | Developers' and Reviewers' +---------+ |
   |            |<----------+
   |            |<----------+                  |    User Agents (Fennec)    |<----------+
   |            |
   |            |                              |                            |
   +-------------+
   +-------------+                              +----------------------------+


=== User Agent / Fennec Modifications ===
=== User Agent / Fennec Modifications ===
* mozApps.install() and mozApps.installPackage() to send the manifest to the APK Factory
* mozApps.install() and mozApps.installPackage() to send the manifest to the APK Factory
* Manual updating UI and [https://github.com/mozilla/apk-factory-service/issues/4 JSON request] ('''TODO: needs detail / spec''')
* Manual updating UI and [https://github.com/mozilla/apk-factory-service/issues/4 JSON request]  
* Periodic updating check
* Periodic updating check


=== Marketplace Modifications ===
=== <span style="font-size:1.2em; color: #f33">Not v1</span> Marketplace Modifications ===
None of these are v1 and, once we actually get it implemented, may not be something we need to do at all.
* [''uiwanted''] Adjust developer flow to opt-out of automatic APK generation
* [''uiwanted''] Adjust developer flow to opt-out of automatic APK generation
* Adjust developer tools to add APK download link to versions page of the app page
* Adjust developer tools to add APK download link to versions page of the app page
* [''uiwanted''] Adjust developer tools to accept a user specified signing key (associated with app)
* [''uiwanted''] Adjust developer tools to accept a user specified signing key (associated with app)
* Adjust reviewer tools to ask for Reviewer APKs ('''TODO: How do we specify the reviewer debug key?''')
* Adjust reviewer tools to ask for Reviewer APKs (make sure the reviewer is using the debug server somehow)
 


=== APK Factory Modifications (this is a new service) ===
=== APK Factory Modifications (this is a new service) ===
Line 109: Line 114:
* API to send and receive keys from the APK Signer
* API to send and receive keys from the APK Signer
* GC to clean up old packages on disk
* GC to clean up old packages on disk
* API to accept a JSON list of manifests and return which have updates ('''TODO: needs spec''')
* API to accept a JSON list of manifests and return which have updates
** App freshness is a function of:
** App freshness is a function of:
*** the manifest
*** the manifest
Line 117: Line 122:
* Have the ability to regenerate all apps (including bumping versionCode) (useful for fixing a security hole in apk-factory-library)
* Have the ability to regenerate all apps (including bumping versionCode) (useful for fixing a security hole in apk-factory-library)


'''TODO: An assumption is that developers will only specify their own key if they are uploading via the Marketplace.  Perhaps that isn't true?  ozten brings up a good point that we'd need auth for this somewhere, marketplace makes sense...'''
* An assumption is that developers will only specify their own key if they are uploading via the Marketplace.  Perhaps that isn't true?  ozten brings up a good point that we'd need auth for this somewhere, marketplace makes sense...
** dvd: wouldn't the APK Factory need to store manifest URL, version code, keys (or link to key). Would also need to check if the manifest fits the app, no? Like what we do in the marketplace if the app has significantly changed from the manifest, then it will not run.
** Not a v1 concern
 
'''(todo:dbialer)''' What kind of scalability are we building this for?  thousands per day?  millions per day?


=== APK Signer Modifications (this is a new service) ===
=== APK Signer Modifications (this is a new service) ===
Confirmed users, Bureaucrats and Sysops emeriti
1,737

edits