Firefox/Go Faster/Client Implementation Plan: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Clarify when we overwrite existing versions)
m (Typo)
 
(12 intermediate revisions by one other user not shown)
Line 1: Line 1:
This is a straw-man client implementation plan that covers the main [[Firefox/Go Faster/Client Requirements|Client Requirements]]. The rough set of bugs needed to implement this are called out.
This is a straw-man client implementation plan that covers the main [[Firefox/Go Faster/Client Requirements|Client Requirements]]. The rough set of bugs needed to implement this are called out.
=Known-good sets=
To reduce the number of possible configurations thus simplifying QA and dependency issues the client implementation uses the concept of a "known-good set" of system add-ons. This is a list of add-on IDs and versions to run for a particular application.
Every application package and update will be distributed with a known-good set of system add-ons for that application. Periodically the application will attempt to discover a new known-good set to be installed and used.
The application will only run with a known-good set of system add-ons enabled or it will not enable any system add-ons and run standalone though since the application ships with a known-good set we should generally never be in this state.
The only exception is in the case where a developer has installed a version of a system add-on in the profile for testing, in this case it essentially overrides the known-good version for that add-on.


=Running system add-ons=
=Running system add-ons=
Line 7: Line 17:
They will be installed in a directory used as a regular add-on install location with a lower priority than the profile install location. This allows developers to test out new versions of system add-ons by installing them using the regular add-ons manager.
They will be installed in a directory used as a regular add-on install location with a lower priority than the profile install location. This allows developers to test out new versions of system add-ons by installing them using the regular add-ons manager.


The specific path to the install location will be '''<profile>/features/<app path hash>'''. "app path hash" is a simple hash of the client's install directory. This means that if a user is using two different versions of the client application (say a developer version and a release version) with the same profile they will use two different directories for the system add-ons allowing them to be updated independently. When a single build updates to a new version it automatically retains the most recently downloaded system add-ons from the previous version. The majority of users use a single build and so will have just one directory in use.
Unlike existing add-on install locations system add-ons will be installed in a filename based on both ID and version, <id>_<version>.xpi. This allows multiple versions of the same add-on to be installed at the same time so if a user is running a developer edition and release version they can rely on different known-good sets.
 
The hash can be short to avoid causing path length problems and quick to generate. Collisions aren't a big problem, the client will recover gracefully automatically.


* '''Bug: Create a features install location with a lower priority to the profile location'''
* '''Bug: Cache the known-good set of add-ons on a per-install basis'''
* '''Bug: Create a features install location with a lower priority to the profile location which lists the current known-good set of add-ons'''
* '''Bug: Hide system add-ons in the features location from the add-ons manager UI'''
* '''Bug: Hide system add-ons in the features location from the add-ons manager UI'''


If a user uninstalls an old build they will be left with a now useless set of system add-ons. Periodically we should check '''<profile>/features''' for unused directories. We can write the current time to the current location and for any locations that are older than a certain date delete them.
If a user uninstalls an old build they will be left with a now useless set of system add-ons. Periodically we should check '''<profile>/features''' for unused add-ons. We can update a manifest with the last-used time of the current add-ons and remove ones that haven't been used in a while.


* '''Bug: Shortly after startup write the current time to the system add-on install location and purge older locations'''
* '''Bug: Shortly after startup write the current time to the system add-on install location and purge older locations'''
Line 24: Line 33:
* '''Bug: Disable automated updates for the system add-ons location'''
* '''Bug: Disable automated updates for the system add-ons location'''


During runtime the client will periodically check for system add-ons to use with the current application version. It will do so by making a HTTPS request to a specific URL which will return a root manifest listing the system add-ons for that application, the versions they are compatible with and the URL of a normal [https://developer.mozilla.org/en-US/docs/Extension_Versioning,_Update_and_Compatibility update manifest] for each add-on.
During runtime the client will periodically make a HTTPS request to a specific URL including data on the application channel, version and OS The manifest returned lists the specific system add-ons that should be used with that application.


Here is a proposed root manifest format. It allows for multiple client applications to use the same manifest file but assumes that most system add-ons will only work with a single client application.
An example response:


  {
  <updates><addons>
  "<app-ID>": {
  <addon id="loop@mozilla.org" URL="https://addons.mozilla.org/addons/loop.xpi" version="12"/>
    "<addon-ID>": {
  <addon id="pdfjs@mozilla.org" URL="https://cdn.addons.mozilla.org/addons/pdfjs.xpi" version="1.4"/>
      "minVersion": "42.0",
  </addons></updates>
      "maxVersion": "45.*",
      "updateManifest": "https://mozilla.com/foo/bar/update.rdf"
    }
  }
  }


The minVersion and maxVersion here are hints for the client application to help reduce the bandwidth when we know that a given system add-on isn't compatible with the application. minVersion defaults to "0" if missing, maxVersion defaults to "*". Most likely a new system add-on will be added with a defined minVersion for the first supported client version with maxVersion omitted. Later when the add-on reaches end of life the maxVersion property would be added. This keeps the need to modify the root manifest to a minimum, only when a new system add-on is created and when one is retired, more complex compatibility information is supported in the update manifest.
If the server returns with an error or malformed XML or the response doesn't include an <addons> section then the result is ignored and the current set of system add-ons is kept. If the <addons> tag is present but empty then it means no system add-ons should be enabled.


If the application version falls in the range then the the update manifest is downloaded and used to find a compatible XPI to install. If the application version doesn't fall in the range then it assumes that there is no compatible XPI available.
Any add-on versions that are not already available locally are downloaded.


* '''Bug: Periodically download the system add-ons root manifest'''
* '''Bug: Download and install new system add-ons'''


If there is no compatible XPI available for an installed system add-on then it will be uninstalled.
Once all the add-on versions listed in the manifest are available locally and are verified to be usable with the current application they are copied to a new directory inside <profile>/features and the known-good set is updated with the new set. Any enabled system add-ons not in the set are disabled, any system add-ons in the set not enabled are enabled.


* '''Bug: Uninstall system add-ons when no compatible version exists for the current application version'''
* '''Bug: Replace the current set of system add-ons with the new known-good set'''
 
If there is a compatible XPI available then it will be downloaded and installed unless that specific add-on version is already installed. Note this differs from normal add-on update where we only download newer versions, here we can also downgrade the system add-on.
 
* '''Bug: Download and install compatible system add-ons when no matching version is installed'''
 
The actual host of the update manifest and the XPIs that it points to doesn't matter to the client code. It could be AMO or something new. Changing the host in the future only requires changing the root manifest, no client changes would be necessary.


=Securing system add-ons=
=Securing system add-ons=
Line 59: Line 57:


* '''Bug: Add-ons manager signature checks should enforce special rules for add-ons installed in the system add-ons location'''
* '''Bug: Add-ons manager signature checks should enforce special rules for add-ons installed in the system add-ons location'''
The root manifest and update manifests should be protected by https at a minimum (no certificate overrides allowed). As they can only point to signed XPIs there may be no need to do any additional security here but we should consult with the security team.


=Bootstrapping=
=Bootstrapping=


In order to give a good experience to users using a new profile or updating after not having used the client in a while the application should ship with all system add-ons currently available for that build. This means including the add-ons in installers, packages, dmgs and the update mars for the application.
In order to give a good experience to users using a new profile or updating after not having used the client in a while the application should ship with a known-good set of system add-ons for that build. This means including the add-ons in installers, packages, dmgs and the update mars for the application.


* '''Bug: Include system add-ons in application packages and updates'''
* '''Bug: Include system add-ons in application packages and updates'''


On the startup of a new version of an application the client will find the add-ons included with the application and compare them to those in the system add-ons install location. If the add-on doesn't exist in the install location or the version there is older than the shipped version or incompatible with the current application then the shipped version will be copied to the install location.
On the startup of a new version of an application the client will replace its known-good set with the new set included in the application.


* '''Bug: On startup with a new application version install any new or updated system add-ons into the system add-on install location"
* '''Bug: On startup with a new application version switch to the shipped known-good set"


=Add-on Dependencies=
=Add-on Dependencies=


Declaring dependencies between add-ons isn't supported. Each add-on is downloaded individually and if there are network issues it will be possible for one system add-on to update when another doesn't. For now if add-ons have dependencies they should either be shipped in Firefox so we can use version checks to enforce them or the add-on should programmatically check that the necessary dependencies are installed.
Add-on dependencies should be managed at the server side. The client will ensure that only a known-good set of system add-ons runs.

Latest revision as of 18:58, 5 October 2015

This is a straw-man client implementation plan that covers the main Client Requirements. The rough set of bugs needed to implement this are called out.

Known-good sets

To reduce the number of possible configurations thus simplifying QA and dependency issues the client implementation uses the concept of a "known-good set" of system add-ons. This is a list of add-on IDs and versions to run for a particular application.

Every application package and update will be distributed with a known-good set of system add-ons for that application. Periodically the application will attempt to discover a new known-good set to be installed and used.

The application will only run with a known-good set of system add-ons enabled or it will not enable any system add-ons and run standalone though since the application ships with a known-good set we should generally never be in this state.

The only exception is in the case where a developer has installed a version of a system add-on in the profile for testing, in this case it essentially overrides the known-good version for that add-on.

Running system add-ons

System add-ons will be installed in the user's profile. This assures that client code can install and update add-ons without needing special privileges and that there can only be one client application accessing the add-ons at a time.

They will be installed in a directory used as a regular add-on install location with a lower priority than the profile install location. This allows developers to test out new versions of system add-ons by installing them using the regular add-ons manager.

Unlike existing add-on install locations system add-ons will be installed in a filename based on both ID and version, <id>_<version>.xpi. This allows multiple versions of the same add-on to be installed at the same time so if a user is running a developer edition and release version they can rely on different known-good sets.

  • Bug: Cache the known-good set of add-ons on a per-install basis
  • Bug: Create a features install location with a lower priority to the profile location which lists the current known-good set of add-ons
  • Bug: Hide system add-ons in the features location from the add-ons manager UI

If a user uninstalls an old build they will be left with a now useless set of system add-ons. Periodically we should check <profile>/features for unused add-ons. We can update a manifest with the last-used time of the current add-ons and remove ones that haven't been used in a while.

  • Bug: Shortly after startup write the current time to the system add-on install location and purge older locations

Discovering system add-ons

By default the add-ons manager will attempt to update system add-ons automatically, this must be disabled so a slightly different mechanism for finding updates can be used.

  • Bug: Disable automated updates for the system add-ons location

During runtime the client will periodically make a HTTPS request to a specific URL including data on the application channel, version and OS The manifest returned lists the specific system add-ons that should be used with that application.

An example response: 

<updates><addons>
  <addon id="loop@mozilla.org" URL="https://addons.mozilla.org/addons/loop.xpi" version="12"/>
  <addon id="pdfjs@mozilla.org" URL="https://cdn.addons.mozilla.org/addons/pdfjs.xpi" version="1.4"/>
</addons></updates>

If the server returns with an error or malformed XML or the response doesn't include an <addons> section then the result is ignored and the current set of system add-ons is kept. If the <addons> tag is present but empty then it means no system add-ons should be enabled.

Any add-on versions that are not already available locally are downloaded.

  • Bug: Download and install new system add-ons

Once all the add-on versions listed in the manifest are available locally and are verified to be usable with the current application they are copied to a new directory inside <profile>/features and the known-good set is updated with the new set. Any enabled system add-ons not in the set are disabled, any system add-ons in the set not enabled are enabled.

  • Bug: Replace the current set of system add-ons with the new known-good set

Securing system add-ons

System add-ons will need to be signed in a way that differentiates them from regular add-ons. This means either using a custom signing certificate or using the AMO signing service. AMO currently has two signing servers set up two sign preliminarily reviewed add-ons and fully reviewed add-ons. Clients can distinguish between the two based on a special string added to the Organizational Unit (OU) of the signing certificate for each add-on. Adding a third signing server that adds a new string to the OU is straightforward but AMO would need some updates to know to pass certain add-ons to that server.

  • Bug: Add-ons manager signature checks should enforce special rules for add-ons installed in the system add-ons location

Bootstrapping

In order to give a good experience to users using a new profile or updating after not having used the client in a while the application should ship with a known-good set of system add-ons for that build. This means including the add-ons in installers, packages, dmgs and the update mars for the application.

  • Bug: Include system add-ons in application packages and updates

On the startup of a new version of an application the client will replace its known-good set with the new set included in the application.

  • Bug: On startup with a new application version switch to the shipped known-good set"

Add-on Dependencies

Add-on dependencies should be managed at the server side. The client will ensure that only a known-good set of system add-ons runs.

Retrieved from "https://wiki.mozilla.org/index.php?title=Firefox/Go_Faster/Client_Implementation_Plan&oldid=1099257"