Personal tools

Services/Sync/Features/Addon Sync

From MozillaWiki

< Services | Sync(Difference between revisions)
Jump to: navigation, search
(fill in definition)
Line 5: Line 5:
 
|Feature version=TBD
 
|Feature version=TBD
 
|Feature health=Blocked
 
|Feature health=Blocked
|Feature status note=Product Management needs to flesh out the feature page so that development can be unblocked.
+
|Feature status note=Product Management has updated the page. Team please review and set to unblocked if sufficient.  
 
}}
 
}}
 
{{FeatureTeam
 
{{FeatureTeam
Line 19: Line 19:
 
}}
 
}}
 
{{FeaturePageBody
 
{{FeaturePageBody
|Feature overview=Add-ons are synchronized between Sync clients. Sync will ensure that add-ons are installed, uninstalled, updated, and enabled as they are changed on each client.
+
|Feature open issues and risks=* Is this only for desktop to desktop add-on syncing? Like AdBlock Plus. If this works across devices great! If not the priority is first to get desktop to desktop working and then worry about mobile. I need to understand the work involved.
|Feature users and use cases=; Install propagation : User installs an add-on on one browser. When a sync occurs, this add-on record is uploaded to the Sync server. When another browser in the Sync profile connects, the add-on is installed automatically.
+
* Why does Sync need to be used for updating Add-ons? Would the user not be prompted to update the add-on through the browser?
 +
* Do we need some prompts or feedback when add-ons are synced, at all?
 +
* Are there any problems with large sets of add-on state being updated at once? Say the user has not used a device for a while and they use it again and a whole bunch of add-ons need to be updated.
 +
|Feature overview=Sync is a service to keep the Firefox experience consistent across multiple devices by ensuring user data is synced across various devices. This feature will enable add-ons to be synced across devices. Add-ons are small pieces of software that add new features or functionality to an installation of Firefox. Add-ons can augment Firefox with new features, foreign language dictionaries, or change its visual appearance. Through add-ons, you can customize Firefox to meet your needs and tastes.
  
; Uninstall propagation : This is the same as install propagation except for uninstalls. An uninstall on one client will trigger an uninstall on another.
+
Sync will ensure that add-ons are installed, uninstalled, updated, and enabled as they are changed on each client.
 +
|Feature users and use cases=Use cases:
 +
; Desktop browser to browser add-on install
 +
** A user has sync set up his work and home desktops
 +
** He goes to AMO and installs an add-on on his work computer
 +
** When he goes home and starts his Firefox browser the new add-on will be usable and shows up in the add-on tab
  
; User enabled state propagation : User opens add-on manager and disables an add-on. Sync will propagate this change to other clients. Ditto for enabling. Sync should not propagate the enabled state if it is due to application compatibility (e.g. version upgrades causing incompatibilities). Only the user's explicit enabling and disabling of an add-on should be propagated.
+
; Desktop browser to browser add-on uninstall
 +
** A user has sync set up his work and home desktops
 +
** At work, he goes to the add-on tab in the browser and removes the add-on
 +
** When he goes home and starts his Firefox browser this add-on will remove itself and not show up in the add-on tab
  
 +
; Desktop browser to browser add-on disable
 +
** A user has sync set up his work and home desktops
 +
** At work, he goes to the add-on tab in the browser and disables an add-on
 +
** When he goes home and starts his Firefox browser this add-on will disable itself and not show up in the add-on tab
 +
 +
; Desktop browser to browser add-on enable
 +
** A user has sync set up his work and home desktops
 +
** At work, he goes to the add-on tab in the browser and enables a disabled add-on
 +
** When he goes home and starts his Firefox browser this add-on will enable itself and not show up in the add-on tab
 +
 +
;Not sure about the update case
 +
 +
;LEFT THESE USE CASES HERE TO MAKE SURE I COVERED EVERYTHING
 +
 +
; Install propagation : User installs an add-on on one browser. When a sync occurs, this add-on record is uploaded to the Sync server. When another browser in the Sync profile connects, the add-on is installed automatically.
 +
 +
; Uninstall propagation : This is the same as install propagation except for uninstalls. An uninstall on one client will trigger an uninstall on another.
 +
 +
; User enabled state propagation : User opens add-on manager and disables an add-on. Sync will propagate this change to other clients. Ditto for enabling.
 
; Update propagation : User updates an add-on on one machine. Sync notices the update and schedules other clients to look for an update.
 
; Update propagation : User updates an add-on on one machine. Sync notices the update and schedules other clients to look for an update.
 
|Feature dependencies=; Sync GUID added to XPI Provider : Sync requires non-deterministic IDs to be affiliated with synchronized records. XPI add-ons currently don't have a suitable ID so once will need to be created.
 
|Feature dependencies=; Sync GUID added to XPI Provider : Sync requires non-deterministic IDs to be affiliated with synchronized records. XPI add-ons currently don't have a suitable ID so once will need to be created.
|Feature requirements=; Non-impact on add-on manager metrics : The presence of Sync should not skew the metrics in the add-on manager and addons.mozilla.org. Currently, some APIs on the AddonManager upload metrics.
+
|Feature requirements=*When a user installs an add-on, the add-on will be installed on all the user's devices connected with Sync.
 +
*When a user uninstalls an add-on, the add-on  will be uninstalled on all the user's devices connected with Sync
 +
*When a user disables an add-on, the add-on  will be disabled on all the user's devices connected with Sync
 +
*When a user enables an add-on, the add-on will be enabled on all the user's devices connected with Sync
 +
*Not sure about the update version requirement.
 +
 
 +
 
 +
; Non-impact on add-on manager metrics : The presence of Sync should not skew the metrics in the add-on manager and addons.mozilla.org. Currently, some APIs on the AddonManager upload metrics.
 
|Feature non-goals=Sync will not synchronize non-XPI add-ons such as plugins, lightweight themes, and search engines. Sync will also not synchronize add-ons installed outside of the currently running profile.
 
|Feature non-goals=Sync will not synchronize non-XPI add-ons such as plugins, lightweight themes, and search engines. Sync will also not synchronize add-ons installed outside of the currently running profile.
 
|Feature functional spec=The Addon Manager maintainers would like to see Sync support all add-on providers so as to not introduce 1st and 2nd class providers.
 
|Feature functional spec=The Addon Manager maintainers would like to see Sync support all add-on providers so as to not introduce 1st and 2nd class providers.

Revision as of 17:15, 1 September 2011

Please use "Edit with form" above to edit this page.

Status

Addon Sync
Stage Definition
Status In progress
Release target TBD
Health Blocked
Status note Product Management has updated the page. Team please review and set to unblocked if sufficient.

Team

Product manager Jennifer Arguello
Directly Responsible Individual Jennifer Arguello
Lead engineer Gregory Szorc
Security lead Yvan Boily
Privacy lead `
Localization lead Axel Hecht
Accessibility lead `
QA lead Tracy Walker
UX lead Alex Faaborg
Product marketing lead Jaclyn Fu
Operations lead `
Additional members Ibai Garcia (SUMO)

Open issues/risks

  • Is this only for desktop to desktop add-on syncing? Like AdBlock Plus. If this works across devices great! If not the priority is first to get desktop to desktop working and then worry about mobile. I need to understand the work involved.
  • Why does Sync need to be used for updating Add-ons? Would the user not be prompted to update the add-on through the browser?
  • Do we need some prompts or feedback when add-ons are synced, at all?
  • Are there any problems with large sets of add-on state being updated at once? Say the user has not used a device for a while and they use it again and a whole bunch of add-ons need to be updated.

Stage 1: Definition

1. Feature overview

Sync is a service to keep the Firefox experience consistent across multiple devices by ensuring user data is synced across various devices. This feature will enable add-ons to be synced across devices. Add-ons are small pieces of software that add new features or functionality to an installation of Firefox. Add-ons can augment Firefox with new features, foreign language dictionaries, or change its visual appearance. Through add-ons, you can customize Firefox to meet your needs and tastes.

Sync will ensure that add-ons are installed, uninstalled, updated, and enabled as they are changed on each client.

2. Users & use cases

Use cases:

Desktop browser to browser add-on install
    • A user has sync set up his work and home desktops
    • He goes to AMO and installs an add-on on his work computer
    • When he goes home and starts his Firefox browser the new add-on will be usable and shows up in the add-on tab
Desktop browser to browser add-on uninstall
    • A user has sync set up his work and home desktops
    • At work, he goes to the add-on tab in the browser and removes the add-on
    • When he goes home and starts his Firefox browser this add-on will remove itself and not show up in the add-on tab
Desktop browser to browser add-on disable
    • A user has sync set up his work and home desktops
    • At work, he goes to the add-on tab in the browser and disables an add-on
    • When he goes home and starts his Firefox browser this add-on will disable itself and not show up in the add-on tab
Desktop browser to browser add-on enable
    • A user has sync set up his work and home desktops
    • At work, he goes to the add-on tab in the browser and enables a disabled add-on
    • When he goes home and starts his Firefox browser this add-on will enable itself and not show up in the add-on tab
Not sure about the update case
LEFT THESE USE CASES HERE TO MAKE SURE I COVERED EVERYTHING
Install propagation 
User installs an add-on on one browser. When a sync occurs, this add-on record is uploaded to the Sync server. When another browser in the Sync profile connects, the add-on is installed automatically.
Uninstall propagation 
This is the same as install propagation except for uninstalls. An uninstall on one client will trigger an uninstall on another.
User enabled state propagation 
User opens add-on manager and disables an add-on. Sync will propagate this change to other clients. Ditto for enabling.
Update propagation 
User updates an add-on on one machine. Sync notices the update and schedules other clients to look for an update.

3. Dependencies

Sync GUID added to XPI Provider 
Sync requires non-deterministic IDs to be affiliated with synchronized records. XPI add-ons currently don't have a suitable ID so once will need to be created.

4. Requirements

  • When a user installs an add-on, the add-on will be installed on all the user's devices connected with Sync.
  • When a user uninstalls an add-on, the add-on will be uninstalled on all the user's devices connected with Sync
  • When a user disables an add-on, the add-on will be disabled on all the user's devices connected with Sync
  • When a user enables an add-on, the add-on will be enabled on all the user's devices connected with Sync
  • Not sure about the update version requirement.


Non-impact on add-on manager metrics 
The presence of Sync should not skew the metrics in the add-on manager and addons.mozilla.org. Currently, some APIs on the AddonManager upload metrics.

Non-goals

Sync will not synchronize non-XPI add-ons such as plugins, lightweight themes, and search engines. Sync will also not synchronize add-ons installed outside of the currently running profile.

Stage 2: Design

5. Functional specification

The Addon Manager maintainers would like to see Sync support all add-on providers so as to not introduce 1st and 2nd class providers.

The solution to this problem will consist of the following:

  1. Optional properties on add-on objects that describe the add-on for purposes of use in Sync
  2. Functions on AddonManager that take above properties and perform actions

For add-on objects, providers wishing to opt in to Sync will expose the following read-only, optional properties:

syncData 
a string that describes data necessary to synchronize the add-on between clients. The format of the string is opaque to Sync code. From the perspective of add-on providers, it should be a JSON object with a *type* field that is used to route it to the proper provider.
syncGUID 
a GUID used to identify this add-on across Sync client instances. The GUID should be generated using Utils.makeGUID() from the Sync code. It is basically a Base64-encoded representation of 9 random bytes.

(These properties will need to be documented at https://developer.mozilla.org/en/Addons/Add-on_Manager/Addon)

The AddonManager will support the following APIs:

getAddonBySyncGUID(syncGUID, callback) 
Obtain an add-on from its Sync GUID. Calls the supplied function when that add-on is retrieved. The callback receives null on unknown add-on or the add-on object (generated from the underlying provider) on success. This API technically isn't required, but it makes the Sync code smaller by pushing out logic and possibly caching.
applyRecords(records, callbackObj) 
This applies an array of records that contain add-on metadata and makes the current state of the world agree with that data as much as possible.

The callbackObj is an object containing the following optional keys:

  • onSuccess - Invoked when an individual record is processed successfully. Arguments are the record itself.
  • onFailure - Invoked when an individual record could not be processed successfully. Arguments are the record itself.
  • onFinished - Invoked when all records have finished processing.

Each record is an object containing the following keys:

  • syncGUID (required) - The Sync GUID for this record
  • syncData (optional) - The .syncData field from an add-on
  • deleted (optional) - If evaluates to true, indicates that the record was deleted. Application of this record should involve trying to delete this add-on if present or no-op if not present.

The implementation of applyRecords() will resemble the following pseudocode:

foreach record in records:
  if record.deleted:
    found = this.getAddonBySyncGUID(record.syncGUID)
    if found:
      this.deleteAddon(found)
    # else nothing to do since record not found
     
    callbackObj.onSuccess(record)
    continue

  existing = this.getExistingAddonBySyncData(record.syncData)
  result = null
  if existing:
    # ensure the Sync GUIDs agree. incoming records always win fight
    if existing.syncGUID != record.syncGUID:
      existing.syncGUID = record.syncGUID

    # TODO this probably requires additional API formalization. Each
    # provider likely has its own semantics. But, common operations would
    # likely include updating common add-on fields (like source and install
    # URLs) and upgrading the add-on version.
    result = existing.updateFromSyncData(record.syncData)
  else:
    result = this.installAddonFromSyncData(syncGUID, syncData)

  result ? callbackObj.onSuccess(record) : callbackObj.onFailure(record)

callbackObj.onFinished()

TODO: we should formalize how the Sync engine gets informed of whether a restart is required to finish the record application. Do we catch this in the tracker observers or install observers local to the sync method of the engine?

The add-ons Sync engine will discover the set of add-ons that can be synced via the following procedure:

  1. Query AddonManager.getAllAddons()
  2. Filter the returned list by items that have the *syncData* and *syncGUID* properties and are installed in the profile add-on scope (addon.scope == AddonManager.SCOPE_PROFILE)
    • Open Issue: Should the AddonManager do this filtering? Should the add-on providers only expose syncData on add-ons installed in the proper location?

The engine will process incoming records by converting them to the API accepted by AddonManager.applyRecords() and will invoke that function with them. This is similar to how the history engine works.

The Sync add-on tracker will install add-on listeners in the AddonManager and will listen for the following events:

  • onEnabled
  • onDisabled
  • onInstalled
  • onUninstalled

When these are observed, the tracker will:

  1. Verify the changed add-on is in the set of Sync-able add-ons (using same heuristics as add-on discovery documented above, if needed)
  2. Mark the GUID as changed
  3. This will result in a new record for that GUID being created automagically. The createRecord() procedure will in turn create the necessary record by querying AddonManager which will be queued for upload to the Sync server.

On start-up, the add-on engine will query AddonManager.getStartupChanges() for changes applied on application start-up. These are not observed by Sync because they occur before Sync is registered and running. Even if Sync does catch them in its tracker, it should be safe to mark records as changed. The only side-effect will be a slightly different modified time.

AddonManager.getStartupChanges() will be queried for the following change types:

  • STARTUP_CHANGE_INSTALLED
  • STARTUP_CHANGE_CHANGED
  • STARTUP_CHANGE_UNINSTALLED
  • STARTUP_CHANGE_DISABLED
  • STARTUP_CHANGE_ENABLED

TODO: Sync should catch all startup change types (the constants above) and react to them. We may need an API or some kind of verification/test to ensure newly-introduced startup changes/constants aren't missed by Sync. (This is all because getStartupChanges() requires a type as a parameter.)

There are some interesting implications for the design proposed above. AddonManager and providers - not Sync - would be responsible for the format of the records containing add-on metadata and this means the onus of ensuring forwards and backwards compatibility (since e.g. a version 9 client could receive a Sync record from a version 14 client and vice-versa) reside in the AddonManager and providers. This might be new territory for AddonManager and providers. Unfortunately, the alternative means lots of AddonManager-specific code in Sync (at best) or giving up provider agnostic add-on sync (at worst).

6. User experience design

`

Stage 3: Planning

7. Implementation plan

`

8. Reviews

Security review

`

Privacy review

`

Localization review

`

Accessibility

`

Quality Assurance review

`

Operations review

`

Stage 4: Development

9. Implementation

`

Stage 5: Release

10. Landing criteria

`


Feature details

Priority P2
Rank 999
Theme / Goal Experience
Roadmap Sync
Secondary roadmap `
Feature list Services
Project `
Engineering team Sync

Team status notes

  status notes
Products ` `
Engineering ` `
Security ` `
Privacy ` `
Localization ` `
Accessibility ` `
Quality assurance ` `
User experience ` `
Product marketing ` `
Operations ` `


Discussion notes from 2011-08-16:

  • add 'guid' column to 'addon' table in extensions.sqlite
  • add GUID support to AddonsManager (have it automatically generated, 12 chars base64url)
  • findings addons shouldn't count as daily update pings, or if they do, is it a problem?
  • generate one record per app per addon, silently drop records from other apps and unknown addons (unknown to AMO)
  • add option to locally disable enabled state sync (the "web developer case") in about:config