Personal tools

Services/Sync/Features/Addon Sync

From MozillaWiki

< Services | Sync(Difference between revisions)
Jump to: navigation, search
(more design details)
Line 65: Line 65:
 
|Feature dependencies=; Sync GUID added to XPI Provider : Sync requires non-deterministic IDs to be affiliated with synchronized records. XPI add-ons currently don't have a suitable ID so once will need to be created.
 
|Feature dependencies=; Sync GUID added to XPI Provider : Sync requires non-deterministic IDs to be affiliated with synchronized records. XPI add-ons currently don't have a suitable ID so once will need to be created.
 
|Feature requirements=*When a user installs an add-on, the add-on will be installed on all the user's devices connected with Sync.
 
|Feature requirements=*When a user installs an add-on, the add-on will be installed on all the user's devices connected with Sync.
*When a user uninstalls an add-on, the add-on will be uninstalled on all the user's devices connected with Sync
+
*When a user uninstalls an add-on, the add-on will be uninstalled on all the user's devices connected with Sync
*When a user disables an add-on, the add-on will be disabled on all the user's devices connected with Sync
+
*When a user disables an add-on, the add-on will be disabled on all the user's devices connected with Sync
 
*When a user enables an add-on, the add-on will be enabled on all the user's devices connected with Sync
 
*When a user enables an add-on, the add-on will be enabled on all the user's devices connected with Sync
 
*Not sure about the update version requirement.  
 
*Not sure about the update version requirement.  
Line 77: Line 77:
 
|Feature functional spec=A syncGUID field has been added to the extensions SQLite database ({{bug|682027}}). All extensions now have a randomly-generated ID suitable for syncing.
 
|Feature functional spec=A syncGUID field has been added to the extensions SQLite database ({{bug|682027}}). All extensions now have a randomly-generated ID suitable for syncing.
  
To track changes, Sync has written a component outside of the add-ons engine called the "Add-ons Reconciler." This provides what the Add-on Manager does not: a persisted state of add-on changes. It installs itself as a listener for Add-on Manager events. When things changed, it records that change and updates it's own minimal copy of the global state. These changes are persisted to disk in a JSON file in the profile directory. Periodically (at the top of a sync) the reconciler examines the state of the world and ensures it agrees with the Add-on Manager.
+
To track changes, Sync has written a component outside of the add-ons engine called the "Add-ons Reconciler." This provides what the Add-on Manager does not: a persisted state of add-on changes. It installs itself as a listener for Add-on Manager events. When things changed, it records that change and updates its own minimal copy of the global state. These changes are persisted to disk in a JSON file in the profile directory, alongside other persistent Sync metadata. Periodically (at the top of a sync) the reconciler examines the state of the world and ensures that it agrees with the Add-on Manager.
  
 
The add-ons engine subscribes to changes in the Add-on Reconciler to trigger instant sync.
 
The add-ons engine subscribes to changes in the Add-on Reconciler to trigger instant sync.
Line 83: Line 83:
 
At the end of every sync, the sync engine requests the reconciler to delete data older than 7 days. This is for privacy reasons as well as to ensure the change log doesn't grow without bound. Even if the change log would grow unbounded, it is doubtful it would ever get so large to become unwieldy because the number of add-on change events (install, uninstall, enable, disable) isn't very high (maybe a few a week).
 
At the end of every sync, the sync engine requests the reconciler to delete data older than 7 days. This is for privacy reasons as well as to ensure the change log doesn't grow without bound. Even if the change log would grow unbounded, it is doubtful it would ever get so large to become unwieldy because the number of add-on change events (install, uninstall, enable, disable) isn't very high (maybe a few a week).
 
|Feature ux design=UX has stated that there will be no new UX for this feature. Individual add-ons may trigger UX when they are modified, but there is nothing we can do about that.
 
|Feature ux design=UX has stated that there will be no new UX for this feature. Individual add-ons may trigger UX when they are modified, but there is nothing we can do about that.
|Feature security review=The feature follows the same security model as other sync engines: add-on records are encrypted using the Sync Key and the IDs for each add-on are randomly generated.
+
|Feature security review=The feature follows the same security model as other sync engines: add-on records are encrypted using Sync's bulk keys (themselves encrypted with the user's Sync Key), and the IDs for each add-on are randomly generated.
  
 
The entire feature is implemented in JavaScript and runs in the chrome process - the same process as Sync.
 
The entire feature is implemented in JavaScript and runs in the chrome process - the same process as Sync.
Line 109: Line 109:
 
When add-ons are synced across devices, the receiving device installs the add-on from addons.mozilla.org, not from Sync. For every add-on to be installed on receiving devices, AMO will see traffic. It will first see a search for this add-on by its ID. If that returns something, the XPI will be downloaded as part of the install process.
 
When add-ons are synced across devices, the receiving device installs the add-on from addons.mozilla.org, not from Sync. For every add-on to be installed on receiving devices, AMO will see traffic. It will first see a search for this add-on by its ID. If that returns something, the XPI will be downloaded as part of the install process.
  
When this feature launches and add-ons are synchronized across devices, AMO could see increased activity. We should keep an eye on this for launch day.
+
When this feature launches and add-ons are synchronized across devices, AMO could see a small amount of increased activity, proportional to the number of multi-device Sync users with the feature enabled and the frequency of add-on changes on those devices. We should keep an eye on this for launch day.
 
}}
 
}}
 
{{FeatureInfo
 
{{FeatureInfo

Revision as of 13:23, 3 December 2011

Please use "Edit with form" above to edit this page.

Status

Addon Sync
Stage Development
Status In progress
Release target TBD
Health OK
Status note Development is in progress.

Team

Product manager Jennifer Arguello
Directly Responsible Individual Jennifer Arguello
Lead engineer Gregory Szorc
Security lead Yvan Boily
Privacy lead `
Localization lead `
Accessibility lead `
QA lead Tracy Walker
UX lead `
Product marketing lead Jaclyn Fu
Operations lead `
Additional members Ibai Garcia (SUMO)

Open issues/risks

Issue/Risk Status

Do we need some prompts or feedback when add-ons are synced, at all?

Probably. Many add-ons still aren't restartless, so installing, uninstalling, enabling, or disabling will require restart. When Sync is driving the add-on install, the Add-on Manager won't display any visual prompts. So, it will be Sync's responsibility to trigger a UI element to prompt restart, if desired. Sync has full control over that element. We could display the one used by the Add-on Manager or make our own.
Answer from Faaborg: Is we do not need any extra UI. If the user happens to go to the Add-ons tab, the updated add-ons should show a restart is necessary.

Are there any problems with large sets of add-on state being updated at once? Say the user has not used a device for a while and they use it again and a whole bunch of add-ons need to be updated.

Stage 1: Definition

1. Feature overview

Sync is a service to keep the Firefox experience consistent across multiple devices by ensuring user data is synced across various devices. This feature will enable add-ons to be synced across devices. Add-ons are small pieces of software that add new features or functionality to an installation of Firefox. Add-ons can augment Firefox with new features, foreign language dictionaries, or change its visual appearance. Through add-ons, you can customize Firefox to meet your needs and tastes.

Sync will ensure that add-ons are installed, uninstalled, disabled,and enabled as they are changed on each client for the same application. This means that desktop to desktop add-ons will sync, and mobile to mobile add-ons will sync.

For the initial release, only XPI extensions and themes from addons.mozilla.org will be supported. In a future release, we can expand scope to add-ons not from addons.mozilla.org.

2. Users & use cases

Desktop browser to browser add-on install
  • A user has sync set up his work and home desktops
  • He goes to AMO and installs an add-on on his work computer
  • When he goes home and starts his Firefox browser the new add-on will be usable and shows up in the add-on tab
Desktop browser to browser add-on uninstall
  • A user has sync set up his work and home desktops
  • At work, he goes to the add-on tab in the browser and removes the add-on
  • When he goes home and starts his Firefox browser this add-on will remove itself and not show up in the add-on tab
Desktop browser to browser add-on disable
  • A user has sync set up his work and home desktops
  • At work, he goes to the add-on tab in the browser and disables an add-on
  • When he goes home and starts his Firefox browser this add-on will disable itself and not show up in the add-on tab
Desktop browser to browser add-on enable
  • A user has sync set up his work and home desktops
  • At work, he goes to the add-on tab in the browser and enables a disabled add-on
  • When he goes home and starts his Firefox browser this add-on will enable itself and not show up in the add-on tab

3. Dependencies

Sync GUID added to XPI Provider 
Sync requires non-deterministic IDs to be affiliated with synchronized records. XPI add-ons currently don't have a suitable ID so once will need to be created.

4. Requirements

  • When a user installs an add-on, the add-on will be installed on all the user's devices connected with Sync.
  • When a user uninstalls an add-on, the add-on will be uninstalled on all the user's devices connected with Sync
  • When a user disables an add-on, the add-on will be disabled on all the user's devices connected with Sync
  • When a user enables an add-on, the add-on will be enabled on all the user's devices connected with Sync
  • Not sure about the update version requirement.


Non-impact on add-on manager metrics 
The presence of Sync should not skew the metrics in the add-on manager and addons.mozilla.org. Currently, some APIs on the AddonManager upload metrics.

Non-goals

Syncing add-on information across different Application IDs
Sync will not synchronize non-XPI add-ons such as plugins, lightweight themes, and search engines. Sync will also not synchronize add-ons installed outside of the currently running profile.

Stage 2: Design

5. Functional specification

A syncGUID field has been added to the extensions SQLite database (bug 682027). All extensions now have a randomly-generated ID suitable for syncing.

To track changes, Sync has written a component outside of the add-ons engine called the "Add-ons Reconciler." This provides what the Add-on Manager does not: a persisted state of add-on changes. It installs itself as a listener for Add-on Manager events. When things changed, it records that change and updates its own minimal copy of the global state. These changes are persisted to disk in a JSON file in the profile directory, alongside other persistent Sync metadata. Periodically (at the top of a sync) the reconciler examines the state of the world and ensures that it agrees with the Add-on Manager.

The add-ons engine subscribes to changes in the Add-on Reconciler to trigger instant sync.

At the end of every sync, the sync engine requests the reconciler to delete data older than 7 days. This is for privacy reasons as well as to ensure the change log doesn't grow without bound. Even if the change log would grow unbounded, it is doubtful it would ever get so large to become unwieldy because the number of add-on change events (install, uninstall, enable, disable) isn't very high (maybe a few a week).

6. User experience design

UX has stated that there will be no new UX for this feature. Individual add-ons may trigger UX when they are modified, but there is nothing we can do about that.

Stage 3: Planning

7. Implementation plan

`

8. Reviews

Security review

The feature follows the same security model as other sync engines: add-on records are encrypted using Sync's bulk keys (themselves encrypted with the user's Sync Key), and the IDs for each add-on are randomly generated.

The entire feature is implemented in JavaScript and runs in the chrome process - the same process as Sync.

For the initial feature drop, synchronized add-ons will be limited to:

  • XPI extensions or themes
  • from the same application ID as other Sync profiles
  • installed in the profile directory
  • installed explicitly by the user (those put in the profile directory by nefarious applications will be ignored - !addon.foreignInstall)
  • installed from addons.mozilla.org
    • In the implementation, the hostname is defined by a preference. the default value is *addons.mozilla.org*

Eventually, we'll want to expand to all XPI extensions and will want to handle add-ons from non-AMO URIs. But, these will be covered by separate feature(s) and security review(s).

UX has deemed it best for the feature to have no explicit UX in the browser, as Sync should be transparent. However, modification of add-ons could result in changes in browser behavior. Toolbars and buttons could appear/disappear "randomly" (corresponding with when Sync runs - which is transparent to the user). Tabs could be opened for add-ons which open tabs on install events. While these aren't security vulnerabilities, they could be perceived as such. "Why is my browser changing - was it hacked?" UX rationalizes it by saying that any add-on behavior must have been triggered by user behavior somewhere. This is true. However, a user might be surprised to see add-ons magically being changed upon (transparent) upgrade to Firefox X. Upon add-on modification, about:addons should reflect the change immediately.

If a user's Sync server credentials (but not the Sync Key) are compromised, the server records would reveal the last modified time of individual add-on records. Without access to a Firefox profile, it would be unknown which add-on each record corresponded to. This data leakage is minor and is on par with other leakages currently in the Sync service.

Sync transmits each add-ons ID between devices. The add-on ID is looked up on addons.mozilla.org. Assuming it is found, Firefox will download the extensions or theme from addons.mozilla.org and install it. The installation process uses existing AddonManager APIs. This will result in more requests to addons.mozilla.org. However, such requests shouldn't be traceable back to Sync. We (theoretically) already trust the privacy model of addons.mozilla.org, so even if data is leaked, it should be no big deal.

This is the first component of Sync which will indirectly communicate with a non-Sync server (addons.mozilla.org). Sync will be talking with it via the AddonRepository JS APIs. Sync assumes those APIs are doing the proper things to secure against MITM attacks, etc.

Privacy review

`

Localization review

`

Accessibility

`

Quality Assurance review

`

Operations review

This feature will utilize a new server-side collection, *addons*. The server people might want to add it to the known/common collections set.

When add-ons are synced across devices, the receiving device installs the add-on from addons.mozilla.org, not from Sync. For every add-on to be installed on receiving devices, AMO will see traffic. It will first see a search for this add-on by its ID. If that returns something, the XPI will be downloaded as part of the install process.

When this feature launches and add-ons are synchronized across devices, AMO could see a small amount of increased activity, proportional to the number of multi-device Sync users with the feature enabled and the frequency of add-on changes on those devices. We should keep an eye on this for launch day.

Stage 4: Development

9. Implementation

`

Stage 5: Release

10. Landing criteria

`


Feature details

Priority P2
Rank 999
Theme / Goal Experience
Roadmap Sync
Secondary roadmap `
Feature list Services
Project `
Engineering team Sync

Team status notes

  status notes
Products ` ; Update syncing cut : Syncing on add-on update is cut because an updating mechanism is already part of Firefox. I will follow up with FF folks to see if Sync does bring value. We can address it later.
Only sync between the same application 
Add-ons will initially only sync between applications of the same application ID. This is because most add-ons aren't compatible across different applications.
Engineering ` `
Security sec-review-complete Notes
Privacy ` `
Localization ` `
Accessibility ` `
Quality assurance ` `
User experience ` `
Product marketing ` `
Operations ` `


Discussion notes from 2011-08-16:

  • add 'guid' column to 'addon' table in extensions.sqlite
  • add GUID support to AddonsManager (have it automatically generated, 12 chars base64url)
  • findings addons shouldn't count as daily update pings, or if they do, is it a problem?
  • generate one record per app per addon, silently drop records from other apps and unknown addons (unknown to AMO)
  • add option to locally disable enabled state sync (the "web developer case") in about:config