CloudServices/Contact-Sync

Meeting

We have a regular meeting on Tuesdays at 9:30am Pacific, in Ian Bicking's Vidyo room.

Notes are kept here: https://id.etherpad.mozilla.org/contacts-sync-fruux

Bugs

• Meta bug for this proof-of-concept
• Sync contacts with CardDAV is another effort by Telefonica people

Goal

The goal was to create a prototype which backs FxOS contacts up to a remote CardDAV server as they are added. A full end-to-end test which:

• listen for new contacts and for modications
• converts mozcontacts to vcards
• provisions a new account on the CardDAV server using an FxA assertion
• pushes the vcard to the remote server

As an extension to this, we wanted to let advanced users push to their own CardDAV servers by specifying the url, username and passwords manually.

Here are two screencasts showing the prototype in action:

• Main functionality
• Using a custom CardDAV server

Code

We created a gaia branch. Everything lives under apps/system/services/contacts.

Auth APIs

There are three different APIs involved in the integration between Firefox Accounts (FxA) and fruux's SabreDAV:

1. Logging into the fruux web dashboard using Persona.
2. Provisioning a fruux username and device-specific password from a FxA assertion.
3. Authenticating each CardDAV requests using HTTP basic auth over TLS.

Provisioning of new fruux accounts or device-specific passwords

Here's how the FxOS app will push a contact to fruux:

1. If it doesn't have a fruux username and password stored (i.e. fresh account), it will call the fruux provisioning API. Otherwise jump to Step 7.
2. The fruux server will receive a FxA assertion which contains a UUID and an email address.
3. If there aren't any user accounts associated with that FxA UUID and email, fruux will create one.
4. Once a user account exists, a new device-specific password will be created.
5. The fruux server will return this username & password to the FxOS app.
6. The FxOS app will store these credentials.
7. Once it has credentials, the FxOS app will use HTTP basic auth in a CardDAV request.
8. If that request fails with a 401 error code, the app will clear its credentials and go back to Step 1, unless it has already tried to reset the credentials (in which case, it silently fails).

User identifier

Initially, fruux will use both the UUID and the email address it gets from FxA to identify users.

Once web logins are done with FxA, the email address will no longer need to be associated with the basic auth credentials.

Revocation of fruux credentials

Since the FxOS application will do silent on-demand provisioning of credentials using FxA assertions as described above, fruux can choose to revoke device-specific passwords whenever it wants, according to its own policies.

Eventually FxA will offer an API for receiving notifications when it recommends that usernames & passwords associated with an FxA UUID be revoked (e.g. when a user reports their device as stolen). That API doesn't exist yet though.

Discussion

The prototype was successful and highlights the fact that we could work with a third-party to develop a system that integrates with Firefox Accounts and provides supporting services to FxOS applications.

Here are a few observations and opinions that came out of this exploration.

CardDAV backend

CardDAV is needed for interop with existing systems that people are using now, either inside organizations, as hosted services or self-hosted by enthusiasts. That shouln't limit our choice of what storage format / approach we use internally, but it means that if we use something else, we'll most likely need to write a CardDAV connector of some kind if we want to let people reuse their existing contact stores.

From a speed of implementation point of view, CardDAV is likely the fastest way to build something that we can ship to users.

Using CardDAV may however limit our ability to use client-side encryption and only host encrypted blobs on the server.

Firefox Sync backend

In the long run, it could be. In the short time, the Sync folks don't really want to see more data types in Sync. So an upcoming rewrite of Sync might be a good fit, but we would likely need an interim solution if contact syncing was deemed a priority.

Using the new DataStore Contacts API

There should be several benefits to using the new DataStore Contacts API. The main one that will affect our sync work will be that we will no longer need to have an immortal event listener waiting for contacts changes; rather, we will be able to view a series of versioned transactions to ascertain what has changed since we last synced our contacts. This opens up the possibility of having the contacts sync service residing in an app (separate from the system app), that can be killed and restarted at any time.

Periodic backups v. immediate uploads

Our (unoptimized) approach as it stands could be more resource-intensive than a periodic backup if one is editing a lot of contacts at once. However, because it's pushing contacts immediately, it means that contacts may get to the server faster.

In terms of processing, our approach is probably more resource-intensive since it requires converting contacts on the phone from their internal representation to vcards.

Should we take it to the next logical step (sync) then it would become much more interesting than a backup since it would allow the merging of contacts between multiple devices.

Datatype-specific backup v. full phone backup

If we stick to just doing backups, then our approach is probably not the right one. We may as well back the whole phone up using the same code. If we move onto full syncing of contacts, then our approach seems to work well.

Using Firefox Accounts for auth

We established a way to bridge between Firefox Accounts and another protocol (http basic auth). Together with Fruux, we made it possible to make this bridge invisible to the user, allowing someone signed into Firefox Accounts to provision an identity with a third-party service and interoperate with it.

This requires our third-party partner to do extra development on their end to accept FxA assertions. Until the OAuth-based API is live, there is no fine-grained approach to authenticatiion / authorization.

UX considerations

Towards the middle of Phase 1, we realized that we could get somewhere with almost no UI at all, so we decided to go ahead with that strategy and make everything fully automatic, providing only an ON/OFF switch (default: ON, just for this prototype) in the existing contact settings.

Going forward, our UX folks need to chat with the FxOS UX folks to design a solution that will fit their overall strategy and take into account:

• backup / restore for the phone
• import / export of contacts
• multiple sources of contacts (e.g. Facebook) that we are not allowed to merge or backup

We believe an Android-style multi-account approach is best (and that sounds like what the FxOS comms folks are hoping to get to as well) but more discussion is needed.

Further work

Here are some tasks that could be done if we wanted to explore this approach further and extend our prototype to syncing of contacts:

• end-to-end test of downloading an existing contact onto the phone
  • vcard to mozcontact conversion
  • API calls to query remote CardDAV server and pull contacts down once
• syncing between two FxOS devices and a server
  • propagate newly created contacts
  • propagate contact deletions
  • handle contact updates
    • pull updated contacts from server and update local ones
    • deal with conflicts between a locally updated contact and a remotely updated one
• syncing between FxOS device and a non-FxOS device (e.g. Evolution)
• optimizations
  • keeping track of etags and synctokens to avoid pulling unchanged contacts
  • batching up updates
• security
  • ability to delete your account
  • revocation of basic auth passwords (requested by FxA)