Gaia/Contacts/Data Refactor: Difference between revisions

Jump to navigation Jump to search

Gaia/Contacts/Data Refactor (view source)

Revision as of 15:50, 5 November 2014

1,875 bytes added ,  5 November 2014
no edit summary
No edit summary
No edit summary
Line 1: Line 1:
== Current situation ==
= Current situation =
The current Contacts app has been growing with time around the [https://wiki.mozilla.org/WebAPI/ContactsAPI MozContacts API], started like a simple application but has been adding more and more requirements.
The current Contacts app has been growing with time around the [https://wiki.mozilla.org/WebAPI/ContactsAPI MozContacts API], started like a simple application but has been adding more and more requirements.


Line 8: Line 8:
Also we added a great feature like contacts merging, everything working under the same MozContacts API.
Also we added a great feature like contacts merging, everything working under the same MozContacts API.


=== Problems ===
== Motivation ==
=== Contacts list performance ===
Due to the design of Contacts app, we should be able to jump to letters and scroll super fast with the letter shortcuts. It wasn't right to add application specific requirements to MozContacts API so we have been doing black magic to keep the performance at 60fps when loading huge amount of contacts (> 2000).


Some problems that we are facing now are the following:
=== Search restrictions ===
Again, adding more and more indexes just cause specific app requirements wasn't a good idea so we had to implement our search based on DOM, which is extremely slow and affects as well to the contacts list rendering time.


* Contacts list performance:
=== Impossible to unmerge contacts ===
** Due to the design of Contacts app, we should be able to jump to letters and scroll super fast with the letter shortcuts. It wasn't right to add application specific requirements to MozContacts API so we have been doing black magic to keep the performance at 60fps when loading huge amount of contacts (> 2000).
During the process of merging contacts we end up loosing the information of the original contacts as we only have a MozContacts API to store this information. So there is no way to unmerge contacts.


* Search restrictions:
=== Dependency with Facebook integration ===
** Again, adding more and more indexes just cause specific app requirements wasn't a good idea so we had to implement our search based on DOM, which is extremely slow and affects as well to the contacts list rendering time.
We want other apps different than the Contacts app to be able to read the contacts data imported from external sources like FB, Hotmail, Gmail, etc. However, we have legal restrictions that avoid us to share the data imported from FB with an app that is not owned by Gaia. This made us take this restricted data out of the MozContacts API because this API is accessible by privileged apps. Initially we had to create an independent IndexedDB with FB information that used to live in the Communications app, but once we were able to use the DataStore API we moved this data to a shared DataStore. So now consumers of contacts information that are interested on FB data need to query the MozContacts API and the FB datastore.


* Impossible to unmerge contacts:
=== Hard to integrate with 3rd party contacts providers ===
** During the process of merging contacts we end up loosing the information of the original contacts as we only have a MozContacts API to store this information. So there is no way to unmerge contacts.
We added extra support for Gmail and Hotmail contacts but this is embedded in the Contacts application itself, which is not scalable. We want to make any 3rd party app capable of providing contacts to have a seamless integration with the Contacts APP.


* Dependency with Facebook integration:
= Architecture =
** We want other apps different than the Contacts app to be able to read the contacts data imported from external sources like FB, Hotmail, Gmail, etc. However, we have legal restrictions that avoid us to share the data imported from FB with an app that is not owned by Gaia. This made us take this restricted data out of the MozContacts API because this API is accessible by privileged apps. Initially we had to create an independent IndexedDB with FB information that used to live in the Communications app, but once we were able to use the DataStore API we moved this data to a shared DataStore. So now consumers of contacts information that are interested on FB data need to query the MozContacts API and the FB datastore.
== Current context ==
=== MozContacts API ===
Right now is frozen, we know that and will be for good a nice way of storing contacts locally, but we won't be able to add extra requirements.
=== DataStore API ===
The [https://wiki.mozilla.org/WebAPI/DataStore Datastore API] is a new API designed to make apps share information or making them collaborate to add data to a common store. It allows synchronization and being able to notify about changes. It's a pretty simple key/value store, where both key and value can be any kind of object, and it doesn't implement any kind of search, just direct access by key.
=== Contacts data consumers (in Gaia) ===
* '''Contacts app''': Obviously. This is the central hub of contacts information in Gaia. We want the user to have access to the whole list of contacts whichever its source is. We want local contacts and contacts imported from external sources like FB, Gmail, Hotmail, etc. The user won't be able to modify the contact information from an external source. All changes done to contacts will be kept locally and won't affect its remote replication. Each contact provider is responsible for its own data. In this case, the Contacts app is only provider and so responsible of the MozContacts API data, although this data can also be modified by other apps with enough permissions.
* '''Messaging app''': Whenever the user sends or receives an SMS or MMS, we need to get the information associated to the contact(s) subject of this message. We are also allowing users to do quick searches by contact name and phone number when composing a new message. We are currently querying the MozContacts API and the FB datastore.
* '''Dialer app''': Same situation as above, but we are only doing searches by phone number. We are currently also querying both stores, MozContacts and FB.
* '''Emergency calls''': We do a basic search by contact ID on the MozContacts store to show the ICE contacts bar if appropriate. No FB search so far, but we want users to be able to set FB contacts as ICE contacts, so we will need it in the future. No need for quick incremental searches by phone number or any other field.
* '''Email app''': The email app queries the MozContacts API by ID and email. No need for incremental searches so far, but we might need it at some point. No queries to the FB datastore.
* '''Search app''': Currently disabled. So far there is a search by name implemented. It should be able to search any contact (include FB data) by any field at some point (?).


* Hard to integrate with 3rd party contacts providers:
== Proposal ==
** We added extra support for Gmail and Hotmail contacts but this is embedded in the Contacts application itself, which is not scalable. We want to make any 3rd party app capable of providing contacts to have a seamless integration with the Contacts APP.
=== Data flow ===
 
==== High level overview ====
== Architecture ==
=== Current context ===
 
* mozContacts API
** Right now is frozen, we know that and will be for good a nice way of storing contacts locally, but we won't be able to add extra requirements.
 
* DataStore API
** The [https://wiki.mozilla.org/WebAPI/DataStore Datastore API] is a new API designed to make apps share information or making them collaborate to add data to a common store. It allows synchronization and being able to notify about changes. It's a pretty simple key/value store, where both key and value can be any kind of object, and it doesn't implement any kind of search, just direct access by key.
 
=== Proposal ===
==== Data flow ====
===== High level overview =====
[[File:IMG 20140619 162203.jpg|800px|x600px]]
[[File:IMG 20140619 162203.jpg|800px|x600px]]


[[File:Contactsrefactor.png|800px|x600px]]
[[File:Contactsrefactor.png|800px|x600px]]


==== Principles ====
=== Principles ===
* There are many apps capable of provide contacts, each app should:
* There are many apps capable of providing contacts, each app should:
** Be the owner of their own information (CRUD).
** Be the owner of their own information (CRUD).
** Provide a datastore name ''contacts'' with the information it shares.
** Provide a datastore name ''contacts'' with the information it shares.
Line 66: Line 68:
** Use directly indexedDB to provide a better peforming contacts list (jump to letter, better cursor walking ...).
** Use directly indexedDB to provide a better peforming contacts list (jump to letter, better cursor walking ...).


=== Requirements ===
== Requirements ==
In order to do this we sill have some requirements that needs to be implemented in the platform to allow us do the following:
In order to do this we sill have some requirements that needs to be implemented in the platform to allow us do the following:
* Expose mozContacts API content as a datastore:  
* Expose mozContacts API content as a datastore:  
Line 72: Line 74:
* Better notification when data changes in Datastores:
* Better notification when data changes in Datastores:
** In order to make the flow happen when changes are performed in the different datastores, we will need to be able to receive system messages if a DS changes in our app: {{bug|1014023}}  [Datastore] Notify apps of changes in datastore without being opened
** In order to make the flow happen when changes are performed in the different datastores, we will need to be able to receive system messages if a DS changes in our app: {{bug|1014023}}  [Datastore] Notify apps of changes in datastore without being opened
=== Questions ===
== FAQ ==
* What happen if a new app is installed and offers contacts?
* What happen if a new app is installed and offers contacts?
We have filled {{bug|968278}} against platform to get notified of datastores of one kind (contacts in this case) being installed or removed to act accordinly.
We have filled {{bug|968278}} against platform to get notified of datastores of one kind (contacts in this case) being installed or removed to act accordinly.
Line 87: Line 89:
* Will this GCD have access to the contacts of an specific app?
* Will this GCD have access to the contacts of an specific app?
We want the user to decide that, which external datastores will be able to being used by the GCDS.
We want the user to decide that, which external datastores will be able to being used by the GCDS.
=== Bugs associated ===
 
== Known issues ==
 
= Implementation =
== Bugs associated ==
[https://bugzilla.mozilla.org/show_bug.cgi?id=968098 Bugzilla link]
[https://bugzilla.mozilla.org/show_bug.cgi?id=968098 Bugzilla link]


Line 103: Line 109:
** {{Bug|1031315}} - [Contacts][Data Refactor]Check performance of building contact details from different DS
** {{Bug|1031315}} - [Contacts][Data Refactor]Check performance of building contact details from different DS
*** Here we tried to check how much it cost to build a contact detail from different sources each time, compared with storing the result in indexeddb and retrieving it.
*** Here we tried to check how much it cost to build a contact detail from different sources each time, compared with storing the result in indexeddb and retrieving it.
*** With 500 contacts, obviously is much faster the indexdDB option, but we wanted to see how much is the difference between bot solutions having 30-40ms to get the data from indexedDb and ~150ms while composing from different sources.
*** With 500 contacts, obviously is much faster the indexedDB option, but we wanted to see how much is the difference between bot solutions having 30-40ms to get the data from indexedDb and ~150ms while composing from different sources.
*** To be sure that we don't consume much space on disk, we decided to cache the detail of those contacts that we visit.
*** To be sure that we don't consume much space on disk, we decided to cache the detail of those contacts that we visit.


Ferjm
Confirmed users
483

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1030805"

Navigation menu