== Terms of Service == By accessing or using the Firefox Sync APIs in connection with the development of your own client software to access the Firefox Sync services (a “Third Party Client”), you acknowledge that you will need to install and use a local version of the Firefox Sync server for multiple account testing and that any use of Mozilla’s hosted Firefox Sync services is subject {{warning|Content has moved to Mozilla’s Firefox Sync Terms of Service at https://services.mozilla.com/tos/. Further, you agree (a) to maintain and link to (including on websites from which your Third Party Client may be downloaded) a separate, conspicuous, and reasonably detailed privacy policy detailing how data collected or transmitted by your Third Party Client is managed and protected; (b) that your Third Party Client will only store data in encrypted form on the Firefox Sync servers operated by Mozilla; (c) that you and your Third Party Client will use the Firefox Sync APIs solely for their intended purpose; (d) that your Third Party Client will not hide or mask its identity as it uses the Services and/or Firefox Sync APIs, including by failing to follow required identification conventions; and (e) that you and your Third Party Client will not use the Firefox Sync APIs for any application or service that replicates or attempts to replicate the Services or Firefox Sync experience unless such use is non-confusing (by non-confusing, we mean that people should always know with whom they are dealing and where the information or software they are downloading came from). You may not imply, either directly or by omission, that your Third Party Client is produced or endorsed by Mozilla. By providing access to the Firefox Sync APIs, Mozilla is not granting you a license to any of our trademarks. == Overview == This page describes how to the client-side Sync API for sync engines (and their helper objects). The focus is on using this API to create a new sync engine to synchronize a new data type. To do this, you'll need to write an engine object that extends the base <tt>SyncEngine</tt> object. You'll also need to extend three helper objects. Here are the base implementations you need to extend, and the files in which they're defined: # <tt>CryptoWrapper</tt>, in <tt>services/sync/modules/record.js</tt># <tt>SyncEngine</tt>, <tt>Store</tt>, <tt>Tracker</tt> in <tt>services/sync/modules/engines.js</tt> It will be very helpful to look at the existing sync engines -- such as the one for bookmarks and the one for history, and their helpers, for guidance. You can find these files at: * <tt>services/sync/modules/engines/bookmarks.js</tt> -- the Record implementations for the various subtypes of bookmarks and the <tt>BookmarkEngine</tt>, <tt>BookmarkStore</tt>, and <tt>BookmarkTracker</tt>.* <tt>services/sync/modules/engines/history.js</tt> -- the <tt>HistoryRec</tt>,<tt>HistoryEngine</tt>, <tt>HistoryStore</tt>, and <tt>HistoryTracker</tt>. == Setting up a JS Module == The code for your custom sync engine should live in a [https://developer.mozilla.org/en/Using_JavaScript_code_modules JS module]. First off we're going to import the files mentioned above at the top the file. We're also going to import ''util.js'' as it contains many useful helpers. <pre> const Cu = Components.utils; Cu.import("resource://services-sync/engines.js"); Cu.import("resource://services-sync/record.js"); Cu.import("resource://services-sync/util.js");</pre> == The Record object == The record object is a wrapper around a single instance of whatever data it is that you are syncing -- a single bookmark, history URL, password or form data entry. For some sync engines, it makes sense to bundle all data into one record. The tabs and preferences engines work that way. There's an object called <tt>CryptoWrapper</tt> defined in <tt>services/sync/modules/record.js</tt>, which handles all the encryption and decryption of your record for you. All you have to do is write an object that extends <tt>CryptoWrapper</tt> and maintains a property called <tt>cleartext</tt>. <tt>cleartext</tt> must be a JSON-able object. Put into it all values that you want to have encrypted, stored on the server, decrypted, and synced up. You may find it useful to write getters and setters for various properties of your record implementation. The skeleton of a sample record implementation: <pre> function FooRecord(collection, id) { CryptoWrapper.call(this, collection, id); } FooRecord.prototype = { __proto__: CryptoWrapper.prototype, _logName: "Record.Foo", ttl: FOO_TTL, // optional get bar() this.cleartext.bar, set bar(value) { this.cleartext.bar = value; }, get baz() this.cleartext.baz, set baz(value) { this.cleartext.baz = value; } };</pre> To save all that typing for declaring the getters and setters, you can also use <tt>Utils.deferGetSet</tt>: <pre> function FooRecord(collection, id) { CryptoWrapper.call(this, collection, id); } FooRecord.prototype = { __proto__: CryptoWrapper.prototype, _logName: "Record.Foo", ttl: FOO_TTL // optional }; Utils.deferGetSet(FooRec, "cleartext", ["bar", "baz"]);</pre> == The Store object == The Store object (which extends <tt>Store</tt>, as defined in <tt>services/sync/modules/engines.js</tt>) has the job of creating and maintaining a set of Record objects from the underlying data. The store must also make updates to the underlying data itself based on incoming Record objects. The majority of the code you write for syncing a new data type will most likely be in the Store implementation. Each Record that the Store keeps track of must be identified by a unique ID (unique among all records in the store) called GUID. Depending of what type of data you're working with, you might already have GUIDs built into your data that you can make use of (note that GUIDs must be allowed to change and should be URL friendly). If not, you may have to invent your own mapping from data objects to GUIDs as long as it's consistent. In this case, it is highly recommended to use the <tt>Utils.makeGUID()</tt> helper to generate new GUIDs: let newGuid = Utils.makeGUID(); Your Store object '''must''' implement the following methods: *<tt>itemExists(id)</tt>*<tt>createRecord(id, collection)</tt>*<tt>changeItemID(oldId, newId)</tt>*<tt>getAllIDs()</tt>*<tt>wipe()</tt>*<tt>create(record)</tt>*<tt>update(record)</tt>*<tt>remove(record)</tt> You may also find it useful to override other methods of the base implementation, for example <tt>applyIncomingBatch</tt> if the underlying storage for your data supports batch operations. === createRecord === The <tt>createRecord( id, collection )</tt> method gets called by the engine to request a new record for an item with a given GUID. It's your Store's responsibility to instantiate a Record object, assign the given GUID, and return it. === itemExists(guid) === Should simply return <tt>true</tt> if an item with the given guid exists in the store, <tt>false</tt> otherwise. === changeItemID(oldId, newId) === Must find the stored item currently associated with the guid <tt>oldId</tt> and change it to be associated with the guid <tt>newId</tt>. === getAllIDs() === Must return an iterable list containing the GUIDs of every item being stored on the local system. This can be a dictionary-type object where the keys are the GUIDs and the values are whatever; or it can simply be a list of GUIDs. This is the method that the engine will call the first time it syncs, in order to get a complete inventory of what data there is that will need to be uploaded to the server. Unless the items you're syncing have their own inherent GUIDs already defined, you'll need to invent GUIDs for all your items at this time. When one of these GUIDs is later passed as an argument to <tt>createRecord()</tt>, you need to return a record based on the matching data. Therefore, it's the responsibility of this method to define the guid -> item mapping, and to store it for later reference by other methods. === wipe() === When this method is called, your Store needs to completely wipe all of its stored data from the local application. That doesn't mean just whatever caches of the data the Store happens to be maintaining; it means the underlying data itself. For instance, <tt>BookmarkStore.wipe()</tt> deletes all bookmarks from the current Firefox profile. How to do this for your particular data type is up to you to figure out. === Create / Update / Remove === The next three methods are called by the sync algorithm when it determines that the state of the local application needs to be changed to keep it up-to-date with the user's remote activities. The sync algorithm will call your Store object with a record to be created, updated, or removed, and it is your Store object's responsibility to apply this change to the local application using whatever methods are neccessary. <tt>create(record)</tt>:Not to be confused with <tt>createRecord()</tt>, this method (which should probably be renamed for clarity soon) tells your Store to create a new item in the local application, based on the data in record. (So for example, <tt>BookmarkStore.create()</tt> adds a new bookmark to the Firefox profile). <tt>update(record)</tt>:The argument is a record which has been remotely modified; your Store should locate the matching local item (presumably using the GUID, which is available in record.id) and update it to the new values provided in <tt>record</tt>. <tt>remove(record)</tt>:The argument is a record which has been remotely deleted; your Store should locate the matching local item and delete it. (Don't rely on the record that is passed in to this method having any of its attributes set correctly except for <tt>.id</tt>. The <tt>.id</tt> should be al you need, anyway.) <tt>applyIncomingBatch</tt>:TODO === Example Store skeleton === <pre> // Maintains the store of all your Foo-type items and their GUIDs. function FooStore(name) { Store.call(this, name); } FooStore.prototype = { __proto__: Store.prototype, itemExists: function(guid) { // Return true if an item with given guid exists in the store. }, createRecord: function(id, collection) { record = new FooRecord(uri); // Look up the data corresponding to this guid, by the mapping // you've defined. // Set the data and the guid on the new record: record.bar = 17; // or whatever record.id = guid; // return the record return record; }, changeItemID: function(oldId, newId) { // Find the item with guid = oldId and change its guid to newId. }, getAllIDs: function() { // Return a list of the GUIDs of all items. Invent GUIDs for any items // that don't have them already, and remember the mapping for later use. }, wipe: function() { /Firefox_Sync/ Delete everything! JavaScript_Client_API}, create: function(record) { // Create a new item based on the values in record }, update: function(record) { // look up the stored record with id = record.id, then set // its values to those of new record }, remove: function(record) { // look up the stored record with id = record.id, then delete it. } };</pre> == Writing a Tracker class == Your tracker class must inherit from <tt>Tracker</tt>, which is defined in <tt>services/sync/modules/trackers.js</tt>. Its purpose in life is to track changes to whatever data type you are syncing. It must maintain a list of GUIDs for objects that have been changed and therefore require syncing. It also has to maintain a "score", which is a number from 0 to 100 which represents "how badly does this data type need to be synced right now". Getting your tracker to track changes in the underlying data is probably easiest to do if you register your tracker as an observer, so that it can receive notifications from one or more Mozilla components. What events you listen for is entirely up to you. You can find out more about registering as an observer here: [link]. === The Score === The '''score''' is stored in <tt>this.score</tt>, a variable defined by the base Tracker class. Set your score by assigning a value to <tt>this.score</tt>. The score number is used by the scheduler to decide when your engine will be synced. Setting it to zero means "I have no need to sync". The higher you set this number, the more urgently the scheduler treats your request. Setting it to 100 means "Sync me immediately". It's up to you to figure out the best way to assign a score based on the current state of whatever data type you're tracking. Your tracker score is automatically reset to zero after each time your engine syncs. === The changed GUID list === The base class maintains a list of GUIDs that need syncing. When your tracker detects that an item has changed, you should add it to this list by calling: this.addChangedID(guid); These GUIDs correspond to the <tt>.id</tt> fields of your Record objects; see the section on the Store class for more about defining and maintaining the mapping between GUIDs and Records. === Example === Here's the skeleton of a sample Tracker class. <pre> function FooTracker(name) { Tracker.call(this, name); // Register yourself as event listener or observer for whatever // you want to track. Note that this may unnecessarily slow down // things for users who don't use Sync. It's a bit smarter to have // yourself notified when to start and stop tracking therefore: Svc.Obs.add("weave:engine:start-tracking", this); Svc.Obs.add("weave:engine:stop-tracking", this); } FooTracker.prototype = { __proto__: Tracker.prototype, _enabled: false, observe: function observe(subject, topic, data) { switch (topic) { case "weave:engine:start-tracking": if (!this._enabled) { // register event handler or observer here ... this._enabled = true; } break; case "weave:engine:stop-tracking": if (this._enabled) { // remove event handler or observer here ... this._enabled = false; } break; }, onEvent: function onEvent() { /* Here is where you'd handle the event. See the documentation for whatever service you are observing to find out what to call this method, what arguments to expect, and how to interpret them. */ var guid = 0; /* Here is where you'd include code to figure out the GUID of the item that has changed... */ this.addChangedID(guid); // Update the score as you see fit: this.score += 10; } };</pre> == Writing an Engine class == Your Engine class serves to tie together your Store, Tracker, and Record classes into a bundle that the Sync service can instantiate and use. You're probably sick of writing subclasses by this point, but don't worry: this one is very easy. I saved it for last because it requires the least code. Your class must derive from the <tt>SyncEngine</tt> class, defined in <tt>services-sync/modules/engines.js</tt>. <tt>SyncEngine</tt> contains a lot of code which handles logic for the core sync algorithm, but your subclass won't need to call any of this directly, unless you are overriding part of the sync algorithm to provide custom sync behavior (an advanced technique outside the scope of this article). A sample Engine class: function FooEngine() { Weave.SyncEngine.call(this, "Foo"); } FooEngine.prototype = { __proto__: Weave.SyncEngine.prototype, _recordObj: FooRecord, _storeObj: FooStore, _trackerObj: FooTracker }; As you can see, there isn't actually any new code here at all; the prototype simply defines some metadata such as the Store and Tracker classes to use, and the human-readable name that will be used in the log files to identify errors and status messages coming from this engine. (Don't forget that you'll need to import Weave. So the top of your file should include:) const Cu = Components.utils; // etc... Cu.import("resource://services-sync/main.js"); == Installing your classes into Sync == You can register your engine in an "weave:service:ready" observer using the <tt>Weave.Engines.register(FooEngine)</tt> function. However, "weave:service:ready" is issued after the "weave:engine:start-tracking" observer notification. If your tracker observes "weave:engine:start-tracking", it's best to have the Sync service register your engine. Stick your engine class on to the Weave object: <pre>Weave.FooEngine = FooEngine;</pre> and add "Foo" to the <tt>services.sync.registerEngines</tt> preference (it's a comma separated list of engine names). You can also set up a component that registers a weave:service:ready observer and at that time, imports main.js as well as your engine. Also, for me, doing Weave.Engines.register(FooEngine) worked better then doing Weave.FooEngine = FooEngine. Reason for this is, Weave will iterate through the registerEngines preference and try to instantiate each engine it has there before you the line mentioned above ever has a chance to execute.This means the constructor to your engine is not ready when Weave tries to instantiate it. == Testing and Debugging your Engine == Set observers in the Tracker to tell when your datatype changes, after which the score needs to be set. You can also add api's to the engine, get a handle of the engine by going Weave.Engines.get("Foo") and call it directly.
