Firefox/Kinto: Difference between revisions

Code updates and overall improvement
(→‎Run the tests: list some more tests)
(Code updates and overall improvement)
Line 1: Line 1:
= Kinto Integration in Firefox =
= Kinto Integration in Firefox =
[[http://www.kinto-storage.org Kinto]] is a simple JSON storage service.


= Key features =
= Key features =


* Diff-based data synchronization
* Diff-based data synchronization
* Offline persistence
* Data integrity/signing
* Data integrity/signing
* Offline persistence
* Peer review to publish data changes
* Admin panel UI
* Admin panel UI
* Firefox Accounts Integration
* ''Built-in telemetry (coming soon)''


= Use Cases =
= Use Cases =


* Certificates blocklist (OneCRL) ''(contact: Mark Goodwin — mgoodwin)''
{|class="wikitable"
* Addons/Plugins/Gfx [[Blocklisting]] ''(contact: Mathieu Leplatre — leplatrem)''
! Component
* <code>storage.sync</code> API ([[WebExtensions]]) ''(contact: Tarek Ziadé tarek)''
! Description
* Fennec assets catalog ''(contact: Sebastian Kaspari — sebastian)''
! Contact
|-
|OneCRL
|Certificates revocation blocklist
|Mark Goodwin — mgoodwin
|-
|HPKP
|HTTP Public Key Pinning
|Mark Goodwin — mgoodwin
|-
|Addons/Plugins/Gfx [[Blocklisting]]
|Black list for unsecure/instable/malicious addons, plugins or graphical drivers
|Mathieu Leplatre — leplatrem
|-
|<code>storage.sync</code>  
|[[WebExtensions]]) API for user data storage
|Ethan Glasser Camp glasserc
|-
|Fennec assets catalog
|Catalog of assets to be downloaded after installation
|Sebastian Kaspari — sebastian
|-
|Fennec experiments
|Data for A/B testing in Fennec
|Sebastian Kaspari — sebastian
|-
|iOS experiments
|Data for A/B testing in Firefox iOS
|Stephan Leroux — sleroux
|}
 
Ideas/WIP
 
* Password manager recipes ''(contact: Matthew Noorenberghe — MattN)''
* Password manager recipes ''(contact: Matthew Noorenberghe — MattN)''


Line 20: Line 57:
= Usage =
= Usage =


Leveraging the [https://github.com/Kinto/kinto-http.js Kinto HTTP client] in Gecko looks like this:
Two client libraries are embedded in Firefox:
 
* [https://github.com/Kinto/kinto-http.js Kinto HTTP client]: for direct interactions with the Kinto HTTP API
* [https://github.com/Kinto/kinto.js Kinto offline client]: for offline persistence in internal SQLite
 
== HTTP client ==


<pre>
<pre>
const { KintoHttpClient } = Cu.import("resource://services-common/kinto-http-client.js");
const { KintoHttpClient } = Cu.import("resource://services-common/kinto-http-client.js", {});


const client = new KintoHttpClient("https://kinto.dev.mozaws.net/v1");
const remote = "https://kinto.dev.mozaws.net/v1";
client.bucket("a-bucket").collection("a-collection")
const headers = {Authorization: "Basic " + btoa("user:pass")};
  .listRecords()
 
  .then(result => ...);
const client = new KintoHttpClient(remote, {headers});
const records = yield client.bucket("a-bucket")
                            .collection("a-collection")
                            .listRecords();
</pre>
</pre>


As for the [https://github.com/Kinto/kinto.js Kinto offline-first client], it is like:
== Offline client ==


<pre>
<pre>
const { loadKinto } = Cu.import("resource://services-common/kinto-offline-client.js");
const { Kinto } = Cu.import("resource://services-common/kinto-offline-client.js", {});
const { FirefoxAdapter } = Cu.import("resource://services-common/kinto-storage-adapter.js", {});


const KintoOfflineClient = loadKinto();
const remote = "https://kinto.dev.mozaws.net/v1";
const headers = {Authorization: "Basic " + btoa("user:pass")};


const client = new KintoOfflineClient({
const kinto = new Kinto({adapter: FirefoxAdapter});
  adapter: Kinto.adapters.FirefoxAdapter,
const collection = kinto.collection("a-collection");
  remote: "https://kinto.dev.mozaws.net/v1",
  bucket: "a-bucket"
});


const collection = db.collection("a-collection");
// Fetch/Publish changes.
const {ok} = yield collection.sync({
  bucket: "a-bucket",
  remote,
  headers});


try {
// Read local collection of records.
  yield collection.db.open();
const records = yield collection.list();
  // Fetch changes from server.
  yield collection.sync();
  // Read local collection of records.
  const records = yield collection.list();
} finally {
  yield collection.db.close();
}
</pre>
</pre>
== Add a new data set ==
The only persons allowed to create buckets/collections in stage/production is the OPs team, so you have to create a Bugzilla ticket and specify everything you need to be created.
* The bucket name, and its permissions (who can read/write/create collections)
* A collection name, and its permissions (who can read/write/create records)
For use-cases where a few administrators manage some data downloaded by all Firefox clients, you have to enable peer reviewing. You will have 3 buckets: a ''staging'' where editors make changes, a ''preview'' bucket where changes are published when a review is requested, and the final bucket where changes are published when the review is approved. You must specify:
* Who can edit (members of the editors group)?
* Who can review/approve data (members of the reviewers group)?
* Who can read data (members of the observers group, eg. QA)? ''Optional''
* A new key for the digital signatures (Autograph) will have to be configured on the server side for your use-case.
For use-cases where a lot of users write data (like storage.sync), the [https://kinto.readthedocs.io/en/latest/api/1.x/quotas.html quota plugin] may have to be configured.




= Specifications =
= Specifications =


Currently, the instance of Kinto used by Firefox clients is hosted at https://firefox.settings.services.mozilla.com/v1/
The distribution package of Kinto that is deployed is [https://github.com/mozilla-services/kinto-dist/ kinto-dist].
There are several instances, depending on the use-case — mainly if clients have write access or not.


== Blocklists ==  
== Blocklists ==  


The goal is to replace the current system based on a single XML file downloaded everyday by several Kinto collections.
Currently, the instance of Kinto used by Blocklist clients is hosted at https://firefox.settings.services.mozilla.com/v1/
 
The goal is to replace the current system based on a single XML file downloaded everyday by several Kinto collections.


* The bucket is '''blocklists'''
* The bucket is '''blocklists'''
Line 99: Line 160:


= Contribute =
= Contribute =
== Upgrade client libraries ==
Two client libraries are embedded in Firefox:
* <code>Kinto/kinto-http.js</code>: for direct interactions with the Kinto HTTP API
* <code>Kinto/kinto.js</code>: for offline persistence in internal SQLite


=== Generate bundles ===
=== Generate bundles ===
Line 114: Line 168:
* [https://github.com/Kinto/kinto.js kinto.js] is the offline-first client for Kinto.
* [https://github.com/Kinto/kinto.js kinto.js] is the offline-first client for Kinto.


With the help of Babel and browsersify, a bundle is generated for Firefox with the minimum transpilation possible (eg. CommonJS require, ES7 decorators).
With the help of Babel and browserify, a bundle is generated for Firefox with the minimum transpilation possible (eg. CommonJS require, ES7 decorators).


==== kinto.js ====
==== kinto.js ====
24

edits