MozillaWiki

Firefox/Kinto

< Firefox
Revision as of 10:22, 28 April 2016 by Leplatrem (talk | contribs) (Add details about implementation)

Kinto Integration in Firefox

Key features

  • Diff-based data synchronization
  • Data integrity/signing
  • Offline persistence
  • Admin panel UI

Use Cases

  • Certificates blocklist (OneCRL) (contact: mgoodwin)
  • Addons/Plugins/Gfx Blocklisting (contact: leplatrem)
  • storage.sync API (WebExtensions) (contact: tarek)
  • Fennec assets catalog (contact: Sebastian Kaspari)
  • Password manager recipes (contact: MattN)

Feel free to come and discuss on #storage :)

Usage

Leveraging the Kinto HTTP client in Gecko looks like this: 

const { KintoHttpClient } = Cu.import("resource://services-common/kinto-http-client.js");

const client = new KintoHttpClient("https://kinto.dev.mozaws.net/v1");
client.bucket("a-bucket").collection("a-collection")
  .listRecords()
  .then(result => ...);

As for the Kinto offline-first client, it is like: 

const { loadKinto } = Cu.import("resource://services-common/kinto-offline-client.js");

const KintoOfflineClient = loadKinto();

const client = new KintoOfflineClient({
  adapter: Kinto.adapters.FirefoxAdapter,
  remote: "https://kinto.dev.mozaws.net/v1",
  bucket: "a-bucket"
});

const collection = db.collection("a-collection");

try {
  yield collection.db.open();
  // Fetch changes from server.
  yield collection.sync();
  // Read local collection of records.
  const records = yield collection.list();
} finally {
  yield collection.db.close();
}


Specifications

Currently, the instance of Kinto used by Firefox clients is hosted at https://firefox.settings.services.mozilla.com/v1/

Blocklists

The goal is to replace the current system based on a single XML file downloaded everyday by several Kinto collections.

See https://bugzilla.mozilla.org/show_bug.cgi?id=1197707

Fennec assets catalog

The goal is to remove the static assets (fonts, hyphenation dicts, etc.) from the distribution package and download them asynchronously using an online Kinto catalog.

  • The bucket is fennec
  • The collection is catalog

See https://bugzilla.mozilla.org/show_bug.cgi?id=1201059


Contribute

Upgrade client libraries

Two client libraries are embedded in Firefox:

  • Kinto/kinto-client.js: for direct interactions with the Kinto HTTP API
  • Kinto/kinto.js: for offline persistence in internal SQLite

Generate bundles

The Kinto client libraries are developed independently on Github:

  • kinto-client is the HTTP client for the Kinto REST API;
  • kinto.js is the offline-first client for Kinto.

With the help of Babel and browsersify, a bundle is generated for Firefox with the minimum transpilation possible (eg. CommonJS require, ES7 decorators).

kinto.js

From the kinto.js repo, generate the moz-kinto-offline-client.js file: 

$ npm run dist-fx

And overwrite it in the Firefox code base: 

$ cp dist/moz-kinto-offline-client.js ../mozilla-central/services/common/kinto-offline-client.js

kinto-client.js

From the kinto-client.js repo, generate the moz-kinto-http-client.js file: 

$ npm run dist-fx

And overwrite it in the Firefox code base: 

$ cp dist/moz-kinto-http-client.js ../mozilla-central/services/common/kinto-http-client.js

Run the tests

First, follow the instructions to build Firefox.

For JavaScript updates only, have a look at Artifacts Builds, trading bandwidth for compilation time. 

$ ./mach build faster
$ ./mach xpcshell-test services/common/tests/unit/test_kinto.js
$ ./mach xpcshell-test services/common/tests/unit/test_storage_adapter.js

Or all tests related to Kinto: 

$ ./mach xpcshell-test services/common/tests/unit/*into*

Gfx blocklist tests

The Gfx test suite requires the debug mode to be enabled. Add this to mozconfig file, rebuild and run the tests: 

ac_add_options --enable-debug

Debug content signature

You can get tests (or Firefox) to give you more information on what the content signature verifier is doing by setting the NSPR_LOG_MODULES environment variable. For example: 

EXPORT NSPR_LOG_MODULES=ContentSignatureVerifier:5,CSTrustDomain:5

TDD mode

Using inotify, we will detect a file change in the dist/ folder and run a series of commands to execute the tests automatically.

First, install inotify-tools: 

sudo apt-get install inotify-tools

Then start an infinite loop with inotify-wait: 

while true; do
    # Wait for a change
    inotifywait -q -e create,modify,delete -r ~/Code/Mozilla/kinto.js/dist
    # Execute these commands
    cp ~/Code/Mozilla/kinto.js/dist/moz-kinto-offline-client.js services/common/kinto-offline-client.js
    ./mach xpcshell-test services/common/tests/unit/test_storage_adapter.js
    ./mach xpcshell-test services/common/tests/unit/test_kinto.js          
done

Source: Antoine Cezar

Submit patch

> Patch are contributed to kinto.js and kinto-client.js, which are first released on NPM.

DO NOT land files that are not tagged officially on upstream repositories.

Become a contributor

Configure SSH key for hg: 

Host hg.mozilla.org
  User user@server.com
  IdentityFile ~/.ssh/contrib_moz

Run integration tests: «Try»

See https://wiki.mozilla.org/ReleaseEngineering/TryServer

Or use a gecko-dev fork from Github, and with install moz git tools 

git push-to-try -t --rev master..HEAD ~/hg/mozilla-central/ -b do -p linux,linux64,macosx64,win32,win64 -u xpcshell -t none

Submit for review

See http://mozilla-version-control-tools.readthedocs.org 

# Commit with link to Bugzilla
hg commit -m "Bug XXXXX - Upgrade <lib> to X.Y.Z"

# Submit to MozReview
hg push review

# Keep a bookmark of your branch to address review.
hg bookmark bug-XXXXX

# Go back to «master»
hg update central

Examples:

Retrieved from "https://wiki.mozilla.org/index.php?title=Firefox/Kinto&oldid=1130031"