MozillaWiki

Labs/Jetpack/JEP/11: Difference between revisions

Page Discussion

Labs/Jetpack/JEP/11 (view source)

Revision as of 00:28, 11 July 2009

18,830 bytes removed ,  11 July 2009
Removed async API, replace with the new sync API - see bug 503466
(Expanded asynchronicty rationale)
(Removed async API, replace with the new sync API - see bug 503466)
Line 10: Line 10:
* Created: 27 May 2009
* Created: 27 May 2009
* Reference Implementation: None
* Reference Implementation: None
* Relevant Bugs: {{bug|496694}}, {{bug|499871}}
* Relevant Bugs: {{bug|503466}}, {{bug|499871}}, {{bug|496694}}
* [[Labs/Jetpack/JEPs|JEP Index]]
* [[Labs/Jetpack/JEPs|JEP Index]]


=== Introduction and Rationale ===
=== Introduction and Rationale ===


This JEP describes a simple mechanism through which Jetpacks can persistently and asynchronously store JS primitives and blobs of JSON data.
This JEP describes a simple mechanism through which Jetpacks can persistently store JS primitives and blobs of JSON data.


This proposal is favored over [https://developer.mozilla.org/Pt/DOM/DOM_Storage DOM Storage] because the latter only supports storing strings, which forces the developer to manually perform error-prone parsing tasks for almost any kind of use case. It should also be noted that the simple storage outlined in this proposal can be implemented on the web using DOM Storage; as such, this proposal should not be considered "breaking the web".
This proposal is favored over [https://developer.mozilla.org/Pt/DOM/DOM_Storage DOM Storage] because the latter only supports storing strings, which forces the developer to manually perform error-prone parsing tasks for almost any kind of use case. It should also be noted that the simple storage outlined in this proposal can be implemented on the web using DOM Storage; as such, this proposal should not be considered "breaking the web".
Line 23: Line 23:
=== Proposal ===
=== Proposal ===


Persistent storage will live at <code>jetpack.storage.simple</code>. The <code>jetpack.storage</code> namespace will provide access to any other available storage systems, such as SQLite, secure/password storage, and so on.  The current <code>jetpack.sessionStorage</code> object, which allows arbitrary JS objects (they need not be JSON-able) to be stored between reloads of a Jetpack within the same Firefox session, will be renamed to <code>jetpack.storage.session</code>.
Simple, persistent storage will live at <code>jetpack.storage.simple</code>. The <code>jetpack.storage</code> namespace will provide access to any other available storage systems, such as SQLite, secure/password storage, and so on.  The current <code>jetpack.sessionStorage</code> object, which allows arbitrary JS objects (they need not be JSON-able) to be stored between reloads of a Jetpack within the same Firefox session, will be renamed to <code>jetpack.storage.session</code>.


==== Asynchronicity and Callbacks ====
Simple storage is really simple.  <code>jetpack.storage.simple</code> is a single, persistent JavaScript object available to each Jetpack feature.  For the most part this object is like any other JavaScript object, and a feature can set whatever properties it wants on it.  To manipulate its persistent data, a feature therefore need only use the various [https://developer.mozilla.org/en/Core_JavaScript_1.5_Reference standard JavaScript functions and operators].  Each feature gets its own, private storage.


Firefox is single-threaded to a large extent. The UI, front-end, content, add-ons, Jetpack features, and many other components all share one main thread.  If any one of them blocks on this thread while waiting for some resource, all suffer.  Blocking while waiting on I/O is especially common and egregious, and going forward it is important that all new APIs in Firefox perform I/O off of the main thread as much as possible.  For this reason, the API described here is asynchronous only.  In the future, Jetpack may move to a model where features are run in separate threads. Until such time, this API will remain asynchronous.
The <code>jetpack.simple.storage</code> object is automatically flushed to disk. How and when it is flushed is an implementation detail -- either periodically and on onload or, perhaps in the future if using catch-alls, when a property is set on the object.  Storage may be flushed manually, however, by calling <code>jetpack.storage.simple.sync()</code>.  The object can be forced to reload its data from disk by calling <code>jetpack.storage.simple.open()</code>, although the data comes loaded automatically.


Because of the API's asynchronous nature, its operations communicate their statuses and results to callers via callbacks.  In the context of simple storage, a callback is either a function or an object that defines the methods <code>onResult</code> and <code>onError</code>.  If a callback is a function, it is called when its associated asynchronous operation successfully completes.  If a callback is an object, its <code>onResult</code> method (if defined) is called when the operation successfully completes, and its <code>onError</code> method (if defined) is called when an error occurs during the operation.  This allows callers to provide a simple function in the common case; only if callers wish to be notified of errors do they need to use the more verbose object form.
==== Flushing Storage to Disk ====


In the remainder of this proposal, <code>onResult</code> is shorthand for either the function form or the <code>object.onResult</code> form.  Since there is only one form for errors, <code>onError</code> is shorthand for the <code>object.onError</code> form.
As described above, the <code>jetpack.storage.simple</code> object is automatically written to disk, but a feature may force flush by calling:
 
The arguments passed to <code>onResult</code> and <code>onError</code> depend on the API method to which the callback is passed.  Each method's description below indicates how it calls its callback.
 
Callbacks are optional, but callers should never assume that an operation completes successfully if it is important that it actually does so.
 
==== Storing a Single Item ====
 
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.set(key, value, callback)
</pre>
 
''Arguments''
 
<code>key</code>: A string uniquely identifying the data to be placed into persistent storage.
 
<code>value</code>: A JS primitive or JSON-able JS object that represents the data to be stored in persistent storage.  If some other data with the given <code>key</code> is already being stored, it is overwritten by <code>value</code>. If <code>value</code> is <code>undefined</code>, this method is equivalent to <code>jetpack.storage.simple.remove</code>.
 
<code>callback</code>: A callback. <code>onResult</code> and <code>onError</code> are called as follows:
 
* <code>onResult(key, value)</code>
* <code>onError(errorMessage, key, value)</code>
 
''Return value''
 
This method has no return value.
 
==== Storing Multiple Items ====
 
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.set(itemsObject, callback)
</pre>
 
''Arguments''
 
<code>itemsObject</code>: An object whose properties and their values identify the data to be placed into persistent storage.  As in the single item form of this method, if some other data with a given key is already being stored, it is overwritten by the value in <code>itemsObject</code>.  If a given value is <code>undefined</code>, its key is removed from the store.
 
<code>callback</code>: A callback. <code>onResult</code> and <code>onError</code> are called as follows:
 
* <code>onResult(itemsObject)</code>
* <code>onError(errorMessage, itemsObject)</code>
 
''Return value''
 
This method has no return value.
 
==== Retrieving a Single Item ====
 
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.get(key, callback)
</pre>
 
''Arguments''
 
<code>key</code>: A string uniquely identifying the data to be retrieved from persistent storage.
 
<code>callback</code>: A callback. <code>onResult</code> and <code>onError</code> are called as follows:
 
* <code>onResult(key, value)</code>
** <code>value</code> is <code>key</code>'s associated data.
** If <code>key</code> does not exist in the store, <code>value</code> is <code>undefined</code>.
* <code>onError(errorMessage, key)</code>
 
''Return value''
 
This method has no return value.
 
==== Retrieving Multiple Items ====
 
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.get(keyArray, callback)
</pre>
 
''Arguments''
 
<code>keyArray</code>: An array of strings, each of which uniquely identifies data to be retrieved from persistent storage.
 
<code>callback</code>: A callback. <code>onResult</code> and <code>onError</code> are called as follows:
 
* <code>onResult(itemsObject)</code>
** The properties and values of <code>itemsObject</code> are the key-value pairs corresponding to the keys in <code>keyArray</code>.
** If a key does not exist in the store, its associated value in <code>itemsObject</code> is <code>undefined</code>.
* <code>onError(errorMessage, keyArray)</code>
 
''Return value''
 
This method has no return value.
 
==== Retrieving All Items ====
 
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.get(callback)
</pre>
 
''Arguments''
 
<code>callback</code>: A callback. <code>onResult</code> and <code>onError</code> are called as follows:
 
* <code>onResult(allItemsObject)</code>
** The properties and values of <code>allItemsObject</code> are all of the key-value pairs in the store.
* <code>onError(errorMessage)</code>
 
''Return value''
 
This method has no return value.
 
==== Removing a Single Item ====
 
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.remove(key, callback)
</pre>
 
Removes the key and its associated data from persistent storage. Calling <code>jetpack.storage.simple.remove(key)</code> is the same as calling <code>jetpack.storage.simple.set(key, undefined)</code>.
 
''Arguments''
 
<code>key</code>: The string uniquely identifying the data to be removed from persistent storage.
 
<code>callback</code>: A callback. <code>onResult</code> and <code>onError</code> are called as follows:
 
* <code>onResult(key)</code>
* <code>onError(errorMessage, key)</code>
 
''Return value''
 
This method has no return value.
 
==== Removing Multiple Items ====


<pre class="brush:js;toolbar:false;">
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.remove(keyArray, callback)
jetpack.storage.simple.sync()
</pre>
</pre>


''Arguments''
''Arguments''


<code>key</code>: An array of strings, each of which uniquely identifies data to be removed from persistent storage.
This method takes no arguments.
 
<code>callback</code>: A callback. <code>onResult</code> and <code>onError</code> are called as follows:
 
* <code>onResult(keyArray)</code>
* <code>onError(errorMessage, keyArray)</code>


''Return value''
''Return value''
Line 177: Line 45:
This method has no return value.
This method has no return value.


==== Removing All Items ====
==== Repopulating Storage ====
 
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.clear(callback)
</pre>
 
''Arguments''
 
<code>callback</code>: A callback. <code>onResult</code> and <code>onError</code> are called as follows:
 
* <code>onResult()</code>
* <code>onError(errorMessage)</code>
 
''Return value''
 
This method has no return value.


==== Testing the Existence of a Single Item ====
As described above, the <code>jetpack.storage.simple</code> object is automatically populated when a feature is loaded, but a feature may force the object to read from disk by calling:


<pre class="brush:js;toolbar:false;">
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.has(key, callback)
jetpack.storage.simple.open()
</pre>
</pre>


''Arguments''
Note that any properties already on the object will be overwritten, but no properties are deleted before loading.
 
<code>key</code>: A string uniquely identifying the data whose existence in persistent storage is to be checked.
 
<code>callback</code>: A callback. <code>onResult</code> and <code>onError</code> are called as follows:
 
* <code>onResult(key, exists)</code>
** <code>exists</code> is <code>true</code> if <code>key</code> exists in the store and <code>false</code> otherwise.
* <code>onError(errorMessage, key)</code>
 
''Return value''
 
This method has no return value.
 
==== Testing the Existence of Multiple Items ====
 
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.has(keyArray, callback)
</pre>


''Arguments''
''Arguments''


<code>keyArray</code>: An array of strings, each of which uniquely identifies data whose existence in persistent storage is to be checked.
This method takes no arguments.
 
<code>callback</code>: A callback. <code>onResult</code> and <code>onError</code> are called as follows:
 
* <code>onResult(existsObject)</code>
** <code>existsObject</code> is a JS object whose properties map to the keys in <code>keyArray</code> and whose values are <code>true</code> if their associated keys exist and <code>false</code> otherwise.
* <code>onError(errorMessage, keyArray)</code>


''Return value''
''Return value''


This method has no return value.
This method has no return value.
==== Enumerating Specific Items ====
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.forEachItem(keyArray, callback)
</pre>
''Arguments''
<code>keyArray</code>: An array of strings, each of which uniquely identifies data to retrieve from the store.
<code>callback</code>: A callback. <code>onResult</code> and <code>onError</code> are called as follows:
* <code>onResult(key, value)</code>
** <code>onResult(key, value)</code> is called for each key given in <code>keyArray</code>.
** If a given key does not exist in the store, <code>value</code> will be <code>undefined</code>.
** Items are enumerated in the same order that their keys are given in <code>keyArray</code>.
* <code>onResult(null, null)</code>
** When all items are exhausted, <code>onResult(null, null)</code> is called to signal that enumeration is complete.
* <code>onError(errorMessage, keyArray)</code>
''Return value''
This method has no return value.
==== Enumerating All Items ====
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.forEachItem(callback)
</pre>
''Arguments''
<code>callback</code>: A callback. <code>onResult</code> and <code>onError</code> are called as follows:
* <code>onResult(key, value)</code>
** <code>onResult(key, value)</code> is called for each key-value pair in the store.
**The order of enumeration is arbitrary.
* <code>onResult(null, null)</code>
** When all items are exhausted, <code>onResult(null, null)</code> is called to signal that enumeration is complete.
* <code>onError(errorMessage)</code>
''Return value''
This method has no return value.
==== Enumerating Keys ====
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.forEachKey(callback)
</pre>
''Arguments''
<code>callback</code>: A callback. <code>onResult</code> and <code>onError</code> are called as follows:
* <code>onResult(key)</code>
** <code>onResult(key)</code> is called for each key in the store.
** The order of enumeration is arbitrary.
* <code>onResult(null)</code>
** When all keys are exhausted, <code>onResult(null)</code> is called to signal that enumeration is complete.
* <code>onError(errorMessage)</code>
''Return value''
This method has no return value.
==== Enumerating Specific Values ====
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.forEachValue(keyArray, callback)
</pre>
''Arguments''
<code>keyArray</code>: An array of strings, each of which uniquely identifies data to retrieve from the store.
<code>callback</code>: A callback. <code>onResult</code> and <code>onError</code> are called as follows:
* <code>onResult(value)</code>
** <code>onResult(value)</code> is called for each key given in <code>keyArray</code>.
** If a given key does not exist in the store, <code>value</code> will be <code>undefined</code>.
** Values are enumerated in the same order that their keys are given in <code>keyArray</code>.
* <code>onResult(null)</code>
** When all values are exhausted, <code>onResult(null)</code> is called to signal that enumeration is complete.
** Because <code>value</code> will be <code>undefined</code> when a given key does not exist, <code>onResult</code> should always use strict equality (<code>===</code> instead of <code>==</code> or <code>!value</code>) when checking for the <code>null</code> signal so that an <code>undefined</code> value is not mistaken for it.
* <code>onError(errorMessage, keyArray)</code>
''Return value''
This method has no return value.
==== Enumerating All Values ====
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.forEachValue(callback)
</pre>
''Arguments''
<code>callback</code>: A callback. <code>onResult</code> and <code>onError</code> are called as follows:
* <code>onResult(value)</code>
** <code>onResult(value)</code> is called for each value in the store.
** The order of enumeration is arbitrary.
* <code>onResult(null)</code>
** When all values are exhausted, <code>onResult(null)</code> is called to signal that enumeration is complete.
* <code>onError(errorMessage)</code>
''Return value''
This method has no return value.
==== Retrieving All Keys ====
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.keys(callback)
</pre>
''Arguments''
<code>callback</code>: A callback. <code>onResult</code> and <code>onError</code> are called as follows:
* <code>onResult(keyArray)</code>
** <code>keyArray</code> contains all the keys of the store in an unordered array.
* <code>onError(errorMessage)</code>
''Return value''
This method has no return value.
==== Retrieving Specific Values ====
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.values(keyArray, callback)
</pre>
''Arguments''
<code>callback</code>: A callback. <code>onResult</code> and <code>onError</code> are called as follows:
* <code>onResult(keyArray, valueArray)</code>
** <code>valueArray</code> contains the values of the store corresponding to the given keys.  The values are ordered in the order that their keys are given.  If a given key does not exist in the store, its corresponding value in the array will be <code>undefined</code>.
* <code>onError(errorMessage, keyArray)</code>
''Return value''
This method has no return value.
==== Retrieving All Values ====
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.values(callback)
</pre>
''Arguments''
<code>callback</code>: A callback. <code>onResult</code> and <code>onError</code> are called as follows:
* <code>onResult(valueArray)</code>
** <code>valueArray</code> contains all the values of the store in an unordered array.
* <code>onError(errorMessage)</code>
''Return value''
This method has no return value.
==== Retrieving the Number of Items ====
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.size(callback)
</pre>
''Arguments''
<code>callback</code>: A callback. <code>onResult</code> and <code>onError</code> are called as follows:
* <code>onResult(numItems)</code>
** <code>numItems</code> is the number of key-value pairs in the store.
* <code>onError(errorMessage)</code>
''Return value''
This method has no return value.
==== Mapping Specific Items ====
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.mapItems(keyArray, mapFunction, callback)
</pre>
Applies <code>mapFunction</code> to items in the store corresponding to the given keys and yields the results in an array.  <code>mapFunction</code> is called for each key in <code>keyArray</code> in the order that the keys are given.
''Arguments''
<code>keyArray</code>: An array of keys.
<code>mapFunction</code>: A function to be applied to each item.  Its return value is pushed onto the array that <code>map</code> ultimately yields.  It is called like so:
* <code>mapFunction(key, value)</code>
** If a given key does not exist in the store, <code>value</code> is <code>undefined</code>.
<code>callback</code>: A callback. <code>onResult</code> and <code>onError</code> are called as follows:
* <code>onResult(keyArray, mappedArray)</code>
** <code>mappedArray</code> contains the results of applying <code>mapFunction</code> to each item with a given key.
* <code>onError(errorMessage)</code>
''Return value''
This method has no return value.
==== Mapping All Items ====
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.mapItems(mapFunction, callback)
</pre>
Applies <code>mapFunction</code> to each item in the store and yields the results in an array.  <code>mapFunction</code> is called for each key-value pair in an arbitrary order.
''Arguments''
<code>mapFunction</code>: A function to be applied to each item.  Its return value is pushed onto the array that <code>map</code> ultimately yields.  It is called like so:
* <code>mapFunction(key, value)</code>
<code>callback</code>: A callback. <code>onResult</code> and <code>onError</code> are called as follows:
* <code>onResult(mappedArray)</code>
** <code>mappedArray</code> contains the results of applying <code>mapFunction</code> to each item in the store.
* <code>onError(errorMessage)</code>
''Return value''
This method has no return value.
==== Reducing Specific Items ====
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.reduceItems(keyArray, initialValue, reduceFunction, callback)
</pre>
Combines items in the store corresponding to the given keys by applying <code>reduceFunction</code> to them and accumulating the result.  This method may be known as "inject" or "fold" in other languages.
For each item with a given key, applies <code>reduceFunction</code> to it and stores the return value in an accumulated result.  <code>reduceFunction</code> is then called with the accumulated result for the next item in the store.  The value ultimately yielded by <code>reduceItems</code> is the value accumulated by applying <code>reduceFunction</code> to all the items.
''Arguments''
<code>keyArray</code>: An array of keys.
<code>initialValue</code>: The accumulated result starts as this value.  It is passed in to the first invocation of <code>reduceFunction</code>.
<code>reduceFunction</code>: A function to be applied to each item.  It should return the new accumulated value.  It is called like so:
* <code>reduceFunction(accumulatedValue, key, value)</code>
<code>callback</code>: A callback. <code>onResult</code> and <code>onError</code> are called as follows:
* <code>onResult(keyArray, accumulatedValue)</code>
** <code>accumulatedValue</code> is the result of applying <code>reduceFunction</code> to each item with a given key.
* <code>onError(errorMessage, keyArray)</code>
''Return value''
This method has no return value.
==== Reducing All Items ====
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.reduceItems(initialValue, reduceFunction, callback)
</pre>
Combines all items in the store by applying <code>reduceFunction</code> to them and accumulating the result.  This method may be known as "inject" or "fold" in other languages.
For each item in the store, applies <code>reduceFunction</code> to it and stores the return value in an accumulated result.  <code>reduceFunction</code> is then called with the accumulated result for the next item in the store.  The value ultimately yielded by <code>reduceItems</code> is the value accumulated by applying <code>reduceFunction</code> to all the items.
''Arguments''
<code>initialValue</code>: The accumulated result starts as this value.  It is passed in to the first invocation of <code>reduceFunction</code>.
<code>reduceFunction</code>: A function to be applied to each item.  It should return the new accumulated value.  It is called like so:
* <code>reduceFunction(accumulatedValue, key, value)</code>
<code>callback</code>: A callback. <code>onResult</code> and <code>onError</code> are called as follows:
* <code>onResult(accumulatedValue)</code>
** <code>accumulatedValue</code> is the result of applying <code>reduceFunction</code> to each item in the store.
* <code>onError(errorMessage)</code>
''Return value''
This method has no return value.
==== Errors and Legal Keys and Values ====
A key must be a string.  The empty string is a legal key.  If an illegal key is passed to any method, that method will throw an <code>IllegalKeyError</code>.  The prototype of <code>IllegalKeyError</code> is <code>TypeError</code>.
A value must be a non-null JS primitive or JSON-able object.  If an illegal value is passed to any method, that method will throw an <code>IllegalValueError</code>.  The prototype of <code>IllegalValueError</code> is <code>TypeError</code>.
Because of the polymorphic nature of this API, if any method is passed an unexpected argument, that method will throw an <code>ArgumentError</code>.  The prototype of <code>ArgumentError</code> is <code>TypeError</code>.
==== Other Notes ====
''Equivalence vs. Equality''
The values passed into <code>set()</code> and returned from <code>get()</code> are all serialized/de-serialized at the time of calling; this means that, for instance, given the following code:
<pre class="brush:js;toolbar:false;">
var original = {foo: 'bar'};
jetpack.storage.simple.set('test', original);
</pre>
the following condition will hold:
<pre class="brush:js;toolbar:false;">
jetpack.storage.simple.get('test', function (key, val) {
  assert(val != original);
});
</pre>
Adw
Confirmed users
764

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/154583"