Storage: Difference between revisions

Storage (view source)

Revision as of 15:55, 13 September 2018

203 bytes added , 13 September 2018

Update the page per discussion with storage advisory group

Andrew Sutherland

Confirmed users

360

edits

@@ Line 1: / Line 1: @@
-Storage is a SQLite database API.  API documentation is available in the [http://developer.mozilla.org/en/docs/Storage Mozilla Developer Center].
+Storage can mean many things in Gecko and Firefox.
-== Proposed Future Development Priorities ==
+Possibilities include:
+* mozStorage, the API that lives under [https://searchfox.org/mozilla-central/source/storage storage/] in the source tree.  This is a C++ wrapper around SQLite that is also exposed to JS via both XPCOM/XPConnect ([https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Storage API docs]) and via a friendlier JS wrapper, [https://developer.mozilla.org/en-US/docs/Mozilla/JavaScript_code_modules/Sqlite.jsm SQLite.jsm].  Bug 1482608 also
+* The profile "storage/" directory that holds per-origin storage managed by the QuotaManager subsystem and its clients: IndexedDB, the Cache API, and the asm.js cache (which will be removed).
-=== Full Text Indexing ===
+== Storage Alternatives ==
-* separation of indexing from storage
-** Firefox could use this to index visited web pages without having to store the contents of the pages themselves.
-** Thunderbird could use this to index email messages without having to store their contents in SQLite (ultimately it might make sense to store contents as well, but that's a larger, longer-term change).
-** Komodo could use this to index source code stored in files on disk.
-* built-in tokenization of Unicode text (i.e. without having to either compile in a large separate library or write custom functions);
-* tokenization of marked up text (HTML, PDF, etc.).
-=== Performance ===
+There are many options for storing data in Gecko.  The Browser Architecture Group has [https://github.com/mozilla/firefox-data-store-docs cataloged] many of the existing uses.
-Dr. Hipp posted to the SQLite users list about a disk seek performance optimization that could be a boon to Places performance:
+If you are looking for a way to store data in Firefox, here are the usual options:
+* JSON flat files.  This usually is not a viable option from C++ because we don't have convenient ways to manipulate JS objects from C++.
-  Creating an index on A,B is equivalent to sorting on A,B.
+* [https://github.com/mozilla/rkv rkv], a "simple, humane, typed Rust interface to LMDB".  LMDB is a memory-mapped flat file database.
+* SQLite via the mozStorage API or bindings.  You should *NOT* directly access the SQLite API.  If you are using rust code and would like a better binding API than mozStorage, that makes sense, but please reach out to the Storage Advisory Group so that we can reach consensus.  We do not want to have N separate bindings to SQLite, especially since mozStorage is responsible for global configuration settings that could fight with other SQLite bindings.
- The sorting algorithm currently used by SQLite requires
- O(NlogN) comparisons, which is optimial.  But it also requires
- O(N) disk seeks, which is very suboptimal.  You don't notice
- all these seeks if your database fits in cache.  But when you
- get into databases of about 3GB, the seeking really slows you
- down.
- A project on our to-do list is to implement a new sorter
- that uses O(1) seeks.  We know how to do this.  It is just
- finding time to do the implementation.