View source for DOM/QuotaManager/SchemaChanges

This page tracks schema changes to Firefox storage subsystems that may cause downgrade problems. It's primarily concerned with QuotaManager and its clients, but other subsystems are listed here as well.

Testing page: https://firefox-storage-test.glitch.me/

Last updated: May 15, 2018 with Firefox 62 on mozilla-central trunk.

'''Schema Changes By Release Version'''<br />

Pending: Currently targeted for Firefox 62
https://bugzilla.mozilla.org/show_bug.cgi?id=1286798 Next generation LocalStorage
* QuotaManager and LocalStorage will be affected, but is currently slated to be downgrade friendly.
Firefox 62
https://bugzilla.mozilla.org/show_bug.cgi?id=1239708
* Places v48
Firefox 61
https://bugzilla.mozilla.org/show_bug.cgi?id=1452660
* Places v47
https://bugzilla.mozilla.org/show_bug.cgi?id=1293445
* Places v46
https://bugzilla.mozilla.org/show_bug.cgi?id=1426554
* Places v45
https://bugzilla.mozilla.org/show_bug.cgi?id=1310295
* Places v44
Firefox 60
https://bugzilla.mozilla.org/show_bug.cgi?id=1313188
* Places v43
https://bugzilla.mozilla.org/show_bug.cgi?id=1346554
* Places v42
Firefox 59
Firefox 58
https://bugzilla.mozilla.org/show_bug.cgi?id=1356531
* Places v41
https://bugzilla.mozilla.org/show_bug.cgi?id=1399879
* Places v40 (soft failures - Places bumps schema version up/down)
Firefox 57
https://bugzilla.mozilla.org/show_bug.cgi?id=1404344
* MAJOR HACKY DOWNGRADE FIX.
* Quota Manager v3 aliased back down to v2.1
* Cache API v27 introduced that masquerades as v25 on disk
* (ServiceWorkerRegistrar changes left intact because that does not result in hard breakage.)
https://bugzilla.mozilla.org/show_bug.cgi?id=1290481
* Quota Manager v3 (storage.sqlite)
* Cache API v26
https://bugzilla.mozilla.org/show_bug.cgi?id=1353636
* ServiceWorkerRegistrar v8
https://bugzilla.mozilla.org/show_bug.cgi?id=1396282
* Places v39 (soft failures)
Firefox 56
https://bugzilla.mozilla.org/show_bug.cgi?id=1352502
* Places v38 (soft failures)
Firefox 55
https://bugzilla.mozilla.org/show_bug.cgi?id=1339081 (primary), https://bugzilla.mozilla.org/show_bug.cgi?id=1314361, https://bugzilla.mozilla.org/show_bug.cgi?id=1311057
* Quota Manager v2 (storage.sqlite)
* IndexedDB v26
* LocalStorage/DOMStorage v2 (soft failures except on debug, https://bugzilla.mozilla.org/show_bug.cgi?id=1345760 )
https://bugzilla.mozilla.org/show_bug.cgi?id=1251238
* ServiceWorkerRegistrar v7
https://bugzilla.mozilla.org/show_bug.cgi?id=977177
* Places v37 (soft failures)
Firefox 54
https://bugzilla.mozilla.org/show_bug.cgi?id=1322111
* Cache API v25
Firefox 53
https://bugzilla.mozilla.org/show_bug.cgi?id=1290944
* ServiceWorkerRegistrar v6 (not QM, handles downgrades by clearing registry but does not break)
https://bugzilla.mozilla.org/show_bug.cgi?id=1181127
* ServiceWorkerRegistrar v5 (not QM, handles downgrades by clearing registry but does not break)
https://bugzilla.mozilla.org/show_bug.cgi?id=1258127
* Places v36 (soft failures)
Firefox 52
https://bugzilla.mozilla.org/show_bug.cgi?id=1264178
* Cache API v24
https://bugzilla.mozilla.org/show_bug.cgi?id=768074
* Cache API v23
https://bugzilla.mozilla.org/show_bug.cgi?id=964561 (store large structured clones as separate files)
* IndexedDB v25 too. This landed the same day as https://bugzilla.mozilla.org/show_bug.cgi?id=1302036 so it used the same v25 schema bump, but was not uplifted to Firefox 51.
https://bugzilla.mozilla.org/show_bug.cgi?id=1302901
* Places v35 (soft failures)
Firefox 51
https://bugzilla.mozilla.org/show_bug.cgi?id=1302036 (via aurora uplift, central was 52) (2016-10-29)
* IndexedDB v25
* JS Structured Clone v8
https://bugzilla.mozilla.org/show_bug.cgi?id=1187335 (2016-09-08)
* Cache API v22
Firefox 50
https://bugzilla.mozilla.org/show_bug.cgi?id=1264053 (via aurora uplift, central was 51)
* IndexedDB v24
* JS Structured Clone v7
Firefox 49
https://bugzilla.mozilla.org/show_bug.cgi?id=1195930 (2016-06-06)
* Quota Manager v1 (storage.sqlite, also '''introduced .metadata-v2''')
* IndexedDB v23
https://bugzilla.mozilla.org/show_bug.cgi?id=1243792 (2016-05-10)
* Cache API v21
(see &quot;by subsystem&quot; for older versions)
<br />
'''Related Changes By Release Version'''<br />

Firefox 53
https://bugzilla.mozilla.org/show_bug.cgi?id=1321550
nsPrincipal::GetOrigin now truncates query and hash parts of the origin for file: URI's. This did not impact schemes other than &quot;file&quot; (and maybe &quot;chrome:///blah&quot;?), because:
* &quot;about&quot;, &quot;moz-safe-about&quot;, &quot;indxeddb&quot; specs all fast-path out and use GetAsciiSpec.
* non-&quot;chrome&quot; protocols which return something to GetAsciiHostPort(). This does not cover file URI. It seems like &quot;chrome:///&quot; and &quot;resource:///&quot; may have been impacted if they use a ContentPrincipal, but neither of those should successfully load as a navigation with a content principal.
https://bugzilla.mozilla.org/show_bug.cgi?id=1286914
* QM learned to support the &quot;moz-extension&quot; scheme.
Firefox 50
https://bugzilla.mozilla.org/show_bug.cgi?id=1280336
* QM learned to support the &quot;about&quot; scheme. Previously it only understood &quot;moz-safe-about&quot;.
Firefox 49
https://bugzilla.mozilla.org/show_bug.cgi?id=1195930
* QM learned to support the &quot;resource&quot; scheme.
<br />
'''By Subsystem''' (alphabetical) and '''how it handles failures:'''<br />
&quot;Hard failures&quot; mean that the subsystem checks the schema version and fails to initialize or work if downgraded. &quot;Soft failures&quot; mean that the subsystem will try and use the database but may encounter failures. &quot;Schema down-rev'ed&quot; means that the schema version is set to the schema version the current version of Firefox understands, which means that the migration logic will be re-run if/when the profile is used on a more modern Firefox release.


Cache API (QuotaManager client). Hard failures on downgrade.
* v27 represented on-disk as v25 (concurrent with QM v2.1 aliasing), Firefox 57, https://bugzilla.mozilla.org/show_bug.cgi?id=1404344
* v26 (concurrent with QM v3), Firefox 57, https://bugzilla.mozilla.org/show_bug.cgi?id=1290481
* v25, Firefox 54, https://bugzilla.mozilla.org/show_bug.cgi?id=1322111
* v24, Firefox 52, https://bugzilla.mozilla.org/show_bug.cgi?id=1264178
* v23, Firefox 52, https://bugzilla.mozilla.org/show_bug.cgi?id=768074
* v22, Firefox 51, https://bugzilla.mozilla.org/show_bug.cgi?id=1187335
* v21, Firefox 49, https://bugzilla.mozilla.org/show_bug.cgi?id=1243792
* v20, Firefox 47, https://bugzilla.mozilla.org/show_bug.cgi?id=1251872
IndexedDB (QuotaManager client). Hard failures on downgrade.
v26 (concurrent with QM v2, LocalStorage v2), Firefox 55, https://bugzilla.mozilla.org/show_bug.cgi?id=1339081
v25 (concurrent with Structured Clone v8), Firefox 51 (via aurora uplift, central was 52), https://bugzilla.mozilla.org/show_bug.cgi?id=1302036
* Note that https://bugzilla.mozilla.org/show_bug.cgi?id=964561 also landed the same day on central for 52 and so used the same schema bump, but was not uplifted to 51.
v24 (concurrent with Structured Clone v7), Firefox 50 (via aurora uplift, central was 51), https://bugzilla.mozilla.org/show_bug.cgi?id=1264053
v23 (concurrent with QM v1), Firefox 49, https://bugzilla.mozilla.org/show_bug.cgi?id=1195930
v22 (concurrent with Structured Clone v6), Firefox 44, https://bugzilla.mozilla.org/show_bug.cgi?id=1201620
v21, Firefox 43, https://bugzilla.mozilla.org/show_bug.cgi?id=1202788
v20, Firefox 43, https://bugzilla.mozilla.org/show_bug.cgi?id=771288
v19, Firefox 43, https://bugzilla.mozilla.org/show_bug.cgi?id=871846
v18, Firefox 40, https://bugzilla.mozilla.org/show_bug.cgi?id=1131776
JS Structured Clone (JS subsystem used by IndexedDB persistence). Hard failures on downgrade.
* v8 (concurrent with IDB v25 bump), Firefox 51 (via aurora uplift, central was 52), https://bugzilla.mozilla.org/show_bug.cgi?id=1302036
* v7 (concurrent with IDB v24 bump), Firefox 50 (via aurora uplift, central was 51), https://bugzilla.mozilla.org/show_bug.cgi?id=1264053
* v6 (concurrent with IDB v22 bump), Firefox 44, https://bugzilla.mozilla.org/show_bug.cgi?id=1201620
LocalStorage/DOMStorage (Independent, soon to be a QuotaManager client).
* v2 (concurrent with QM v2, IDB v26), Firefox 55, https://bugzilla.mozilla.org/show_bug.cgi?id=1314361
* v1, Firefox 46, https://bugzilla.mozilla.org/show_bug.cgi?id=1165214
Places (Never going to be a quota manager client). Soft failures on downgrade, schema down-rev'ed.
* v48, Firefox 62, https://bugzilla.mozilla.org/show_bug.cgi?id=1239708
* v47, Firefox 61, https://bugzilla.mozilla.org/show_bug.cgi?id=1452660
* v46, Firefox 61, https://bugzilla.mozilla.org/show_bug.cgi?id=1293445
* v45, Firefox 61, https://bugzilla.mozilla.org/show_bug.cgi?id=1426554
* v44, Firefox 61, https://bugzilla.mozilla.org/show_bug.cgi?id=1310295
* v43, Firefox 60, https://bugzilla.mozilla.org/show_bug.cgi?id=1313188
* v42, Firefox 60, https://bugzilla.mozilla.org/show_bug.cgi?id=1346554
* v41, Firefox 58, https://bugzilla.mozilla.org/show_bug.cgi?id=1356531
* v40, Firefox 58, https://bugzilla.mozilla.org/show_bug.cgi?id=1399879
* v39, Firefox 57, https://bugzilla.mozilla.org/show_bug.cgi?id=1396282
* v38, Firefox 56, https://bugzilla.mozilla.org/show_bug.cgi?id=1352502
* v37, Firefox 55, https://bugzilla.mozilla.org/show_bug.cgi?id=977177
* v36, Firefox 53, https://bugzilla.mozilla.org/show_bug.cgi?id=1258127
* v35, Firefox 52, https://bugzilla.mozilla.org/show_bug.cgi?id=1302901
* ...stopping because ESR52 is as far back as we care.
Push (preliminary investigation suggests hard DB breakage on downgrade, but there may be an error handling path I'm missing: 10/17/2017) (each push service has its own database with its own revision, although they track the underlying IDB schema)
pushAndroidGCM
* v5, Firefox 47, https://bugzilla.mozilla.org/show_bug.cgi?id=1214338
pushHttp2
* v5, Firefox 43, https://bugzilla.mozilla.org/show_bug.cgi?id=1172502
pushapi (websocket)
* v5, Firefox 43, https://bugzilla.mozilla.org/show_bug.cgi?id=1185544
Quota Manager (aka QM). Hard failures on downgrade, including hard failures for all
* v3 (storage.sqlite, concurrent with Cache API v26), Firefox 57, https://bugzilla.mozilla.org/show_bug.cgi?id=1290481
* v2 (storage.sqlite, concurrent with IDB v26, LocalStorage v2), Firefox 55, https://bugzilla.mozilla.org/show_bug.cgi?id=1339081
* v1 (storage.sqlite, introduced .metadata-v2, concurrent with IDB v23), Firefox 49, https://bugzilla.mozilla.org/show_bug.cgi?id=1195930
ServiceWorkerRegistrar: (Not currently a quota manager client, although the Cache API is and that's also essential for ServiceWorkers to operate.) Currently handles downgrades by deleting PROFILE/serviceworker.txt, effectively removing all ServiceWorkers and orphaning/leaking their storage (DOM Cache API currently has no reconciliation between used chrome storage and registrations).
* v8, Firefox 57, https://bugzilla.mozilla.org/show_bug.cgi?id=1353636
* v7, Firefox 55, https://bugzilla.mozilla.org/show_bug.cgi?id=1251238
* v6, Firefox 53, https://bugzilla.mozilla.org/show_bug.cgi?id=1290944
* v5, Firefox 53, https://bugzilla.mozilla.org/show_bug.cgi?id=1181127
* v4, Firefox 47, https://bugzilla.mozilla.org/show_bug.cgi?id=1247970
* v3, Firefox 46, https://bugzilla.mozilla.org/show_bug.cgi?id=1229795
* v2, Firefox 43, https://bugzilla.mozilla.org/show_bug.cgi?id=1189235
<br />
'''Error Reporting''' (when things go wrong, what hints do we have?):<br />

Quota Manager:
QM_WARNING reports the specific file and line where an error occurred to the console service using a classification of &quot;quota&quot;. (This happens via mozilla::dom::quota::ReportInternalError and nsContentUtils::LogSimpleConsoleError.) NS_WARNING will also be invoked with the generated string, which will be logged to stderr/logcat (&quot;Gecko&quot;). This general mechanism has been around since https://bugzilla.mozilla.org/show_bug.cgi?id=1124189 landed in Firefox 38. Prior to this landing, there were a number of nsPrintfCString's used to build a message that was logged via NS_WARNING,
* &quot;Origin '%s' failed to parse, handled tokens: %s&quot; reported during origin parsing which will occur during origin initialization during QM's initialization phase at startup. (It may also occur during an attempt to create a new origin #ifndef RELEASE_OR_BETA, see the &quot;Preventing creation...&quot; error below.) Around since https://bugzilla.mozilla.org/show_bug.cgi?id=1124189 landed in Firefox 38.
* &quot;Preventing creation of a new origin directory which is not supported by our origin parser or is obsolete!&quot;: #ifndef RELEASE_OR_BETA in Firefox 53 as part of https://bugzilla.mozilla.org/show_bug.cgi?id=1286914
* There are a number of other origin parsing errors that may happen too.
* UNKNOWN_FILE_WARNING(_leafName) sub-type, &quot;Something (%s) in the directory that doesn't belong!&quot; reported during origin initialization and/or upgrades. New in Firefox 55 as part of https://bugzilla.mozilla.org/show_bug.cgi?id=1339081.
IndexedDB:
IDB_WARNING pre-dates QM_WARNING and is effectively identical to it, the only difference being it logs to the console with a classification of &quot;indexedDB&quot;. It has existed since https://bugzilla.mozilla.org/show_bug.cgi?id=958359 landed in Firefox 30, although it only gained sprintf powers in https://bugzilla.mozilla.org/show_bug.cgi?id=1339826 in Firefox 55. There is also an IDB_REPORT_INTERNAL_ERR variant without the NS_WARNING, plus an IDB_ENSURE_TRUE and IDB_ENSURE_SUCCESS that build on IDB_REPORT_INTERNAL_ERR.
* &quot;Maintenance finished with error: %s&quot; for Maintenance errors. Firefox 55, https://bugzilla.mozilla.org/show_bug.cgi?id=1339826
<br />
To improve:<br />

* Upgrade QM &quot;version from future&quot; NS_WARNING to QM_WARNING: https://searchfox.org/mozilla-central/source/dom/quota/ActorsParent.cpp#4847
<br />
'''Upcoming Changes'''<br />
The following bugs are expected to result in schema changes:<br />

* Support Alternate Data (ex: JS Bytecode) in DOM Cache: https://bugzilla.mozilla.org/show_bug.cgi?id=1336199 will store additional blob files on disk
* LocalStorage overhaul: https://bugzilla.mozilla.org/show_bug.cgi?id=1286798 should move LocalStorage to QuotaManager storage (potentially using a hybrid-upgrade approach) and address quantum dom labeling issues.
* ServiceWorkerRegistrar overhaul: https://bugzilla.mozilla.org/show_bug.cgi?id=1183245
* Simple QuotaManager client: https://bugzilla.mozilla.org/show_bug.cgi?id=1361330
<br />
'''Current Versions'''<br />
The list of current subsystem versions with searchfox queries to make it easier to tell when things have changed. This duplicates the &quot;By Subsystem&quot; info up above but makes it easier to read.


Cache API
* https://searchfox.org/mozilla-central/search?q=kLatestSchemaVersion+%3D&case=false&regexp=false&path=dom%2Fcache%2FDBSchema.cpp
* const int32_t kLatestSchemaVersion = 27;
IndexedDB
* https://searchfox.org/mozilla-central/search?q=kMajorSchemaVersion+%3D&case=false&regexp=false&path=dom%2FindexedDB%2FActorsParent.cpp
* const uint32_t kMajorSchemaVersion = 26;
JS Structured Clone
* https://searchfox.org/mozilla-central/search?q=%23define+JS_STRUCTURED_CLONE_VERSION&case=false&regexp=false&path=js%2Fpublic%2FStructuredClone.h
* #define JS_STRUCTURED_CLONE_VERSION 8
LocalStorage
* https://searchfox.org/mozilla-central/search?q=%23define+CURRENT_SCHEMA_VERSION&case=false&regexp=false&path=dom%2Fstorage%2FStorageDBThread.cpp
* #define CURRENT_SCHEMA_VERSION 2
Places
* https://searchfox.org/mozilla-central/search?q=%23define+DATABASE_SCHEMA_VERSION&case=false&regexp=false&path=toolkit%2Fcomponents%2Fplaces%2FDatabase.h
* #define DATABASE_SCHEMA_VERSION 47
Push
pushAndroidGCM:
* https://searchfox.org/mozilla-central/search?q=const+kPUSHANDROIDGCMDB_DB_VERSION+%3D&case=false&regexp=false&path=dom%2Fpush%2FPushServiceAndroidGCM.jsm
* const kPUSHANDROIDGCMDB_DB_VERSION = 5; // Change this if the IndexedDB format changes
pushHttp2
* https://searchfox.org/mozilla-central/search?q=const+kPUSHHTTP2DB_DB_VERSION+%3D&case=false&regexp=false&path=dom%2Fpush%2FPushServiceHttp2.jsm
* const kPUSHHTTP2DB_DB_VERSION = 5; // Change this if the IndexedDB format changes
pushapi (websocket)
* https://searchfox.org/mozilla-central/search?q=const+kPUSHWSDB_DB_VERSION+%3D&case=false&regexp=false&path=dom%2Fpush%2FPushServiceWebSocket.jsm
* const kPUSHWSDB_DB_VERSION = 5; // Change this if the IndexedDB format changes
QuotaManager
* https://searchfox.org/mozilla-central/search?q=kMajorStorageVersion+%3D&case=false&regexp=false&path=dom%2Fquota%2FActorsParent.cpp
* const uint32_t kMajorStorageVersion = 2;
* const uint32_t kMinorStorageVersion = 1;
ServiceWorkerRegistrar
* https://searchfox.org/mozilla-central/search?q=%23define+SERVICEWORKERREGISTRAR_VERSION&case=false&regexp=false&path=dom%2Fserviceworkers%2FServiceWorkerRegistrar.h
* #define SERVICEWORKERREGISTRAR_VERSION &quot;8&quot;


<br />