View source for FirefoxOS/Device Storage/Worker Design

= Phase 1 =

== Expose DOMRequest to workers ==

See [https://bugzilla.mozilla.org/show_bug.cgi?id=1167650 bug 1167650].

DOMRequest is currently only accessible on the main thread. It be supported by workers we must:

# Allow it to be constructed using nsIGlobalObject* (fed into DOMEventTargetHelper) in addition to nsPIDOMWindow*. The underlying Promise needs that to get a JS context and check state.
# DOMRequestService::FireSuccessAsync (FireSuccessAsyncTask by extension) and DOMRequestService::FireErrorAsync must be changed to dispatch to the current thread and remove the assert checks for main thread. mozilla::ThreadsafeAutoSafeJSContext should be switched for AutoJSAPI and Init(nsIGlobalObject*) (gotten from DOMRequest::GetParentObject); this will assert that the current thread is owning thread.
# Add Exposed=(Window,Worker) to the WebIDL definition.

== Volume manager ==

See [https://bugzilla.mozilla.org/show_bug.cgi?id=1166320 bug 1166320].

Volume service is designed to be accessed off the main thread, but there are problems with the current implementation.

# When a volume is removed via nsVolumeService::RemoveVolumeByName, the actual removal of the pointer from mVolumeArray is not protected by the array lock.
# When a volume is updated via nsVolumeService::UpdateVolume, clients of the service may already hold a pointer to the volume object being updated and be concurrently accessing its properties, including non-primitives such as nsString. Rather than update the object in place and risk indeterminate behaviour, the volume service should store a new volume object while holding the array lock. This way the client will have a consistent (if stale) state and may retry as appropriate if the operation fails as a result (this race condition should already be considered).

As a result, from the perspective of the volume service client, a volume object's properties are immutable.

Operations such as mount, unmount and format, may only be completed in the parent process, and thus are not a concern for web workers.

== Abstract use of preferences to dedicated module ==

Preferences can only be accessed on the main thread. Workers should access a cached, thread safe version of that in a dedicated module. This should merge uses of preferences throughout the code, including GlobalDirs and OverrideRootDir.

This module should be a singleton, created by nsLayoutStatics::Initialize and destroyed via ClearOnShutdown. If the singleton is missing, all device storage APIs should fail (rather than attempt to initialize given they may not be on the main thread). It should register as a listener to the preferences module to update the cache if any settings change.

Mutex protected accessors should be provided for:
* device.storage.prompt.testing
* device.storage.writable.name
* device.storage.testing
* device.storage.overrideRootDir

As well as accessors for paths on platforms other than Gonk (as exported by GlobalDirs):
* videos
* pictures
* music
* sdcard

Ideally the concept of overrideRootDir would be abstracted if possible so that nsDOMDeviceStorage can ignore it.

== Consolidate and cache use of permissions ==

When checking the permissions, we must run on main thread to possibly put up allow/deny dialog and to query the parent process (if a child). To minimize delegating to the main thread from workers, we should cache the permission check result. In addition this should reduce the latency of device storage APIs due to removing an extra round trip between the parent and child processes.

In nsDOMDeviceStorage, store cached values (allow/deny) and cache state (un/initialized) for each. These are accessed on worker thread only. There are 4 access types used in conjunction with permission "device-storage:XXXtype":
* read
* write
* created
* undefined

nsDOMDeviceStorage::DeviceStorageRequest is used for this purpose presently. Other classes (i.e. DeviceStorageCursorRequest) should be changed to inherit/override methods on this instead of duplicating permission functionality. Our own versions of ::Allow and ::Cancel (e.g. ::AllowDs and ::CancelDs) should be provided and are intended to be run on the worker thread context, in addition to exposing the owning thread.

nsDOMDeviceStorage::CheckPermission should be added which accepts a DeviceStorageRequest and the permission to be verified. If the cache is initialized, it should trigger DeviceStorageRequest::AllowDs or ::CancelDs inline without posting a message. If the cache is uninitialized, it should post a runnable to the main thread to trigger nsContentPermissionUtils::AskPermission. When the permission manager's ::Allow/::Cancel is triggered, it should post a runnable to the worker thread to store the permission (i.e. nsDOMDeviceStorage::StorePermission) and then trigger DeviceStorageRequest::AllowDs or ::CancelDs as appropriate.

(Note that if it is a child process, it will most likely need to pass the request back onto the main thread in order proxy it to the parent process via PContent in the intial implementation.)

There is a special case for checking the "webapps-manage" permission. This is queried in nsDOMDeviceStorage::Init which is a synchronous function. As a result, for workers, this must be cached in the worker load info (see WorkerPrivate::GetLoadInfo) which when created is run on the main thread. A static helper function should be provided by nsDOMDeviceStorage (i.e. ::CheckAppsStoragePermission) to do the actual query, which may be reused when nsDOMDeviceStorage objects are created on the main thread. An example of something similar to this today is WorkerLoadInfo::mIndexedDBAllowed.

This is a special case for adding an event listener. While one may register without any permissions, one cannot receive the events unless one is permitted (otherwise the event is dropped). Presently we overload DOMEventTargetHelper::AddEventListener to trigger this permission check, which has several variations. We should switch to overloading DOMEventTargetHelper::EventListenerWasAdded which is meant for this purpose in derived classes and is triggered no matter how the listener is added.

== Observers ==

The observer service can only be accessed on the main thread. Workers should access a cached, thread safe version of that in a dedicated module. This should merge uses of observers throughout the code, including FileUpdateDispatcher.

This module (ObserverManager) should be a singleton, created by nsLayoutStatics::Initialize and destroyed via ClearOnShutdown. If the singleton is missing, all device storage APIs should fail (rather than attempt to initialize given they may not be on the main thread).

When an nsDOMDeviceStorage object is created/destroyed, it should register/deregister as a listener. Any ObserverManager::Observe notifications should be proxied on the current thread to the nsDOMDeviceStorage objects, after doing any common work such as querying the volume manager.

The following events will need to be pushed to the nsDOMDeviceStorage objects on the worker threads:
* nsPref:changed
* volume-state-changed
* file-watcher-update
* disk-space-watcher

The following events may be processed inside the ObserverManager:
* file-watcher-notify
* download-watcher-notify

There is a special case for event listeners as presently it adds/removes an observer if a system listener is added to nsDOMDeviceStorage (see ::AddSystemEventListener and ::RemoveEventListener). Since nsDOMDeviceStorage (and going forward the ObserverManager) already listen on those events, this code should be removed as there is no need for a special case (additionally, there does not appear to be any system listeners using nsDOMDeviceStorage, so this is dead code.)

Rather than accept nsDOMDeviceStorage objects directly, it should define an interface which it implements so that other objects (i.e. DeviceStorageAreaListener) may reuse the same module to receive volume state change notifications.

== Type checker (DeviceStorageTypeChecker) ==

The string bundle service can only be accessed on the main thread. This module will need to be initialized / destroyed by nsLayoutStatics::Initialize / ClearOnShutdown. It is otherwise thread safe already.

== Used space cache (DeviceStorageUsedSpaceCache) ==

This is only accessed on parent process, so there is no child process concerns. However since workers can also exist on the parent, it needs to be made thread safe. It may continue to be initialized on demand since it has no dependencies on the main thread. All operations besides initialization appear to run on a dedicated IO thread. As such, only the ::CreateOrGet method needs to be protected via a mutex.

== Device storage IPC API with child-parent processes ==

PContent is the IPC object which marshalls access to the device storage IPC APIs. This is a singleton and can only be accessed on the main thread.

Sending messages happens in the following functions:
* ContinueCursorEvent::Continue
* nsDOMDeviceStorageCursor::Allow
* DeviceStorageRequest::Allow
* FileUpdateDispatcher::Observe

For FileUpdateDispatcher::Observe, this will always happen on the main thread and can be left as is. For the others, the caller may not be the main thread, in which case it must create a runnable to proxy the call to the main thread.

Receiving the responses happens in DeviceStorageRequestChild::Recv__delete__. This will need to create a runnable to proxy the response to the owning thread of the DeviceStorageRequest.

== Expose Device Storage to workers ==

The appropriate WebIDL Exposure tags should be added to all device storage interfaces and the Navigator device storage partial interfaces.

Similar to Navigator, WorkerNavigator is exposed on worker threads. WorkerNavigator::GetDeviceStorage and ::GetDeviceStorageAreaListener must be added, along with new corresponding factory methods in device storage.

DeviceStorageAreaListener should be updated to listener on the ObserverManager, same as nsDOMDeviceStorage, in order to receive the volume state change messages.

All existing device storage test cases should be updated/cloned to run in both a worker and the main thread contexts.

= Phase 2 (to be broken down) =

== Device storage IPC API with child-parent processes ==

PContent causes device storage to depend on main thread in the child process. There are two alternatives to this:
# Move the device storage IPC APIs to a new IPC object which descends from PBackground.
# Switch to the new POSIX file IPC APIs as currently in review in [https://bugzilla.mozilla.org/show_bug.cgi?id=1141021 bug 1141021]. Also descends from PBackground.

All IPC objects can only be accessed on the thread in which they are created. When one uses PBackground, one is responsible for creating the IPC objects and thus may control what thread is used. In the singleton case, we would still need to post our IPC requests to another thread, but it would be a dedicated IO thread rather than the main thread, allowing better use of multi-core resources (in theory).

If it is possible (unconfirmed) to have an IPC object for each worker thread, then the thread context switch could be avoided.

== Code reorganization ==

* Modules which have an inherent main thread dependency (i.e. observers, preferences) should be moved outside nsDeviceStorage.cpp to a dedicated file.
* There is code duplication between nsDOMDeviceStorage and DeviceStorageParent; this should be consolidated into a dedicated module/file that both depend. This equally applies to a potential switch to the POSIX IPC APIs.
* Any code that must execute on the parent should be moved into DeviceStorageParent.
* Any code that must execute on the child should be moved into DeviceStorageChild.
* Accesses to parent/child paths should be abstracted from users (i.e. DeviceStorageService proxies to DeviceStorageParent/Child as appropriate) via a thin/inlined layer.
* What remains in nsDeviceStorage.cpp should be DOM oriented and run primarily in the thread context of the object owner.