Necko/Cache/Plans/Draft Proposal

API changes proposal

Before you start reading, fixing this requirement depends more on the implementation than on the API, keep this in mind when reading please.

Few implementation points, to defend the API changes: - entries intended to be stored on disk will also be held in memory cache (a write-back cache schema) for quick access - entries will be shared by consumers and will allow write and concurrent read by other consumers (no blocking until an entry is closed) - all disk reads and writes will be only handled on a background thread - appcache will use the same storage mechanism, this will effectively get rid of synchronous sqlite writes and reads on the main thread

I present all new functions and classes as non-COM, but we will probably need to turn most of them to be XPCOM accessible and with interfaces to expose them to tests and JS chrome.

Functions used by nsHttpChannel, nsWyciwygChannel and nsFtpChannel (used by Necko)

You query for an object that represents a storage (similar to the current nsICacheSession interface). Obtaining this object doesn't involve any I/O, it just gives you an API to use to open (or evict) entries in desired cache in an asynchronous way.

• Get a storage to only look at and store in the memory cache

CacheStorage* mozilla::net::MemoryCacheStorage(nsILoadContextInfo* aLoadContextInfo);

• Get a storage to look in memory and then disk cache / store in both ; when aLookupAppCacheFirst == true then appcache is first looked up to satisfy opening requests - used for top level loads only ; for reference why we need to look appcache up first please see the offline application cache specification, explanation is out of scope of this email

CacheStorage* mozilla::net::DiskCacheStorage(nsILoadContextInfo* aLoadContextInfo,
                                             bool aLookupAppCacheFirst);

• When having appcache object in hands (obtained via AppCacheIndex or onAppCacheEntryAvailable (both described bellow) or by association of sub resource channels with an appcache) create or open entries using that particular appcache ; again, refer to the offline application cache specification for more info

CacheStorage* mozilla::net::AppCacheStorage(nsIApplicationCache* aAppCache);

• Helper interface holding information (whichever only available) about the loading context - I want this to avoid pollution of the 3 args all over the code

interface nsILoadContextInfo
{
    // Private flag of the loading docshell
    readonly bool private;

    // App ID and inBrowser of the load context
    readonly int appid;
    readonly bool inbrowser;
}

• This class is what more or less stands for current nsICacheSession, that it self is planned to be removed (too generic and too confusing)

class CacheStorage
{
    // delete any existing entries on open (shift-reload)
    const uint32_t CACHE_OPEN_TRUNCATE = 1;

    void asyncOpenURI(nsIURI* aURI, uint32_t aFlags,
                      nsICacheEntryOpenCallback* aCallback);
    void asyncDoomURI(nsIURI* aURI, some_kind_of_callback_if_neccessary);
    void asyncEvictAll(some_kind_of_callback_if_neccessary);

    // async, iterates information about the storage status and entries,
    // similar to current visitor API, but completely async
    void asyncGetInfo(nsICacheStorageInfoCallback* aCallback);
}

• The async callback, similar to current nsICacheListener, but improved for better interaction with appcache

interface nsICacheEntryOpenCallback
{
    // HTTP disk + memory cache lookup result
    // aNew argument:
    //     true - this entry is new and the consumer (channel) is the first
    // one that asked for it, hence it's responsible for data write
    //     false - this entry has data or other channel is currently 
    // in process of filling the entry with response from a server, we can read it
    void onHttpCacheEntryAvailable(nsICacheEntryDescriptor* aEntry, 
                                   bool aNew, nsresult aRv);

    // appcache lookup result ; entry may not be found, but the URI 
    // may be part of a namespace defined by an appcache, hence we must 
    // provide appcache object only to work with
    void onAppCacheEntryAvailable(nsICacheEntryDescriptor* aEntry,
                                  sIApplicationCache* aAppCache, nsresult aRv);
}

• Application cache creation and writing specific stuff, used mainly by nsOfflineCacheUpdate and appcache eviction API used by ctrl-shift-del dialog.

• This is actually a singleton (service)

AppCacheIndex* mozilla::net::AppCacheIndex();

class AppCacheIndex
{
    // The new cache should be created as PINNED, i.e. with less 
    // restrictive eviction politic
    const uint32_t PINNED = 1;

    // creates new version of appcache in the 
    // group (=manifestURI+appId+browserFlag)
    nsIApplicationCache* newAppCache(nsIURI* aManifestURI, uint32_t aFlags,
                                     nsILoadContextInfo* aLoadContextInfo);

    // same as above, just alters the profile directory to store to ; 
    // used for preload capabilities
    nsIApplicationCache* newCustomAppCache(nsIURI* aManifestURI, uint32_t aFlags,
                                           nsIFile* profileDirectory, 
                                           nsILoadContextInfo* aLoadContextInfo);

    // async, look for an existing most recent appcache for the group
    void getAppCache(nsIURI* aManifestURI, 
                     nsILoadContextInfo* aLoadContextInfo, 
                     nsIApplicationCacheCallback*);

    // throw all appcached entries away
    void evictAll(some_kind_of_callback_if_neccessary);
    // only belonging to a host name (i.e. manifestURL.asciiHost = hostname)
    void evictByHostName(string hostname, some_kind_of_callback_if_neccessary);
    // only evict data belonging to an app
    void evictByApp(appid, browserflag, some_kind_of_callback_if_neccessary);

    // async, get iteration info ; here may be more params to limit the output
    void getInfo([optional] nsILoadContextInfo* aLoadContextInfo, 
                 nsIAppCacheIndexInfoCallback* aCallback);
}

nsICacheEntryDescriptor

It will remain mostly unchanged. It may surprise you: "we want this to be async!". No. We want this to be non-blocking. And it depends on implementation that can be much smarter now. Every method of cache descriptor won't ever block, i.e. no potentially I/O blocked locks being acquired on the main thread and no I/O on the main thread directly at all.

Set/GetMetaDataElement will operate on memory only. When having the descriptor in hands, no I/O will be needed to work with metadata.

SetExpirationTime will be the same case. Persistence of the value will be ensured asynchronously.

Few changes to nsICacheEntryDescriptor:

I plan to be able to also store objects (side streams) to the cache entry. So, when opening a stream, there will be one more argument: the stream name. For the raw data, it may be "content-data". A different name will be used for processed image, JS or DOM. This way we can attach any sub-streams to the entry or use the entry to contain data different then just content. Exact API for this is for discussion, since instead of sub-stream we may add sub-blobs rather.

Since the access granted will always be the same (both read and write) there is no need for accessGranted attribute.

Since keeping a cache descriptor doesn't block opening the same entry again by a different consumer, there is no need for Close() method.

We can also discuss API that can in a sync and non-I/O blocked way (no callback) provide info about URI an http channel wants to load like heuristic validity or validity based on the expiration header or whether to send a conditional request. However, that is out of scope of this first API draft discussion.