MozillaWiki

LocalStorage: Difference between revisions

Page Discussion

LocalStorage (view source)

Revision as of 09:47, 25 November 2006

1,487 bytes removed ,  25 November 2006
m
Reverted edit of Wpiocs, changed back to last version by Udayan
m (Reverted edit of Wpiocs, changed back to last version by Udayan)
 
(10 intermediate revisions by 4 users not shown)
Line 82: Line 82:
the api.
the api.


:roc: I think the right thing to do here is to make an offline application just a set of pinned pages in the cache. Then you can write a single set of pages that can function online or offline. HTTP caching instructions take care of application updating. We could automatically allow a site to pin N% of your disk space, if your free disk space is >M% of total; if they try to go over that, the user has to approve it.
:[[User:Roc|Roc]] I think the right thing to do here is to make an offline application just a set of pinned pages in the cache. Then you can write a single set of pages that can function online or offline. HTTP caching instructions take care of application updating. We could automatically allow a site to pin N% of your disk space, if your free disk space is >M% of total; if they try to go over that, the user has to approve it.


:Instead of providing a manifest, a more Web-friendly approach might be to crawl the page. Basically, when you select a page for offline viewing (which could be as simple as just bookmarking it), we pin it and also download and pin any of the resources we would if we were doing Save As --- images, stylesheets, scripts, IFRAMEs. In addition we could provide a new 'rel' value for <a> links that says "crawl this link for offline viewing". This would be really easy for Web developers to use and maintain and not too difficult to implement, I suspect. Application crawling and download could be done in the background.
:Instead of providing a manifest, a more Web-friendly approach might be to crawl the page. Basically, when you select a page for offline viewing (which could be as simple as just bookmarking it), we pin it and also download and pin any of the resources we would if we were doing Save As --- images, stylesheets, scripts, IFRAMEs. In addition we could provide a new 'rel' value for <a> links that says "crawl this link for offline viewing". This would be really easy for Web developers to use and maintain and not too difficult to implement, I suspect. Application crawling and download could be done in the background.
::[[User:Blizzard|Blizzard]] Pinning pages in the cache sounds like a great idea as a way to implement this, but I don't think using a heuristic that eventually asks the user to add storage is probably the right way to go.  Users don't often know how big their cache is nor how much space they have left on their hard drive.  I did have a hook for the actual application to throw the space dialog, but that's a little different than the browser throwing that dialog.  That requires the app knowing how much space is allocated to it and how much space it is using.  The nice thing about that is the app can avoid ever throwing that dialog by expiring data instead of requiring the user to just add more data.
::I also think that it's important that we keep the maifest separate from the pages themselves for a few reasons:
::<ol><li>It's hard to know where an offline application "starts."  That is, if you're on page X does that app start on page X or on another page entirely?  The manifest is the logical starting point for the "bookmark" and can also contain "start" information.  i.e. which cached page should be loaded when you are offline and the bookmark is loaded? <li>When do you expire data?  If you update your offline app how do you tell when certain pages are unused?  If there's no single location for all of the pages to be found upgrades can get a lot more challenging and there's a good chance that we'll end up with some unpredictable heuristic. <li>Raw crawling misses a lot of data.  Urls and locations that are accessed via javascript and building form data on the fly can create locations that won't show up in a crawl.  <li>It would be a lot of work to maintain all of the possible links in all of your pages.  I think it would be a lot easier just to have a simple text-based file that contains a list of resources for all of the pages.  And a simple <link> page that links to that manifest.  Everything is explicit and well-understood in that case.</ol>
:::[[User:udayan|udayan]]One more set of problems arises with deployment of applications in geogaphically remote areas. These areas have connectivity on a very limited basis ( not 24 X 7 ), connectivity is of poor quality ( speed and reliability ) and connectivity is expensive. Also the remoteness means applications deployed in such areas need to be remotely managable, as physical access is not always feasible. This creates a need for the applications to be : (a) Remote depoyable and managable and (b) Applications must work in offline mode ( forms data based data entry, locally cached, submitted on detection of connectivity, upgrades to application and things like form templates to be cownloaded on detection of connectivity ).
:::For this we need the ability in the browser to be able to explicitly tag content  as "offline capable", where "offline capable" would imply a set of things and not just being able to "pin" content in a cache.


===Storage===
===Storage===
Line 148: Line 159:
mail.yahoo.com and address.yahoo.com.)
mail.yahoo.com and address.yahoo.com.)


[[User:Roc|Roc]] I think that trying to provide structured storage on the client is hopeless for the same reasons that trying to provide structured storage in a filesystem is hopeless. Whatever model we choose will never be the right model for the majority of applications. Furthermore, it diverges from today's Web programming models. I think we should just provide a simple filesystem API --- essentially, a persistent hashmap from string keys to string values --- slightly enhanced cookies. Remember that developers will want to at least do all the things they do with cookies today, including obfuscation, encryption, and on our side, quota management. People can build libraries on top of this if they want to. They can build libraries for indexing, LIFO cache management and so on.
:[[User:Roc|Roc]] I think that trying to provide structured storage on the client is hopeless for the same reasons that trying to provide structured storage in a filesystem is hopeless. Whatever model we choose will never be the right model for the majority of applications. Furthermore, it diverges from today's Web programming models. I think we should just provide a simple filesystem API --- essentially, a persistent hashmap from string keys to string values --- slightly enhanced cookies. Remember that developers will want to at least do all the things they do with cookies today, including obfuscation, encryption, and on our side, quota management. People can build libraries on top of this if they want to. They can build libraries for indexing, LIFO cache management and so on.


[[User:VladVukicevic|VladVukicevic]] 18:00, 7 Jul 2005 (PDT): That's my thinking as well -- something like "supercookies" instead of fully queryable structured storage.  At the very least, they'd be much simpler to start to use.  We can also provide helpers for serializing a JS Object to/from the value format, which should take care of most consumers' needs.  I wouldn't try to just use the cookies api though, since unlike any local-cookies (brownies?) these would never be transmitted to a remote server as part of a request.  I think a simple hash map would fit well with the AJAX model as well.
::[[User:VladVukicevic|VladVukicevic]] 18:00, 7 Jul 2005 (PDT): That's my thinking as well -- something like "supercookies" instead of fully queryable structured storage.  At the very least, they'd be much simpler to start to use.  We can also provide helpers for serializing a JS Object to/from the value format, which should take care of most consumers' needs.  I wouldn't try to just use the cookies api though, since unlike any local-cookies (brownies?) these would never be transmitted to a remote server as part of a request.  I think a simple hash map would fit well with the AJAX model as well.


[[User:Roc|Roc]] I agree so I deleted that part of my comment.
:::[[User:Roc|Roc]] I agree so I deleted that part of my comment.


===UI===
===UI===
Line 194: Line 205:
users.
users.


:roc: I really like the bookmarks UI idea. When we're in offline mode bookmarks for pages that are not available should be disabled with a tooltip explaining why. Although maybe everything that gets bookmarked should be downloaded for offline access anyway.
:[[User:Roc|Roc]] I really like the bookmarks UI idea. When we're in offline mode bookmarks for pages that are not available should be disabled with a tooltip explaining why. Although maybe everything that gets bookmarked should be downloaded for offline access anyway.


===APIs===
===APIs===


The apis have a few easy rules:
====The apis have a few easy rules====


1. Think about the use cases.
1. Think about the use cases.
Line 207: Line 218:
will) add those later through their own libraries.
will) add those later through their own libraries.


Known use cases:
====Known use cases====


1. Storage <-> XML-RPC bridge.  It's clear that people will want to
1. Storage <-> XML-RPC bridge.  It's clear that people will want to
Line 223: Line 234:
that XML document.
that XML document.


Deployment APIs
====Deployment APIs====
 
  window.supportsCapability(string aCapability);
 
    Check to see if the api supports aCapability.
 
  Storage-related APIs
 
  Creating and querying storage types.
 
  window.storage.create(string aName, dict aDescription)
 
    aName is the name of the storage.  The description is an array of
    dictionaries that describes the structure of the storage.  For
    example:
 
    var desc = [ { name: "col1", type: "int", key: true },
                  { name: "col2", type: "string" },
                  { name: "col3", type: "cache" ]
    window.storage.create("foo", desc);
 
    The key: entry in the dictionary notes that this column will hold
    primary keys.
 
    Supported types include:
 
      int - An integer of size XXX.
 
      double - A floating point number of size XXX.
 
      string - A string in unicode format (UCS-2?  Should be at least
        UTF-8.)
 
      blob - A binary chunk of data.  Each entry can have its own
      mime type.  This mime type can be set by the application or is
      automatically set when the blob is filled from the network.
 
      cache - this special type is actually a double saying that the
      storage module should manage caching in this particular storage
      module.  If the application starts to fill to the maximum
      storage capacity, entries with lower values will be deleted
      before entries with higher values.  However, entries with
      negative values will never be deleted.  Applications can not
      predict when entries will be removed from the storage unit.
 
      Note: add a default mime type above to a column descriptor?
 
  window.storage.delete(string aName);
 
    Deletes a particular piece of storage by name.
 
  window.storage.getNames()
 
    Returns an array of strings with the names of all of the storage
    units defined.
 
  window.storage.getDesc(string aName);
 
    Returns an array of dictionaries describing the storage unit
    defined.  See window.storage.create() above.
 
  window.storage.addDataByName(string aName, dictionary aValues);
 
    This will add a new row to aName storage and the column names
    will be specified as the keys in aValues.
 
  window.storage.addData(string aName, array aValues);
 
    This will add a new row using the offset in the aValues array as
    the offset in the aName storage.
 
  window.storage.deleteDataByName(string aName, dictionary aValues);
 
    This will delete any rows in aName storage that match the
    key/value pairs in aValues.
 
  window.storage.deleteData(string aName, array aValues);
 
    This will delete any rows in aName storage that match the values
    specified in aValues.
 
  window.storage.getRowsByName(string aName, dictionary aValues);
 
    This will return any rows that match the specified values.  Note
    that there's no equiv. getRows() call.  This is because you have
    to be able to leave out rows for which you don't know the value
    and 'null' is the only way to pass that information in an array,
    and you should still be able to match against the 'null' value in
    the database.
 
  window.storage.search(string aName, searchDesc aSearchDescriptor);
 
    This is the powerful search function.  You will need to build a
    query object that allows you to search.
 
    The search descriptor is an array of dictionaries that includes
    arbitrarily complex search functions in reverse polish notion.
    For example:
 
    var foo = [ {and: [ {name: {equals: "bob"}}, {addr: {equals: "bar"}} ] }
                {or:  [ {status: {equals: "on"}} ] } ];
 
    This is the same as
 
    if ((name == "bob" && addr == "bar") || status == "on")
 
    Note: Ugly!  Can we just pass JS evals and then get the decision
    tree?  Also, string matching using substrings and regex?  How
    about user-defined functions?
 
  window.storage.getByName(string aName, object key)
 
    This assumes that you set the key attribute when you created the
    storage.  It will return the row that matches the key.
 
  window.storage.getByOffset(string aName, number offset)
 
    This will get the n-th offset item in the storage.
 
  window.storage.getCount(string aName)
 
    This will get the number of rows in aName storage.
 
  window.storage.iterate(string aName, func callback)
 
    This will iterate over every row in aName storage calling your
    callback for each one.


  window.storage.getUsageAllowed();
A few different APIs have been proposed:
  window.storage.getUsage(string aName);


    getUsageAllowed() returns the number of bytes that the app can
1. A database api that's based on a classic relational databaes model.  That is, tables and rows.
    use for storage.  If you pass 'null' to getUsage it will return
    the total number of bytes in use.  getUsage with a storage name
    will give the number of bytes in use for a specific piece of
    storage.


  Notes:
2. A simple dictionary system.  A single-level lookup based on a simple string key.


    add .sort to sort a table?
3. No API.  Just the ability to just cache local pages.


    How about updates?
====Functional Coverage====


  Networking APIs
Required APIs probably include:


    How to specificy which specific row to download?  Maybe with a
1. A way to access local storage and read information out of the database.
    add()?  Should be something like:


    window.storage.addFromNetwork(aName, rowdesc, url);
2. Assuming that we want to go with a system which allows a huge amount of storage, a way to access and query (?) that data.


    Same problem with setting the mime type for a particular row in a
3. A way to handle page transitions from one page to another.  This would probably be a client-side equiv to form handling on the server side.
    particular column - need a way to specify that.
Mvl
Confirmed users
503

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/9195...43113"