|
|
| Line 1: |
Line 1: |
| =Proposal: A Plan for Profile Sharing=
| | {{Warning|The proposal previously on this page is no longer relevant and has been removed. See the page history if prior plans are interesting to you. |
|
| |
|
| [mailto:darin@meer.net Darin Fisher]
| | --Benjamin Smedberg |
| | | }} |
| ==Overview==
| |
| | |
| The purpose of this document is to outline a plan for enabling the sharing
| |
| of files in the user [[profile]] by multiple mozilla-based applications running
| |
| on the same system. Profile sharing has two main goals:
| |
| | |
| * To enable embedders to write Mozilla-based applications without having to consider profile details. This is something embedders of Trident (MSHTML) take for granted.
| |
| * To allow Mozilla's applications to share specific preferences, certificates, and possibly other profile resources. For example, the Forumzilla extension to Thunderbird might benefit from being able to access the cookies stored in a Firefox user's profile.
| |
| | |
| Long-term, other goals include making it possible to launch two instances of
| |
| Firefox and have each instance use the same profile. This requires Firefox
| |
| to share all of its profile contents, which will be costly to implement.
| |
| It's unclear whether Thunderbird would ever wish to support full profile
| |
| sharing with other instances of the Thunderbird application.
| |
| | |
| A separate, long-term goal is to enable profile sharing across hosts. This is a
| |
| much more complex problem because it involves coordination between different
| |
| machines. This proposal does not address the problem of sharing profile data
| |
| across multiple hosts.
| |
| | |
| ==Requirements==
| |
|
| |
| The major requirements for profile sharing include:
| |
| | |
| * Enable sharing of profile data between multiple Mozilla-based applications running on the same system.
| |
| * Store shared profile data in a directory that is independent of application specific profile data. This directory should only contain files that are shared by applications participating in this profile sharing scheme.
| |
| * An application should be able to choose the location of the shared profile directory. This is a requirement in order for profile sharing to be ''per profile''.
| |
| * Access to the shared profile directory must be gated exclusively by this profile sharing scheme in order to ensure the integrity of shared profile data.
| |
| | |
| * An application should be able to choose the profile data it shares. Moreover, in the case of shared preferences, an application should be able to choose the set of preferences it shares with the platform. This level of granularity may be appropriate for other shared profile data as well.
| |
| * Profile sharing should not mandate the creation of new file formats for the shared profile data.
| |
| * The profile sharing scheme must be ''frozen'' to a degree sufficient to ensure compatibility with future applications.
| |
| | |
| ==Files to be Shared==
| |
| | |
| The existing Mozilla-based applications store a number of different files
| |
| in their profiles. Many of these files are application specific. Some,
| |
| however, can be generalized to the web-platform, and therefore are likely of
| |
| interest to other applications built on the Mozilla platform. These are
| |
| the files that we are most interested in sharing.
| |
| | |
| ===Cache===
| |
| The web cache is by far the largest collection of files stored in the user
| |
| profile. Because of its large footprint on disk, sharing the disk cache is
| |
| advantageous since it avoids duplicating a web cache for each Mozilla-based
| |
| application. NOTE: Thunderbird does not enable the disk cache for mail,
| |
| but extensions like Forumzilla might benefit from Thunderbird having access
| |
| to a shared disk cache. Moreover: Seamonkey mail uses the disk cache for
| |
| HTTP content loaded into the mail frame. This gives Seamonkey mail an
| |
| advantage over Thunderbird when a user revisits a piece of HTML mail.
| |
| | |
| ===Certificates===
| |
| It is common for corporations to provide certificates to employees for use
| |
| to sign and seal emails as well as to access internal websites. Therefore
| |
| there is need for Firefox and Thunderbird to share certificates to avoid
| |
| having to manage separate certificate databases.
| |
| | |
| sharing may even be seen as a barrier by some enterprises upgrading to the
| |
| new apps from Seamonkey or maybe IE+OE (IE and OE access certificates from a registry entry managed by the operating system).
| |
| | |
| The NSS team is currently working on prototypes of shared databases using SQLite.
| |
| | |
| ===Cookies===
| |
| Cookies are an important part of the web-platform. By supporting shared
| |
| cookies, we enable other applications to inherit user identity when they
| |
| access web sites. This could be used by Forumzilla, for example, to make
| |
| it so that the view of slashdot.org is properly configured per the
| |
| user's preferences for that site as configured using Firefox.
| |
| | |
| ===Preferences===
| |
| | |
| Sharing preferences is perhaps the main use case for profile sharing. The
| |
| standard example is a user's proxy settings which all Mozilla-based
| |
| applications need in order to connect to the internet. Allowing users to
| |
| avoid having to re-enter these settings for each application would be very
| |
| beneficial. Moreover, when the user needs to make changes to the proxy
| |
| settings, the user will appreciate the fact that those changes will be
| |
| reflected to all Mozilla-based applications. An independent options
| |
| dialog for shared preferences might make sense, but this is probably a
| |
| detail best left for another proposal ;-)
| |
| | |
| ===What About Other Files?===
| |
| | |
| Most of the other files in the profile are application specific. For example,
| |
| bookmarks, history, address book, etc. are all very specific to individual
| |
| applications. Some files, like the fastload file and localstore.rdf are also
| |
| application specific since their contents will depend on the application that
| |
| generated them.
| |
| | |
| In the future, it may be desirable to share some of these files with other
| |
| similar applications or with other instances of the same application. But,
| |
| again the primary focus of profile sharing is to support the Mozilla toolkit.
| |
| And, that means starting with sharing the web-platform specific profile data.
| |
| | |
| <div style="border: 2px solid green; padding: 0.5em;">
| |
| '''Note from bsmedberg:'''
| |
| | |
| to reduce the app-author-barrier to shared profiles, I intend to provide a RDF
| |
| datasource that does sharing. This will probably not be my fancy database-driven
| |
| datasource, but something that uses a simple N-triples format and keeps all the
| |
| data in memory.
| |
| | |
| The IPC interfaces should be frozen for 2.0
| |
| | |
| ''darin: Agreed.''
| |
| </div>
| |
| | |
| | |
| <div style="border: 2px solid green; padding: 0.5em;">
| |
| '''Note from colmsmyth:'''
| |
| | |
| Mozilla's role as a platform would be enhanced if non-Mozilla applications could also access it's data. The key data elements for non-Mozilla apps are: cache, bookmarks, history.
| |
| | |
| This suggests two requirements:
| |
| | |
| 1. use standard data formats (HTML, CSV, XML or binary formats that have bindings in many languages)
| |
| | |
| 2. use textual protocols to allow other apps to synchronize with the IPC file-locking daemon.
| |
| | |
| There is a potential 3rd requirement: minimize downtime (and increase reliability) for non-random-access files by writing them first to a temporary file and then moving them over the old file when complete. This allows simple applications that don't need (or can't use) the file-locking/synchronization mechanism to get access to data 99% of the time, e.g. the bookmarks.
| |
| </div>
| |
| | |
| ==A Scheme for Profile Sharing==
| |
| | |
| ===The IPC Daemon and the Transaction Manager Extension===
| |
| | |
| The [http://lxr.mozilla.org/mozilla/source/ipc/ipcd/ IPC daemon]
| |
| was developed for the purpose of being used to enable profile sharing. It
| |
| serves as a central process that clients can connect to in order to
| |
| communicate with one another or to communicate with code running in the
| |
| IPC daemon itself. The IPC daemon is extensible via a simple "plug-in"
| |
| [http://lxr.mozilla.org/mozilla/source/ipc/ipcd/daemon/public/ipcModule.h API]
| |
| | |
| The main benefit of the IPC daemon is that it can be used to synchronize
| |
| access to shared profile data. It provides basic locking functionality
| |
| that does not depend on filesystem or operating system locking primitives
| |
| (that are known to be buggy in some cases). Moreover, via the
| |
| [http://lxr.mozilla.org/mozilla/source/ipc/ipcd/extensions/transmngr/transaction manager]
| |
| IPC daemon extension, the IPC daemon can be used to actively synchronize in-memory and in-process copies of databases.
| |
|
| |
| Details on the transaction manager design are available
| |
| [http://www.mozilla.org/projects/embedding/profileSharing/transaction-manager-tech-spec.html here].
| |
| The transaction manager is a good solution for the problem of sharing preferences
| |
| and cookies since it minimally perturbs the backends for those modules. The
| |
| transaction manager API provides synchronized reading and writing as well as
| |
| notifications about changes to the databases.
| |
|
| |
| A client of the transaction manager simply attaches itself to a named queue,
| |
| waits for permission to get read access to the corresponding files in the
| |
| profile, reads those files, and then listens for change notifications on
| |
| that queue. When the process makes changes to its copy of the database, it
| |
| posts those changes to the queue so other processes can update their in-memory
| |
| copies. When a process wishes to write its in-memory copy to disk, it again
| |
| requests permission from the transaction manage and upon receiving permission
| |
| it writes the files out to disk. The transaction manager makes sure that
| |
| a client requesting write access receives all changes to the database before
| |
| granting it write access. In that way the transaction manager ensures
| |
| integrity of the serialized databases and protects against data loss.
| |
| Inotherwords, an application could not inadvertantly stomp over the data
| |
| set by another application unless it ignored notifications from the
| |
| transaction manager.
| |
| | |
| The transaction manager's protocol is very simple and is not at all profile
| |
| specific. It can even be used to synchronize memory-only databases that are
| |
| not persisted to the profile.
| |
| | |
| ===Supporting a Shared Disk Cache===
| |
| | |
| The disk cache cannot be shared easily or efficiently using the transaction
| |
| manager. We really have two choices when it comes to sharing the disk cache.
| |
| We could develop an IPC daemon extension that manages the cache's data structure,
| |
| with a protocol that clients use to gain read or write access to particular
| |
| sections of files on disk (recall that some disk cache files are stored in
| |
| the block files).
| |
|
| |
| Another solution for sharing the disk cache involves utilizing the IPC
| |
| daemon only to manage reader/writer locks. Then each client process would
| |
| acquire the appropriate lock(s) and proceed to access the data structures
| |
| of the cache directly. This probably involves mapping and unmapping
| |
| the _CACHE_MAP_ and _CACHE_00X_ files frequently. It remains to be seen if
| |
| this would be prohibitively costly in terms of performance. This solution
| |
| also locks down more of the cache format since future applications will
| |
| need to store files in the cache in a format that older applications can
| |
| still understand.
| |
| | |
| The IPC daemon module solution doesn't have as many constraints since we can
| |
| hide many of the details of the cache data structure behind an IPC protocol.
| |
| This may be enough of a reason to go with this kind of solution.
| |
| | |
| ===Supporting Shared NSS Databases===
| |
| | |
| The NSS .db files can be shared by creating a library with the name
| |
| <code>SHLIB_PREFIX"rdb."SHLIB_SUFFIX</code>. NSS will open this library
| |
| and look for a symbol named <code>rdbopen</code>. It should have the
| |
| following signature:
| |
| | |
| DB *rdbopen(const char *appName, const char *prefix, const char *type, int flags);
| |
| | |
| The only description of this API that I could find (outside of the source itself) is
| |
| [http://www.mail-archive.com/mozilla-crypto@mozilla.org/msg05139.html here].
| |
| | |
| The current thought is to use SQLite. This removes the need for any central server, or the use of IPC to talk to it. The NSS team has a prototype.
| |
| | |
| ==Next Steps==
| |
| | |
| * Agree on general plan for profile sharing</li>
| |
| * Agree on location for shared profile data. Maybe something like this:<br>~/'''.mozilla-common'''/''<profile-name>''/ <br>~/.firefox/''<profile-name>''/ <br>~/.thunderbird/<''<profile-name>''/
| |
| * Bring shared preferences and cookies online first (target 1.8 easily).</li>
| |
| * Bring shared certificates and cache online next (post 1.8 likely).</li>
| |
| * What about ensuring that the cache is stored locally? Need system API help.</li>
| |
| | |
| <div style="border: 2px solid green; padding: 0.5em;">
| |
| '''Note from bsmedberg:'''
| |
| | |
| We need to specify how profile locking and profile sharing interact. In particular,
| |
| can the profile lock files contain "locator" information to other apps that are already running which need to share profile information?
| |
| | |
| Example:
| |
| * One instance of firefox begins running. There is no profile.lock, so it writes an "IPC locator" into that file: "socket: ./sharing.socket" or something like that.
| |
| * A second instance of firefox is started. It sees that profile.lock already exists, so it reads the file, and tries to use the IPC mechanism to talk to "socket: ./sharing.socket".
| |
| | |
| By adding new "locator" types in the future we could support special situations like a central profile server, or multiple machines sharing a profile through TCP sockets.
| |
|
| |
| ''darin: This sounds like an excellent idea to me.''
| |
| </div>
| |
| | |
| ----
| |
| What about using DBus at least on *nix platforms for ipc? Note that a win32 port also exists (though not maintained) and the license is similar to X/BSD.
| |
Warning: The proposal previously on this page is no longer relevant and has been removed. See the page history if prior plans are interesting to you.
Warning: The proposal previously on this page is no longer relevant and has been removed. See the page history if prior plans are interesting to you.