Site-Specific Preferences

Status

Content Preferences, a prototype extension implementing some of the proposed functionality, has been developed and is being distributed on AMO. The prototype implements the persistent datastore, service, controller, sidebar, and handlers plus sidebar widgets for the text zoom, page style, and character encoding settings.

Two functional requirements have been added to the Firefox 3 product requirements document: a framework for site-specific preferences comprising the datastore, service, and controller (bug 378547) and persistent text zoom (bug 378549).

A meeting to review the API, extensibility, and security of the feature is scheduled for Thursday, May 3 at 11:00AM PDT (6:00PM UTC) at 1-800-707-2533, pin 369, conf 224 (intl: 1-650-215-1282, ext 91, conf 224).

Overview

Motivation

The primary purpose of this feature is to enable users to set preferences for certain core browser and content settings (f.e. text zoom, page style, and character encoding) on a site-specific rather than tab- or page-specific basis and to persist those preferences across page visits and browsing sessions.

A secondary purpose is to provide a site-specific persistent datastore for preferences provided by extensions.

For this feature, the words setting and preference are used as follows:

setting

a runtime configuration option that affects Firefox's appearance or behavior

preference

a user-defined value for a setting

Use Cases

Text Zoom

Every day, Joe visits a web site whose text size is too small for him to read it easily. After going to the site, Joe uses the View > Text Size > Increase command to increase the browser's text zoom value until the text is a reasonable size.

Joe has to do this each time he goes to the site, and when he leaves the site, he has to use the View > Text Size > Decrease command to return the text zoom to its normal value for the other sites he browses.

If the text zoom setting was site-specific, Joe would only have to change it once for the site whose size is too small, and he would never have to reset the setting to its normal value after leaving the site.

Note: John Lilly and Dan Veditz have described personal use cases which aren't satisfied by a site-specific text zoom.

John sometimes increases the text zoom for the browser in order to browse the web (including potentially multiple sites) while sitting farther away from his monitor. Afterwards, he decreases the text zoom back to its normal value.

Dan changes the text zoom each time he switches between two displays (I think his laptop's built-in display and an external monitor).

Although site-specific text zoom doesn't work well in these cases, tab-specific text zoom, which is what we currently have, probably doesn't work well either, because it makes them set (and later reset) the text zoom setting for every tab.

Character Encoding

-> Verify that the character encoding functionality currently in Firefox is accurately described in this section.

Jane lives in a country whose primary language uses a non-Western encoding. She frequently visits web sites whose character encoding is specified incorrectly (or not at all) by the server or page and is not correctly auto-detected by Firefox.

Every time she visits such a site, she has to set the correct encoding for it using the View > Character Encoding menu, but because Firefox only stores the character encoding for pages that she bookmarks, and then only for the specific bookmarked page (not for all pages on that site), Jane finds that she repeatedly has to set the encoding.

If the character encoding setting was site-specific (instead of or in addition to the bookmark-based page-specific mechanism), then Jane would no longer have to set the character encoding each time she visited a site for which it was set incorrectly. Instead, she could set it once and have Firefox reapply that encoding each subsequent time she visited the site.

Requirements

Note: these are suggested requirements for Firefox 3. Those of them that become actual requirements for the release will get added to the Firefox3/Product Requirements Document.

Service

• P1 FR a service that stores and permits chrome code to get, set, remove, and observe changes to preferences for arbitrary settings on a site-specific basis
• P1 FR support for defining sites by host name
• P1 FR support for defining sites by ETLD + 1
• P3 FR a hidden preference that allows users to switch between definitions of a site
• P3 FR ability for extensions to define and register additional site definitions

Controller

• P2 FR a controller for each browser window that simplifies the process of listening for location changes and retrieving preferences for specific sites by defining an API that pref handlers can use for the purpose

Sidebar

• P? FR a sidebar or other UI area that displays and allows users to set and unset the consolidated group of site-specific preferences
• P? FR support for setting the default value for each site-specific setting

Text Zoom Setting

• P1 FR the value of the text zoom setting, which is set by commands in the View > Text Zoom menu or via the scrollwheel, is automatically recorded and restored on a site-specific basis
• P1 FR a mechanism for changing the default value of the setting

Page Style Setting

• P3 FR the value of the page style setting, which is set by commands in the View > Page Style menu, is automatically recorded and restored on a site-specific basis
• P3 FR a mechanism for setting the default value of the setting to either "page default" or "no style"

Character Encoding Setting

• P2 FR the value of the character encoding setting, which is set by commands in the View > Character Encoding menu, is automatically recorded and restored on a site-specific basis

Other Settings?

(are there other settings important and common enough to add to the core?)

• Language preferences, as implemented in Dictionary Switcher extension, for example.

Schedule

• 2006 December 12: Content Preferences (cpref) extension, version 0.1 (announcement) - first prototype implementation, with support for site-specific text zoom
• December 13: cpref, version 0.1.1 - minor bug fix release
• 2007 January 26: cpref, version 0.1.2 - minor bug fix release
• February 13: cpref, version 0.2 (announcement) - second prototype implementation, with API updates and support for scrollwheel-based text zoom changes
• February 14: API/architecture discussion started - blog post mda.firefox article
• March XX: API/security review
• March XX April 24: cpref version 0.3 - third prototype implementation, with page style and character encoding handlers, undo buttons, and datastore/API updates
• May 1: cpref version 0.3.1 - minor bug fix release
• May 3: API, extensibility, security review
• March/April May: land service on trunk
• April May: cpref version 0.4 - fourth prototype implementation?
• April/May: land additional elements of feature (controller, handlers for core prefs, sidebar or other UI) on trunk
• May: release stable implementation of elements not destined for the core as extensions

Also see history of extension on AMO.

Repository

The code for the feature currently lives in the extension. The service, controller, and text zoom handler should land in the Mozilla CVS repository. The sidebar and other handlers should either land in the Mozilla repository or a Mozdev project repository depending on whether it ends up as a core component or remains as an extension.

Design & Implementation

Site-specific preferences in Firefox 3 comprises:

1. a persistent datastore for saving prefs across sessions;
2. an application-wide service with an API for getting and setting prefs;
3. a controller for each browser window with an API for observing pref-related events;
4. a handler for the text zoom setting.

To this core functionality, the extension adds:

1. handlers for the page style and character encoding settings;
2. a sidebar for viewing and editing site-specific prefs.

Persistent Datastore

The persistent datastore is a SQLite database that lives in the profile directory. It gets created automatically if it doesn't exist when the service gets instantiated. Like Places, this feature records the database schema version in the user_version pragma so existing databases can be updated if the schema is modified in future versions of Firefox.

The schema, as specified by a JavaScript data structure in the service's code, is as follows:

 _dbSchema: {
   groupers:   "id           INTEGER PRIMARY KEY, \
                name         TEXT NOT NULL",

   groups:     "id           INTEGER PRIMARY KEY, \
                name         TEXT NOT NULL, \
                grouperID    INTEGER NOT NULL REFERENCES groupers(id)",

   settings:   "id           INTEGER PRIMARY KEY, \
                name         TEXT NOT NULL",

   prefs:      "id           INTEGER PRIMARY KEY, \
                groupID      INTEGER REFERENCES groups(id), \
                settingID    INTEGER NOT NULL REFERENCES settings(id), \
                value        BLOB"
 },

Some notes:

1. The schema uses arbitrary integers to identify all records, even though all tables have another unique column (or, for the prefs table, a unique combination of columns), primarily because in my experience this approach makes future schema changes easier. Integer IDs may additionally provide minor space savings and performance improvements, although they can also make client code more complex and require additional queries for some transactions, which can hurt performance.
2. The schema tracks groupers (components that determine the sites to which URIs belong) as a way of namespacing sites created by different groupers. An earlier version of the schema didn't do this, and I think we should revert this change, making groupers responsible for namespacing sites in cases where collisions are possible, because I haven't found a good use case for this functionality, and it adds significant additional complexity.
3. Pref values are stored in a column with a BLOB declared type because that type has a NONE type affinity, which means SQLite will do no type conversion on values inserted into the column, maximizing round-trip integrity of data passed from consumers through the service to the datastore and back. The actual extent of this integrity across types supported by nsIVariant is yet to be determined and should be investigated via unit tests.
4. The prefs.groupID column is nullable to enable the service to store global preferences (i.e. prefs that apply to all sites, the entire web). We could make consumers store such prefs using the existing nsIPrefService, but that would add significant complexity to consumers, which would have to support two distinct and incompatible interfaces (nsIPrefService doesn't use nsIVariant). Supporting global prefs in the site-specific pref service, on the other hand, is straightforward and adds relatively little complexity to its code.

Service

The service is implemented as a JavaScript XPCOM component with contract ID @mozilla.org/content-pref/service;1 that implements the nsIContentPrefService interface.

Core Methods

The service's core API comprises the following four methods for accessing prefs:

 nsIVariant getPref(in nsIURI uri, in AString name);
 void setPref(in nsIURI uri, in AString name, in nsIVariant value);
 boolean hasPref(in nsIURI uri, in AString name);
 void removePref(in nsIURI uri, in AString name);

All four methods take as input a URI belonging to the site in question and a name for the preference. The service is responsible for extracting the name of the site from the URI. The service defines a site as an effective domain (ETLD+1).

Should the consumer be responsible for extracting site names from URIs so that consumers can set preferences for non-standard site groupings? Or should it be possible for consumers to register a grouper that handles non-standard groupings for certain prefs/URIs?

The name of the preference is an arbitrary string. Preference namespacing is the responsibility of the consumer, as it is with application-wide preferences accessed via nsIPrefBranch.

The setPref method also takes as input an nsIVariant representing the value of the preference, while the getPref method returns such an nsIVariant. Consumers can set the value of a preference to null (nsIVariant::VTYPE_EMPTY), and the service distinguishes between preferences set to null and those that have not been set at all by representing the latter with the undefined value (nsIVariant::VTYPE_VOID).

Consumers can pass null as the value of the uri parameter to access a global preference.

Getting All Prefs

The getPrefs method allows consumers to retrieve all preferences for a given site:

 nsIPropertyBag getPrefs(in nsIURI uri);

The method returns an instance of @mozilla.org/hash-property-bag;1, which implements both the nsIPropertyBag and the nsIPropertyBag2 interfaces. Bag keys are preference names, while bag values are preference values.

Observers

The service also implements methods for adding and removing observers:

 void addObserver(in AString name, in nsIObserver observer, in boolean holdWeak);
 void removeObserver(in AString name, in nsIObserver observer);

When a pref changes, the service notifies observers via the content-pref-changed topic.

What's the right name for this topic?

The notification is similar to the one nsIPrefBranch2 sends with the nsPref:changed topic, but it has two key differences:

• nsIContentPrefService takes only an exact preference name (nsIPrefBranch2 permits a consumer to watch an entire preference branch by passing in a branch prefix like "foo.bar.")
• nsIContentPrefService passes the new value of the pref in notification's subject parameter, which is an nsIPropertyBag containing the following properties:
  • group: the site (or other grouping) to which the preference belongs;
  • name: the name of the preference;
  • oldValue: the old value of the preference;
  • newValue: the new value of the preference.

Should the service support preference branches?

Should the service pass a custom component with a custom interface (like the LiveTitleNotificationSubject component with the nsILiveTitleNotificationSubject interface) instead of a property bag?

Is there any use for the notification's data parameter?

Groupers

Groupers are components that categorize URIs into sites. They implement a simple interface, nsIContentURIGrouper:

   readonly attribute AString name;
   AString group(in nsIURI uri);

The name attribute is an arbitrary identifier for the grouper that the service uses to associate a preference in the database with the grouper responsible for determining its group. The group method returns the name of the group to which a given URI belongs.

Should we just use the contract ID as the arbitrary identifier for the grouper, and how do we retrieve that from an instance of the component? Note: this attribute is only necessary if we want to store groupers in the database.

The default grouper categorizes URIs by effective TLD + one additional hostname segment (i.e. example.com or bbc.co.uk).

Should the default grouper categorize URIs by entire hostname?

The service implements an attribute representing the grouper it uses to extract a site from a URI:

 attribute nsIContentURIGrouper grouper;

The attribute is read/write so that extensions can change the way URIs get grouped by setting this attribute.

Would it make more sense for this to be read-only and for the service to use the grouper specified by an application-wide preference?

Controller

The controller API is implemented as a ContentPrefController JavaScript object in the chrome window context of browser windows.

Shouldn't this be called a "helper" and the "handler" be called a "controller" given the handler's clear "controller" role as mediator between the datastore "model" and the setting "view"?

It implements the following API for pref handlers to register themselves for notification of changes to preferences for the page currently being displayed in the browser:

 void addObserver(in AString name, in nsIObserver observer);
 void removeObserver(in AString name, in nsIObserver observer);

Should the controller hold weak references to observers?

The controller registers itself as a web progress listener and DOMContentLoaded event listener for the tab browser widget and then notifies its observers when the widget's location changes (nsIWebProgressListener::onLocationChange) or a DOMContentLoaded event fires for a browser in the widget.

For location changes, the controller notifies observers with the "content-pref-location-changed" topic, the URI of the new location as the subject, and the value of the site-specific preference for the new location (if any) as the data.

For DOMContentLoaded events, the controller notifies observers with the "content-pref-dom-content-loaded" topic, the document object whose content was loaded as the subject, and the value of the site-specific preference for the document (if any) as the data.

Are these the right topic names for these notifications?

Text Zoom Handler

The text zoom handler is a TextZoomHandler JavaScript object in the chrome window context of browser windows. It registers itself with the controller as a "text.zoom" preference handler and takes over responsibility for handling the View > Text Size commands (both menu items and keyboard shortcuts for them) as well as scrollwheel events that are configured to signify text zoom changes.

When the user changes the text zoom for the current page via any mechanism, the handler applies the change to the text zoom setting and then sets the site-specific "text.zoom" preference for the page to the new value of the setting.

Should we provide an affordance for setting a global text zoom preference?

If we provide a mechanism for setting a global text zoom preference, should that preference be applied in addition a site-specific preference or instead of one? For example, if the global preference is 120% and the site-specific preference for a given site is 130%, should we set the text zoom for the site to 130% or 156% (100 * 1.3 * 1.2)?

When the user browses to a different page (i.e. the location of the tab browser widget changes) the handler applies the site-specific preference, if any, handed to it by the controller. Otherwise, it applies the global preference, if we provide a mechanism for setting this preference. Otherwise it applies the default value (100%).

Cross-Cutting Concerns

API Changes

This feature changes no existing APIs, but it does add several APIs. First, it adds a application-wide "service" API for getting and setting site-specific preferences. Second, it adds a window-specific "controller" API for registering as a pref handler and observing pref-related events. Third, it adds a "sidebar" API for adding controls to a sidebar presenting a consolidated view of preferences for the current site.

Extensibility

Extensibility of this feature is a top priority, since numerous and popular extensions provide users with ways to alter browser settings affecting site appearance and behavior, and the ability to set preferences for those settings on a site-specific basis is a common function of (or feature request for) those extensions.

Examples of extensions that permit site-specific control over browser settings include Flashblock and NoSquint. An experimental version of the HashColouredTabs+ extension supports site-specific functionality via the version 0.2 of the Content Preferences prototype extension.

This feature should be extensible in the following ways:

1. it should be possible for extensions to get and set site-specific preferences via the same APIs that core code uses;
2. it should be possible for extensions to overlay themselves onto the site preferences sidebar so that their controls for setting preferences are visible alongside (or perhaps below) core controls and are as usable as the core controls;
3. extensions should be able to register themselves as pref handlers in the same way that core code registers themselves as pref handlers and receive the same notifications that core code receives.

Should it be possible for extensions to override core pref handlers so that they can implement enhanced/modified functionality to that provided by the core handlers?

Should it be possible for extensions to provide additional ways of defining sites so that users who want a different way of looking at sites can get it and to promote innovation in site definition? F.e. one could say that all Mozilla domains (mozilla.org, mozilla.com) are a single site because they all relate to the Mozilla project, or one could define a site as every domain that uses an SSL certificate issued to the same organization, or every site that uses a certain stylesheet (for browser settings like text zoom and page style that affect style), or perhaps finer-grained controls that define different paths on a certain host as separate sites.

Customization

This feature adds customization controls to a sidebar, which has potential impact on many preferences and customization controls, particularly the Preferences > Content pane and items in the View menu like Text Zoom, Page Style, and Character Encoding.

The potential impact includes:

• items in the View menu might start to affect browser settings on a site-specific basis rather than a tab- or page-specific basis;
• the Content pane might provide an option for viewing and editing preferences for sites;
• other site-specific lists in the Preferences dialog like the list of cookies and extension whitelists might be combined with UI for viewing preferences for sites generally;
• items in the View menu and Content prefpane might be removed or modified in favor of UI in the consolidated preference view.

Performance

There are two notable potential consequences for a performance regression from this functionality.

First, as the user goes from site to site, the controller will get preferences for each site from the service, which queries the datastore for them. There is some cost to doing this, but it's not yet clear what that is. A test for this would be to populate a database with some site-specific preferences and then measure the amount of time it takes for the controller to retrieve a set of preferences for a site.

Second, because settings may be changed on a site-specific basis, switching from one site to another may now prompt costly changes to browser settings.

The effect of this is mitigated by the fact that such changes will only take place when the user has explicitly set preferences for a site, so users who don't use the feature won't be affected, and users who do use the feature probably are willing to experience the regression in exchange for the functionality provided.

But we should keep an eye on that latter assertion, since users may assume that site-specific preferences (at least those built into the core product) do not cause performance regressions and thus not understand why certain sites load more slowly than others. At a minimum, we must make it obvious and easy to undo a site-specific preference. And if settings built into the core product turn out to be costly, we should reconsider making them site-specific or provide additional feedback to the user about the cost (f.e. by providing progress or some other indication that the preference was applied) to inform the user about the cost so she can make an informed decision whether or not to accept it.

Reliability/Stability

In general, this feature should not impact reliability or stability, although it could expose or amplify the effect of bugs in core browser functionality (f.e. the code that adjusts text zoom) to the extent that it increases the frequency by which that functionality is invoked.

Security

The service and the controller are only available to chrome code, so they should have no security impact.

Site-specific preferences can be loosely grouped into "browser" and content categories. Browser prefs are those that affect the browser object (browser, docShell, presShell, etc.) embedded into each tab in Firefox, f.e. the text zoom preference. Content prefs are those that affect the document object for a given web page which is loaded into a browser object, f.e. the page style preference.

Existing prefs of either type should have no additional security impact, although they could expose security holes in core browser functionality to the extent that they increase the frequency by which that functionality is invoked.

New content prefs could have additional security impact, although no such prefs are being proposed yet.

To the extent that this functionality makes it easier to write extensions that add content prefs, however, it may cause the number of such extensions, and thus the danger of security holes in them, to increase.

Privacy

In general, site-specific preferences are not expected to be particularly high value or personal, but they can provide some information about users should be considered private. Preferences are stored persistently in a file in the user's profile, so they are subject to the same security measures we use to protect other personal data in the profile directory. Preference data is transiently stored in the memory of chrome objects to which content does not have access and in chrome controls to which content does not have access.

Global Audience

Since this feature introduces new UI, care should be taken to ensure that it remains accessible in the same way that the current non-site-specific UI is accessible.

L10n requirements apply as they do to other chrome.

Web Compatibility

Users might find that some of their preferences are incompatible with web sites, so it should be easy to remove a preference that results in an incompatibility.

To the extent this feature and its popularity causes web sites to become less insistent on appearing and behaving in a particular way, this feature should have a positive impact on web compatibility.