MozillaWiki

Extension Manager:Sqlite Storage

Revision as of 13:25, 9 July 2008 by Mossop (talk | contribs)

Current Situation

Currently the extension manager caches add-on metadata, blocklist and disabled status, downloads, pending installs and known updates. This is an incomplete list of the highlights:

  • Add-on metadata (majority from install.rdf)
    • id
    • version
    • type
    • hidden
    • appManaged
    • targetApplication entries (modified by update checks)
    • default metadata
    • localized metadata
    • dependencies
    • appDisabled (set based on blocklist, compatibility and security)
    • userDisabled (user controlled)
    • available update (added by update checks)
  • Pending installs (essentially a subset of the above items, added so they appear in the UI before restarting)
  • Downloads (Used to display download progress in the UI)

The UI uses a template to display items, however it does not use the real extensions.rdf datasource. Instead a dynamic datasource is provided that calculates values for certain attributes:

  • Add-on metadata
    • iconURL (may come from the real datasource, may be a direct file uri to a theme's icon)
    • previewImage (a direct file uri to the theme's preview image)
    • aboutURL (hidden if the add-on is disabled)
    • installDate (returns the mtime of the extension's directory)
    • compatible (returns whether the extension is compatible with the application)
    • providesUpdatesSecurely (whether the update path is secure)
    • satisfiesDependencies (whether all dependencies are available)
    • blocklisted (asks the blocklist service if the add-on is blocklisted)
    • opType (any pending operation for the add-on)
    • appManaged (can only return true if the extension is installed by the app)
    • hidden (can only return true if the extension is in a restricted install location)
    • locked (can only return true if the extension is in a restricted install location)
    • textual data (retrieved from properties, the localized area or the default area as appropriate)
    • isDisabled (based on appDisabled and userDisabled)
    • updateable (checks preferences and permissions to see if this add-on can be updated)
  • Downloads
    • state (current install state)
    • progress (current download progress)

Every pane of the UI is currently built using an rdf based template. Switching panes swaps in a new template in a bunch of complex code. The Get Add-ons pane and Plugins pane are made available through the use of two in-memory datasources added to the templates composite datasource.

Much of the UI is dependant on the dynamically generated properties listed above to display the text in the correct locales and show state correctly.

Proposed Changes

Rather than being customised for displaying appropriately in the UI the database should instead be properly normalised with whatever information we need now and may need in the future. It should not include transient data that is only needed before the app restarts to keep the UI correct.

This means that information about add-ons waiting to be installed and downloaded will not be in the database. They instead should be held in in-memory data structures. It follows that the UI then will not be entirely generated by templates. Instead the Get Add-ons, Plugins, Updates and Installation panes will be programmatically handled. For the first two that should be relatively simple. The Updates and Installation panes will have to depend on notifications from the extension manager, much of which are already available on the install listener interface.

It is also likely not going to be possible to use templating for the extension lists. Firstly the sqlite templates do not respond to changes in the database, something which we could handle in a custom way for the add-ons manager but is unlikely to be implemented as standard. Secondly the large number of computed properties that the UI uses to display things correctly makes using a regular database query difficult. Many could potentially be handled with custom sqlite functions or custom code in the UI itself but I believe the best course of action is instead to effectively hide the database from the UI and use a proper API to get data on the add-ons, either extending or replacing nsIUpdateItem.

Database Schema

The database is made up of 6 tables:

  • addon (addonid, installLocation, version, ...., userDisabled)
  • targetApplication (addonid, version, application, maxVersion, minVersion)
  • dependencies (addonid, installLocation, requirement, minVersion, maxVersion)
  • localized (addonid, installLocation, locale, id)
  • localizedData (id, name, ...)
  • update (addonid, version, updateHash, updateURL)

There are some key points suggested in this design.

The addon table contains all known add-ons in all install locations. This is different from the current situation where we only cache data for the add-ons in the install location with the highest priority. This makes various future behaviours possible including using the newest version of an add-on regardless of installLocation, using any of the same add-on based on compatibility/blocklisting etc. We can create a view in the database that returns only the currently active add-ons for the common query cases.

The targetApplication table is keyed on addonid and version rather than being a foreign key into the addon table. It should keep targetApplication information for all installLocations, since an add-on might be installed with the same version in different installLocations there is no need to duplicate that information in the database. Likewise known updates have their targetApplication info here. We could potentially think about caching this information almost permanently such that future installs of a previously installed add-on may not need another compatibility update check.

Might want to do the same for the dependencies, however right now they come only out of the install.rdf and are never updated by remote data.

Still need to consider what generated properties should be held in the database. We could go with just appDisabled as currently, or we could store blocklisted, compatible and updatesSecurely separately.

API

The following interface is what the UI would use to display add-on information for both installed, pending and downloading add-ons. These are just rough sketches, many blanks. 

interface nsIAddon : nsISupports
{
  readonly attribute AString  id;
  readonly attribute AString  version;
  readonly attribute AString  name;

  //... other metadata ...

  readonly attribute boolean  isDisabled;
  readonly attribute boolean  isBlocklisted;

  //... other programatically generated items ...
}

interface nsIAddonDownload : nsIAddon
{
  const unsigned long STATE_DOWNLOADING = 0;
  // Whatever states we need to represent different points of the install
  const unsigned long STATE_PENDING_RESTART = 42;

  readonly attribute int  state
  readonly attribute long progress;
  readonly attribute long maxProgress;
}

Installed items would be initialised from the database. Requests for things like localized data can be request additional data from the database after the fact.

For a pending install/downloading add-on the nsIAddon interface is mostly unfilled, things like the version are unavailable until after the download is complete and the install.rdf read. If they are updates then the id and name and some other information can be prefilled from the original item. This would enable us to reject updates that turn out to not be the id and version expected.

We could consider moving the add-on operations (uninstall, disable, enable etc.) onto the nsIAddon interface.

The UI (and any other callers) should be able to request the current active (meaning visible, not necessarily enabled) add-ons list, the current available update list and the current installs list from the extension manager. Maybe also the full list of add-ons in all install locations.

Major Tasks

  • API Implementation
  • UI Work
  • Replace XPInstall based downloader?
Retrieved from "https://wiki.mozilla.org/index.php?title=Extension_Manager:Sqlite_Storage&oldid=99513"