From MozillaWiki
Jump to: navigation, search

Evolution of future work on Places, the SQLite-driven Bookmarks and History system.


Cherry-picked list of Places related fixes. Minor changes or cleanups are excluded.

Firefox 5

Bugs Type Description
bug 660036 IO Workarounds a bug in Android < 2.3, that causes places.sqlite-wal to grows indefinitely on first Sync.
bug 649211 SUPPORT Places Maintenance output was broken, not giving any feedback to users.
bug 636924 UI Makes copied livemarks not appear as common folders in sidebar.
bug 599641 PERF Runs ANALYZE after each expiration step. SQLite query planner uses this data.

Firefox 6

Bugs Type Description
bug 555474 UI Avoids bookmark tooltips interrupting Drag'n'Drop actions.
bug 620627 l10n Fixes localization of grouped history views
bug 652379 API Fixes results for a Places query pointing to a wrong folder id.
bug 630240 PERF Avoid full refreshes of history results to avoid locking the main-thread too long.
bug 359809 UI Fixes keyword searches encoding in location bar
bug 639105 UI History menu was ignoring some added visits
bug 656188 PERF Caches last fetched bookmarks info to speed up requests.
bug 630622 UI Updates the Library design to the browser Aero style.
bug 629620 PERF Copied bookmarks won't inherit anymore all annotations, since they are new entitities.
bug 631374 UI Fixes tags going out of view on toggle in bookmarks dialog.
bug 658135 PERF SQlite should not create transactions for a batch of read-only statements (like Autocomplete)
bug 630225 API Frecency can now be used to sort queries.
bug 524091 API Microsummaries have been removed from the core product.
bug 633274 API Bookmarks observers now pass out more information, included GUIDs.
bug 625325
bug 625322
UI Some menuitems in Bookmarks and History menu have been repositioned.
bug 663344 SUPPORT Fixes a case where weekly maintenance may cause history loss.

Firefox 7

Bugs Type Description
bug 661135 PERF Simplifies some autocomplete queries.
bug 633266
bug 662806
API Pass GUID to history observers. Helps Sync.
bug 661445
bug 663269
bug 663104
PERF Various fixes to livemarks code.
bug 611568 UI File/Import has been moved to the Library window.
bug 660109 API Allow to distinguish expiration removals, to avoid history dataloss on Sync.
bug 416459 UI Cut actions are now properly implemented. Fixes UI, saves some IO and has better Sync performances.
bug 578268 CLEANUP MOZ_MORKREADER is no more in Firefox.
bug 593566 SUPPORT Fixed a bug that caused broken bookmarks export.
bug 645963 UI Text-only toolbar view now correctly shows the chevron in pinstripe.
bug 655270
bug 655273
UI history.pushstate causes missing title or icon

Firefox 8

Bugs Type Description Status
bug 564900
bug 591289
bug 669905
UI Integrate downloads in the Library IN PROGRESS
bug 671001
Perf Places telemetry IN PROGRESS
bug 674210
Perf New expiration heuristics (memshrink) IN PROGRESS
bug 658305
Perf Limit journal for mobile IN PROGRESS
bug 669040
Cleanup Remove Mork code IN PROGRESS
bug 636603
API mozIAsyncHistory fixes IN PROGRESS


Bugs Description
Schema changes
Async bookmarks API
Async annotations API
in-content UI
Finish UpdatePlaces
Audit Mobile API usage
Limit WAL Size
History backup
Memory usage reduction

Schema Changes

  • GOAL: Database filesize reduction, performances, healthier management
  • ETA: Firefox 7
  • PRECONDITIONS: History backup

Create history backups

This is a precondition to stop caring about corruptions, downgrades, compatibility.

  • PROPOSAL: per-host backups and expiration, csv-append-like.
  • PRECONDITIONS out-of-mainthread file access? (append and remove)
  • BUG: TBD

Coalesce titles across history and bookmarks

Strings are the largest database offender, we never use both titles and all the queries have to coalesce titles. Users want us to update titles automatically when they change if they didn't manually set a different bookmark title.

  • PROPOSAL: keep only one title in moz_places, track whether it's expirable.
  • BUG: TBD

Tags out of moz_bookmarks

Tags are actually folders, this is unefficient and takes a lot of filesize for nothing, searches are slow as well since subqueries suffer from missing indices.

  • PROPOSAL: tags in a separate relational table.
  • BUG: TBD

Livemarks out of moz_bookmarks

Livemarks are actually folders of temporary bookmarks, this is unefficient and causes unneeded database fragmentation.

  • PROPOSAL: redesign from scratch.
  • BUG: TBD

Remove dynamic containers support

They never worked, nuke them.

  • PROPOSAL: Remove all related code, redesign from scratch.
  • BUG: TBD

Kill moz_bookmarks table

Once titles are flattened to moz_places and dynamic containers killed, most of bookmarks data consists of hierarchical information, then just make them so.

  • PROPOSAL: Flatten isBookmark tracking in moz_places, have tags and folders just be hierarchical descriptions of these entries.
  • PRECONDITIONS: flattened bookmarks in moz_places, tags out of bookmarks.
  • BUG: TBD

Re-evaluate history tracking information

We track each single step in the history visit chain, but these data are not really useful as of today, the UI mostly cares about the destination page, not about the full redirects chain, and walking it up is slow and unefficient. Sessions tracking has been removed, and adds useless data. Sync also missed some of these data.

  • PROPOSAL: Simplify, remove redundant information, track only what matters.
  • BUG: TBD

Keywords are unefficient

Currently these are a mostly-null column in moz_bookmarks, we should allow a 1_keyword->n_uris but not the opposite.

  • PROPOSAL: Convert keywords to annotations? they are already being lazily cached by bookmarks service for performance and IO.
  • BUG: TBD
  • PRECONDITIONS: Better annotations management

Annotations are unefficient

Most of the annotations are simple boolean stuff, so they could be categorized as: dataless, with-data. Dataless annotations shouldn't take all the space they are today. With-data annotations on the other side allow for binary data, but that data should NOT be stored in the database, let's allow the filesystem to handle them. Side notes: Having separate page and item annotations is confusing for both API and code, unfortunately we allow to annotate folders and this could be problematic to flatten. Could favicons become binary annotations?

  • PROPOSAL: Split dataless annotations to save space and get better performances.
  • BUG: TBD

Evaluate Microsummaries removal

They are our largest js component, but only a few users use them, and never take off completely as a standard.

  • PROPOSAL: Move microsummaries to an add-on.
  • BUG: bug 524091

Evaluate bookmarks roots removal

Roots caused more bugs than benefits, UI can better use annotated folders in a simple 1-root hierarchy.

  • PROPOSAL: Annotate special folders, 1-root hierarchy
  • BUG: TBD

Async APIs

  • GOAL: No UI-blocking tasks
  • ETA: Firefox 7

Simpler and async querying API in js

The classical querying API is complicated by xpcom, we need a deCOM version of it, this was partially done for Jetpack, must be finished.

  • PROPOSAL: Finish the work started for Jetpack, with a js module.
  • BUG: TBD

Finish async favicons service

Favicons service is already partially async, but the work has to be completed on all APIs.

  • PROPOSAL: Make all interface methods async.
  • BUG: TBD

Finish async containerOpen

Folders work, but it was never used live.

  • PROPOSAL: Evaluate if this is still the wanted approach.
  • BUG: TBD

Async bookmarks API

Largely wanted for Sync and mobile.

  • PROPOSAL: split a new interface, gather from what we did for history.
  • BUG: TBD

Better Notifications

  • GOAL: More performant observers
  • ETA: Firefox 5

Avoid useless notifications traffic

Just changing position could sometimes fire a removed and a added notifications.

  • PROPOSAL: Reduce observers traffic.
  • BUG: TBD

In-content UI

  • GOAL: Better user experience
  • ETA: Firefox 7

Thumbnails service

Any in content UI will probably want pages screenshots, this issue can be coalesced with Sync needs, also look at other vendors suggestions for speed dial thumbs.

  • PROPOSAL: Allow web authors to specify their own thumbs or make a new one with a tentative algo.
  • BUG: TBD

Breadcrumb navigation

As in most today-ish OS, simplify navigation in the Places hierarchies.

  • PROPOSAL: History or Bookmarks breadcrumbs.
  • BUG: TBD

Drag&Drop chrome-content

D&D code needs to be simplified and allow interaction with in-content UI.

  • BUG: TBD

Kill the Library

Should be moved in-content, long treeviews should be paged.

  • BUG: TBD

Donwloads in Places

Move downloads manager to a toolbar widget, and allow to track and manage all downloads in the in-content UI.

  • BUG: TBD

Better search-ability

We should allow the user to filter results from history and bookmarks better, also to allow more granular privacy removals.

  • PROPOSAL: Revised in-content search UI
  • BUG: TBD

Fulltext indexing

  • GOAL: No tables scan on search
  • ETA: TBD

Find a good tokenizer

The pages case is special, since the same profile can have visits in different locales, the tokenizer should be per URI. LibICU is nice but huge.

  • BUG: TBD

Measure perf win/loss

FTS can give nice boosts, but also perf loss in some case, have to figure out the right compromise.

  • BUG: TBD

Sync Needs

Mobile Needs

  • WAL size: force checkpoint
  • Code audit for bad API usage
  • Async Bookmarks
  • Database size reduction
  • Memory cache reduction

To-be-parsed stuff

  • investigate use of templates
  • more natural dragging through menus (diagonal dragging)
  • Places performance tests
  • Fix Cut in frontend, to avoid dataloss
  • deCOM internal bookmarks and history observers
  • Add UI for quick tagging
  • Allow to search folders
  • More reduction of writes and fsyncs
  • reduce query volume
  • awesomebar extensibility (frecency bonus, external providers, ...)
  • tags in HTML export/import?
  • Pluggable views
  • make Most Visited better (show most visited typed uris)
  • support for (bug 331228)
  • deCOM feedParser