Places:Plan

Evolution of future work on Places, the SQLite-driven Bookmarks and History system.

Timeline

Cherry-picked list of Places related fixes. Minor changes or cleanups are excluded.

Firefox 5

Bugs	Type	Description
bug 660036	IO	Workarounds a bug in Android < 2.3, that causes places.sqlite-wal to grows indefinitely on first Sync.
bug 649211	SUPPORT	Places Maintenance output was broken, not giving any feedback to users.
bug 636924	UI	Makes copied livemarks not appear as common folders in sidebar.
bug 599641	PERF	Runs ANALYZE after each expiration step. SQLite query planner uses this data.

Firefox 6

Bugs	Type	Description
bug 555474	UI	Avoids bookmark tooltips interrupting Drag'n'Drop actions.
bug 620627	l10n	Fixes localization of grouped history views
bug 652379	API	Fixes results for a Places query pointing to a wrong folder id.
bug 630240	PERF	Avoid full refreshes of history results to avoid locking the main-thread too long.
bug 359809	UI	Fixes keyword searches encoding in location bar
bug 639105	UI	History menu was ignoring some added visits
bug 656188	PERF	Caches last fetched bookmarks info to speed up requests.
bug 630622	UI	Updates the Library design to the browser Aero style.
bug 629620	PERF	Copied bookmarks won't inherit anymore all annotations, since they are new entitities.
bug 631374	UI	Fixes tags going out of view on toggle in bookmarks dialog.
bug 658135	PERF	SQlite should not create transactions for a batch of read-only statements (like Autocomplete)
bug 630225	API	Frecency can now be used to sort queries.
bug 524091	API	Microsummaries have been removed from the core product.
bug 633274	API	Bookmarks observers now pass out more information, included GUIDs.
bug 625325 bug 625322	UI	Some menuitems in Bookmarks and History menu have been repositioned.
bug 663344	SUPPORT	Fixes a case where weekly maintenance may cause history loss.

Firefox 7

Bugs	Type	Description
bug 661135	PERF	Simplifies some autocomplete queries.
bug 633266 bug 662806	API	Pass GUID to history observers. Helps Sync.
bug 661445 bug 663269 bug 663104	PERF	Various fixes to livemarks code.
bug 611568	UI	File/Import has been moved to the Library window.
bug 660109	API	Allow to distinguish expiration removals, to avoid history dataloss on Sync.
bug 416459	UI	Cut actions are now properly implemented. Fixes UI, saves some IO and has better Sync performances.
bug 578268	CLEANUP	MOZ_MORKREADER is no more in Firefox.
bug 593566	SUPPORT	Fixed a bug that caused broken bookmarks export.
bug 645963	UI	Text-only toolbar view now correctly shows the chevron in pinstripe.
bug 655270 bug 655273	UI	history.pushstate causes missing title or icon

Firefox 8

Bugs	Type	Description	Status
bug 564900 bug 591289 bug 669905	UI	Integrate downloads in the Library	IN PROGRESS
bug 671001	Perf	Places telemetry	IN PROGRESS
bug 674210	Perf	New expiration heuristics (memshrink)	IN PROGRESS
bug 658305	Perf	Limit journal for mobile	IN PROGRESS
bug 669040	Cleanup	Remove Mork code	IN PROGRESS
bug 636603	API	mozIAsyncHistory fixes	IN PROGRESS

Future

Bugs	Description
	Schema changes
	Async bookmarks API
	Async annotations API
	in-content UI
	Finish UpdatePlaces
	Audit Mobile API usage
	[places-next-wanted]
	Limit WAL Size
	History backup
	Memory usage reduction

Schema Changes

• GOAL: Database filesize reduction, performances, healthier management
• ETA: Firefox 7
• LEADER: TBD
• PRECONDITIONS: History backup

Create history backups

This is a precondition to stop caring about corruptions, downgrades, compatibility.

• PROPOSAL: per-host backups and expiration, csv-append-like.
• PRECONDITIONS out-of-mainthread file access? (append and remove)
• BUG: TBD

Coalesce titles across history and bookmarks

Strings are the largest database offender, we never use both titles and all the queries have to coalesce titles. Users want us to update titles automatically when they change if they didn't manually set a different bookmark title.

• PROPOSAL: keep only one title in moz_places, track whether it's expirable.
• BUG: TBD

Tags out of moz_bookmarks

Tags are actually folders, this is unefficient and takes a lot of filesize for nothing, searches are slow as well since subqueries suffer from missing indices.

• PROPOSAL: tags in a separate relational table.
• BUG: TBD

Livemarks out of moz_bookmarks

Livemarks are actually folders of temporary bookmarks, this is unefficient and causes unneeded database fragmentation.

• PROPOSAL: redesign from scratch.
• BUG: TBD

Remove dynamic containers support

They never worked, nuke them.

• PROPOSAL: Remove all related code, redesign from scratch.
• BUG: TBD

Kill moz_bookmarks table

Once titles are flattened to moz_places and dynamic containers killed, most of bookmarks data consists of hierarchical information, then just make them so.

• PROPOSAL: Flatten isBookmark tracking in moz_places, have tags and folders just be hierarchical descriptions of these entries.
• PRECONDITIONS: flattened bookmarks in moz_places, tags out of bookmarks.
• BUG: TBD

Re-evaluate history tracking information

We track each single step in the history visit chain, but these data are not really useful as of today, the UI mostly cares about the destination page, not about the full redirects chain, and walking it up is slow and unefficient. Sessions tracking has been removed, and adds useless data. Sync also missed some of these data.

• PROPOSAL: Simplify, remove redundant information, track only what matters.
• BUG: TBD

Keywords are unefficient

Currently these are a mostly-null column in moz_bookmarks, we should allow a 1_keyword->n_uris but not the opposite.

• PROPOSAL: Convert keywords to annotations? they are already being lazily cached by bookmarks service for performance and IO.
• BUG: TBD
• PRECONDITIONS: Better annotations management

Annotations are unefficient

Most of the annotations are simple boolean stuff, so they could be categorized as: dataless, with-data. Dataless annotations shouldn't take all the space they are today. With-data annotations on the other side allow for binary data, but that data should NOT be stored in the database, let's allow the filesystem to handle them. Side notes: Having separate page and item annotations is confusing for both API and code, unfortunately we allow to annotate folders and this could be problematic to flatten. Could favicons become binary annotations?

• PROPOSAL: Split dataless annotations to save space and get better performances.
• BUG: TBD

Evaluate Microsummaries removal

They are our largest js component, but only a few users use them, and never take off completely as a standard.

• PROPOSAL: Move microsummaries to an add-on.
• BUG: bug 524091

Evaluate bookmarks roots removal

Roots caused more bugs than benefits, UI can better use annotated folders in a simple 1-root hierarchy.

• PROPOSAL: Annotate special folders, 1-root hierarchy
• BUG: TBD

Async APIs

• GOAL: No UI-blocking tasks
• ETA: Firefox 7
• LEADER: TBD

Simpler and async querying API in js

The classical querying API is complicated by xpcom, we need a deCOM version of it, this was partially done for Jetpack, must be finished.

• PROPOSAL: Finish the work started for Jetpack, with a js module.
• BUG: TBD

Finish async favicons service

Favicons service is already partially async, but the work has to be completed on all APIs.

• PROPOSAL: Make all interface methods async.
• BUG: TBD

Finish async containerOpen

Folders work, but it was never used live.

• PROPOSAL: Evaluate if this is still the wanted approach.
• BUG: TBD

Async bookmarks API

Largely wanted for Sync and mobile.

• PROPOSAL: split a new interface, gather from what we did for history.
• BUG: TBD

Better Notifications

• GOAL: More performant observers
• ETA: Firefox 5
• LEADER: TBD

Avoid useless notifications traffic

Just changing position could sometimes fire a removed and a added notifications.

• PROPOSAL: Reduce observers traffic.
• BUG: TBD

In-content UI

• GOAL: Better user experience
• ETA: Firefox 7
• LEADER: TBD

Thumbnails service

Any in content UI will probably want pages screenshots, this issue can be coalesced with Sync needs, also look at other vendors suggestions for speed dial thumbs.

• PROPOSAL: Allow web authors to specify their own thumbs or make a new one with a tentative algo.
• BUG: TBD

Breadcrumb navigation

As in most today-ish OS, simplify navigation in the Places hierarchies.

• PROPOSAL: History or Bookmarks breadcrumbs.
• BUG: TBD

Drag&Drop chrome-content

D&D code needs to be simplified and allow interaction with in-content UI.

• PROPOSAL: TBD
• BUG: TBD

Kill the Library

Should be moved in-content, long treeviews should be paged.

• PROPOSAL: TBD
• BUG: TBD

Donwloads in Places

Move downloads manager to a toolbar widget, and allow to track and manage all downloads in the in-content UI.

• PROPOSAL: TBD
• BUG: TBD

Better search-ability

We should allow the user to filter results from history and bookmarks better, also to allow more granular privacy removals.

• PROPOSAL: Revised in-content search UI
• BUG: TBD

Fulltext indexing

• GOAL: No tables scan on search
• ETA: TBD
• LEADER: TBD

Find a good tokenizer

The pages case is special, since the same profile can have visits in different locales, the tokenizer should be per URI. LibICU is nice but huge.

• PROPOSAL: TBD
• BUG: TBD

Measure perf win/loss

FTS can give nice boosts, but also perf loss in some case, have to figure out the right compromise.

• PROPOSAL: TBD
• BUG: TBD

Sync Needs

• Async bookmarks (bug 519514, see also https://wiki.mozilla.org/Places/AsyncAPIsForSync#nsIBookmarkInfo)
• use GUIDs in queries (folder=guid), in json and html backups
• finish UpdatePlaces (see dependents of bug 606966)
• use url strings not nsIURI (bug 636393)

Mobile Needs

• WAL size: force checkpoint
• Code audit for bad API usage
• Async Bookmarks
• Database size reduction
• Memory cache reduction

To-be-parsed stuff

• investigate use of templates
• more natural dragging through menus (diagonal dragging)
• Places performance tests
• Fix Cut in frontend, to avoid dataloss
• deCOM internal bookmarks and history observers
• Add UI for quick tagging
• Allow to search folders
• More reduction of writes and fsyncs
• reduce query volume
• awesomebar extensibility (frecency bonus, external providers, ...)
• tags in HTML export/import?
• Pluggable views
• make Most Visited better (show most visited typed uris)
• support for browser.chrome.favicons (bug 331228)
• deCOM feedParser