Places:Plan
Evolution of future work on Places, the SQLite-driven Bookmarks and History system.
Contents
- 1 Timeline
- 2 Schema Changes
- 2.1 Create history backups
- 2.2 Coalesce titles across history and bookmarks
- 2.3 Tags out of moz_bookmarks
- 2.4 Livemarks out of moz_bookmarks
- 2.5 Remove dynamic containers support
- 2.6 Kill moz_bookmarks table
- 2.7 Re-evaluate history tracking information
- 2.8 Keywords are unefficient
- 2.9 Annotations are unefficient
- 2.10 Evaluate Microsummaries removal
- 2.11 Evaluate bookmarks roots removal
- 3 Async APIs
- 4 Better Notifications
- 5 In-content UI
- 6 Fulltext indexing
- 7 Sync Needs
- 8 Mobile Needs
- 9 To-be-parsed stuff
Timeline
Cherry-picked list of Places related fixes. Minor changes or cleanups are excluded.
Firefox 5
| Bugs | Type | Description |
|---|---|---|
| bug 660036 | IO | Workarounds a bug in Android < 2.3, that causes places.sqlite-wal to grows indefinitely on first Sync. |
| bug 649211 | SUPPORT | Places Maintenance output was broken, not giving any feedback to users. |
| bug 636924 | UI | Makes copied livemarks not appear as common folders in sidebar. |
| bug 599641 | PERF | Runs ANALYZE after each expiration step. SQLite query planner uses this data. |
Firefox 6
| Bugs | Type | Description |
|---|---|---|
| bug 555474 | UI | Avoids bookmark tooltips interrupting Drag'n'Drop actions. |
| bug 620627 | l10n | Fixes localization of grouped history views |
| bug 652379 | API | Fixes results for a Places query pointing to a wrong folder id. |
| bug 630240 | PERF | Avoid full refreshes of history results to avoid locking the main-thread too long. |
| bug 359809 | UI | Fixes keyword searches encoding in location bar |
| bug 639105 | UI | History menu was ignoring some added visits |
| bug 656188 | PERF | Caches last fetched bookmarks info to speed up requests. |
| bug 630622 | UI | Updates the Library design to the browser Aero style. |
| bug 629620 | PERF | Copied bookmarks won't inherit anymore all annotations, since they are new entitities. |
| bug 631374 | UI | Fixes tags going out of view on toggle in bookmarks dialog. |
| bug 658135 | PERF | SQlite should not create transactions for a batch of read-only statements (like Autocomplete) |
| bug 630225 | API | Frecency can now be used to sort queries. |
| bug 524091 | API | Microsummaries have been removed from the core product. |
| bug 633274 | API | Bookmarks observers now pass out more information, included GUIDs. |
| bug 625325 bug 625322 |
UI | Some menuitems in Bookmarks and History menu have been repositioned. |
| bug 663344 | SUPPORT | Fixes a case where weekly maintenance may cause history loss. |
Firefox 7
| Bugs | Type | Description |
|---|---|---|
| bug 661135 | PERF | Simplifies some autocomplete queries. |
| bug 633266 bug 662806 |
API | Pass GUID to history observers. Helps Sync. |
| bug 661445 bug 663269 bug 663104 |
PERF | Various fixes to livemarks code. |
| bug 611568 | UI | File/Import has been moved to the Library window. |
| bug 660109 | API | Allow to distinguish expiration removals, to avoid history dataloss on Sync. |
| bug 416459 | UI | Cut actions are now properly implemented. Fixes UI, saves some IO and has better Sync performances. |
| bug 578268 | CLEANUP | MOZ_MORKREADER is no more in Firefox. |
| bug 593566 | SUPPORT | Fixed a bug that caused broken bookmarks export. |
| bug 645963 | UI | Text-only toolbar view now correctly shows the chevron in pinstripe. |
| bug 655270 bug 655273 |
UI | history.pushstate causes missing title or icon |
Firefox 8
| Bugs | Type | Description | Status |
|---|---|---|---|
| bug 564900 bug 591289 bug 669905 |
UI | Integrate downloads in the Library | IN PROGRESS |
| bug 671001 |
Perf | Places telemetry | IN PROGRESS |
| bug 674210 |
Perf | New expiration heuristics (memshrink) | IN PROGRESS |
| bug 658305 |
Perf | Limit journal for mobile | IN PROGRESS |
| bug 669040 |
Cleanup | Remove Mork code | IN PROGRESS |
| bug 636603 |
API | mozIAsyncHistory fixes | IN PROGRESS |
Future
| Bugs | Description |
|---|---|
| Schema changes | |
| Async bookmarks API | |
| Async annotations API | |
| in-content UI | |
| Finish UpdatePlaces | |
| Audit Mobile API usage | |
| [places-next-wanted] | |
| Limit WAL Size | |
| History backup | |
| Memory usage reduction |
Schema Changes
- GOAL: Database filesize reduction, performances, healthier management
- ETA: Firefox 7
- LEADER: TBD
- PRECONDITIONS: History backup
Create history backups
This is a precondition to stop caring about corruptions, downgrades, compatibility.
- PROPOSAL: per-host backups and expiration, csv-append-like.
- PRECONDITIONS out-of-mainthread file access? (append and remove)
- BUG: TBD
Coalesce titles across history and bookmarks
Strings are the largest database offender, we never use both titles and all the queries have to coalesce titles. Users want us to update titles automatically when they change if they didn't manually set a different bookmark title.
- PROPOSAL: keep only one title in moz_places, track whether it's expirable.
- BUG: TBD
Tags out of moz_bookmarks
Tags are actually folders, this is unefficient and takes a lot of filesize for nothing, searches are slow as well since subqueries suffer from missing indices.
- PROPOSAL: tags in a separate relational table.
- BUG: TBD
Livemarks out of moz_bookmarks
Livemarks are actually folders of temporary bookmarks, this is unefficient and causes unneeded database fragmentation.
- PROPOSAL: redesign from scratch.
- BUG: TBD
Remove dynamic containers support
They never worked, nuke them.
- PROPOSAL: Remove all related code, redesign from scratch.
- BUG: TBD
Kill moz_bookmarks table
Once titles are flattened to moz_places and dynamic containers killed, most of bookmarks data consists of hierarchical information, then just make them so.
- PROPOSAL: Flatten isBookmark tracking in moz_places, have tags and folders just be hierarchical descriptions of these entries.
- PRECONDITIONS: flattened bookmarks in moz_places, tags out of bookmarks.
- BUG: TBD
Re-evaluate history tracking information
We track each single step in the history visit chain, but these data are not really useful as of today, the UI mostly cares about the destination page, not about the full redirects chain, and walking it up is slow and unefficient. Sessions tracking has been removed, and adds useless data. Sync also missed some of these data.
- PROPOSAL: Simplify, remove redundant information, track only what matters.
- BUG: TBD
Keywords are unefficient
Currently these are a mostly-null column in moz_bookmarks, we should allow a 1_keyword->n_uris but not the opposite.
- PROPOSAL: Convert keywords to annotations? they are already being lazily cached by bookmarks service for performance and IO.
- BUG: TBD
- PRECONDITIONS: Better annotations management
Annotations are unefficient
Most of the annotations are simple boolean stuff, so they could be categorized as: dataless, with-data. Dataless annotations shouldn't take all the space they are today. With-data annotations on the other side allow for binary data, but that data should NOT be stored in the database, let's allow the filesystem to handle them. Side notes: Having separate page and item annotations is confusing for both API and code, unfortunately we allow to annotate folders and this could be problematic to flatten. Could favicons become binary annotations?
- PROPOSAL: Split dataless annotations to save space and get better performances.
- BUG: TBD
Evaluate Microsummaries removal
They are our largest js component, but only a few users use them, and never take off completely as a standard.
- PROPOSAL: Move microsummaries to an add-on.
- BUG: bug 524091
Evaluate bookmarks roots removal
Roots caused more bugs than benefits, UI can better use annotated folders in a simple 1-root hierarchy.
- PROPOSAL: Annotate special folders, 1-root hierarchy
- BUG: TBD
Async APIs
- GOAL: No UI-blocking tasks
- ETA: Firefox 7
- LEADER: TBD
Simpler and async querying API in js
The classical querying API is complicated by xpcom, we need a deCOM version of it, this was partially done for Jetpack, must be finished.
- PROPOSAL: Finish the work started for Jetpack, with a js module.
- BUG: TBD
Finish async favicons service
Favicons service is already partially async, but the work has to be completed on all APIs.
- PROPOSAL: Make all interface methods async.
- BUG: TBD
Finish async containerOpen
Folders work, but it was never used live.
- PROPOSAL: Evaluate if this is still the wanted approach.
- BUG: TBD
Async bookmarks API
Largely wanted for Sync and mobile.
- PROPOSAL: split a new interface, gather from what we did for history.
- BUG: TBD
Better Notifications
- GOAL: More performant observers
- ETA: Firefox 5
- LEADER: TBD
Avoid useless notifications traffic
Just changing position could sometimes fire a removed and a added notifications.
- PROPOSAL: Reduce observers traffic.
- BUG: TBD
In-content UI
- GOAL: Better user experience
- ETA: Firefox 7
- LEADER: TBD
Thumbnails service
Any in content UI will probably want pages screenshots, this issue can be coalesced with Sync needs, also look at other vendors suggestions for speed dial thumbs.
- PROPOSAL: Allow web authors to specify their own thumbs or make a new one with a tentative algo.
- BUG: TBD
As in most today-ish OS, simplify navigation in the Places hierarchies.
- PROPOSAL: History or Bookmarks breadcrumbs.
- BUG: TBD
Drag&Drop chrome-content
D&D code needs to be simplified and allow interaction with in-content UI.
- PROPOSAL: TBD
- BUG: TBD
Kill the Library
Should be moved in-content, long treeviews should be paged.
- PROPOSAL: TBD
- BUG: TBD
Donwloads in Places
Move downloads manager to a toolbar widget, and allow to track and manage all downloads in the in-content UI.
- PROPOSAL: TBD
- BUG: TBD
Better search-ability
We should allow the user to filter results from history and bookmarks better, also to allow more granular privacy removals.
- PROPOSAL: Revised in-content search UI
- BUG: TBD
Fulltext indexing
- GOAL: No tables scan on search
- ETA: TBD
- LEADER: TBD
Find a good tokenizer
The pages case is special, since the same profile can have visits in different locales, the tokenizer should be per URI. LibICU is nice but huge.
- PROPOSAL: TBD
- BUG: TBD
Measure perf win/loss
FTS can give nice boosts, but also perf loss in some case, have to figure out the right compromise.
- PROPOSAL: TBD
- BUG: TBD
Sync Needs
- Async bookmarks (bug 519514, see also https://wiki.mozilla.org/Places/AsyncAPIsForSync#nsIBookmarkInfo)
- use GUIDs in queries (folder=guid), in json and html backups
- finish UpdatePlaces (see dependents of bug 606966)
- use url strings not nsIURI (bug 636393)
Mobile Needs
- WAL size: force checkpoint
- Code audit for bad API usage
- Async Bookmarks
- Database size reduction
- Memory cache reduction
To-be-parsed stuff
- investigate use of templates
- more natural dragging through menus (diagonal dragging)
- Places performance tests
- Fix Cut in frontend, to avoid dataloss
- deCOM internal bookmarks and history observers
- Add UI for quick tagging
- Allow to search folders
- More reduction of writes and fsyncs
- reduce query volume
- awesomebar extensibility (frecency bonus, external providers, ...)
- tags in HTML export/import?
- Pluggable views
- make Most Visited better (show most visited typed uris)
- support for browser.chrome.favicons (bug 331228)
- deCOM feedParser
