Firefox/Namoroka/Improved form history

	THIS PAGE IS A WORKING DRAFT
	The page may be difficult to navigate, and some information on its subject might be incomplete and/or evolving rapidly. If you have any questions or ideas, please add them as a new topic on the discussion page.

The goal is to make form history more intelligent about how it stores are retrieves form data. Currently, satchel displays saved data alphabetically to the user based on prefix matches with what the user has typed. Making form history more like the "Awesomebar" will increase the usefulness of this feature.

Overview

• User interface with the dropdown of suggestions will remain the same with a possible addition of a delete button
• Consider data on last use, first use, and # uses in ordering of suggestions
  • see bug 474880 & bug 370117
• Use groupings/containers of relevancy so that result order is more stable between small changes to the query
• Ability to search within history instead of just head matches
  • These results should be ranked lower or possibly only shown if there are no results that matched from the head
• Don't have to always consider field names (especially if no matching results found for a field)
• Use Javascript port of autocomplete portions - bug 469443
• Use preferences to allow some user control over history results
• Consider keeping track of what results are used based on what was typed at the time (similar to the awesomebar)

Use Cases

• Street Addresses
  
  • You forget the street number for an address but remember the street name. Just type the street name ie. "Landings" and the previously typed address "1981 Landings Drive" will be displayed
• Previous searches
  • If the user previous searched for "perl program" and "programming perl", typing "perl" should show both entries.
• Email Addresses
  • If you remember a portion of an email address (such as the domain), you will be able to see previously submitted email addresses from that domain

Progress

1. Sort dropdown results by frecency - bug 370117
   • patch reviewed and landed on 1.9.2
2. Allow searching within existing form history entries - bug 446247
   • Match by words using what is available in SQLite.
   • Bonuses for word prefix matches
   • patch reviewed and landed on 1.9.2
3. Consider other improvements
   • bug 507271 - Improve/remove bucketing of satchel entries
   • Remove/hide some of the new preferences
   • Look into adaptive learning for form history (to learn what entry is selected for a given input)
   • Consider showing matches from other form fields

Implementation Notes

• Dropdown with no search term could be treated as a special case
• Could consider average number of uses per day (lastUsed - firstUsed)
  • Otherwise a history table could be used for relevancy calculations
  • Could give bonus for old and new entries
• Test with appropriate indexes in SQLite
• Benchmark time taken for various relevancy calculations
• Weight word boundaries higher
• After typing in the middle of existing text, dropdown is not updated to reflect the change
• Look at about:config prefs for urlbar to get ideas
• Use a custom SQLite function to improve word boundary detection
• Adjust weighting of non-word boundary matches
• Change/remove bucketing

Frecency Calculation

a = timesUsed
b = lastUsed - firstUsed
c = lastUsed multiple
d = agedBonus (for entries that have survived expiration)

frecency = a/b * c * d

Configuration

• browser.formfill.
  • agedWeight - multiply the score by this value for entries that have survived expiration
  • boundaryWeight - how much weight (in %) to add to word boundary matches
  • bucketSize - range of relevancy values to group together and sort alphabetically
  • maxTimeGroupings - maximum number of timeGroupingSize units to give a bonus to
  • prefixWeight - multiply the score by this value for exact prefix matches
  • timeGroupingSize - number of seconds in a time container

Existing form history bugs from form manager to consider

• bug 474880 - Chronological sort
• bug 370117 - form autocomplete should sort by frequency of use
• bug 252486 - Option to disable form manager (saved form information) for secure websites (https)
• bug 385741 - Want to be able to exclude sites from form autofill
• bug 260908 - Leading/trailing spaces should be trimmed from AutoComplete
• bug 457843 - Hovering mouse pointer across autocomplete options yields unexpected textbox result
• bug 380963 - Providers of form-fill data should be able to register with satchel
• bug 409795 - Data filled in a web is used in others if field name is the same
• bug 414524 - Form autocomplete should sort numbers numerically
• bug 424646 - form history autocomplete doesn't show on input fields that are focused too early (e.g. in newly opened tabs)
• bug 439716 - Form Manager should be a JavaScript-based component
• bug 469443 - Form Manager Storage should be a JavaScript-based component
• bug 444728 - autocomplete disregards maxlength for input fields
• bug 448540 - Form auto-complete stops working after Enter is pressed
• bug 463913 - Ability to delete saved entries directly in autofill dropdowns
• bug 320720 - Ability to remove all saved data (entries) in a single form
• bug 492701 - form history should cap the number of fields saved per form submission
• bug 488796 - form history should not save extremely large values
• bug 497019 - form fill should integrate with system address book
• bug 446247 - Autocomplete should match any part of the string
• bug 497541 - Autocomplete dropdown should update when text inserted before end
• bug 487556 - satchel should send a notification when adding/removing/changing an item
• bug 510168 - Emphasize matches in AwesomeComplete

References

• Blog Post: "Improved Form History"