Firefox/Projects/PlacesQueryAPIRedesign

From MozillaWiki
Jump to navigation Jump to search
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

Places Query API

Create a sensible, easy to use Query API for Places for Firefox.next & Jetpack. This API should make it possible to do targeted queries against history and bookmarks with a minimum of code.

Goals/Use Cases

  • An elegant and easy to use API. Fx devs, Jetpack and extension developers will all benefit.
  • allows us to prototype the new ui for Fx.next
  • 'Pluggable' results handling Easy wrappable querying object and results
  • Focus on JS usage
  • Paging Support - Sounds like a great use of Generators! dropped, there is no perf gain, can be a follow-up
  • Fetch individual record by Id or other property

(After talking to the Labs guys about Weave, we should try to provide a plugin-interface whereby you can allow a developer to keep track of and index random JSON objects.

Dan Mills has example code that was created for the people store. Link coming soon.)

Non Goals

  • Tailoring the API to tree views
  • Creating the perfect API (lets iterate)
  • Create an abstract datastore
  • Intended as a snap-in replacement of the current API Future follow-up through a live-updating wrapper
  • Wrap current sync and XPCOM API

Status

  • Old prototype landed in Jetpack 0.8, currently being revised
  • Team
    • API: ddahl, mak
    • UX/UI: not required so far, future users could though
  • Implementation bugs: bug 522572 bug 545700 bug 531940 bug 543888
  • Next Steps
    • Iterate the feel of the API with the Places team and other interested developers. Ideas and feedback are encouraged. Build working examples to encourage feedback.

Timeline / Milestones

  • 2010/02 Landed in Jetpack 0.8
  • 2010/03 Work on "skeleton" API design with Marco
  • 2010/05 New redesign, collecting feedback and proceeding.

Delivery Requirements

  • This is purely additive.
  • Coordination with Jetpack as this should be coded once, checked into places and imported into Jetpack's implementation.

Constraints

  • Final implementation should use only Async Storage APIs

Dependencies

Testing

  • Comprehensive xpcshell tests
  • Example/Documentation style Browser Chrome tests Not for now, when treeviews will use this.
  • TDD

Related Projects

  • Jetpack

Additional Details

The initial API "sketches" are here: [1]

Faaborg has been mocking up designs and putting a lot pf thought into the UX: http://blog.mozilla.com/faaborg/2009/10/13/browsing-your-personal-web

Inspiration

Links to code we should study...

Gloda

Gloda: [2]

facet.js: [3]

Gloda Fundamental Attribute provider: [4]

Explicit Attribute Provider: [5]

FacetView: [6]

Current Sketches

function callback(results) {
  // results is a simple array of ResultItem, it's pretty easy to wrap it
  // if more complex management is needed.
  if (results.length == 0)
      dump("En empty results array indicates last results push\n");
  else
      dump("Collected " + results.length + " results this turn\n");
}

// Results are pushed to the callback as soon as they are available.
new PlacesQuery([object]QueryConf, [function]callback, [scope]thisObject);

// It's possible to pass multiple QueryConf objects, results will be merged
// based on the .merge attribute of each config.
// First query results are always unioned.
// In case of multiple queries, sort, group and limit of the first query will
// be used for the global result.
new PlacesQuery([QueryConf1, QueryConf2], callback, thisObject);

// It's possible to create the query, but run it later.
let query = new PlacesQuery(QueryConf);
query.execute(callback, thisObject);

/* The query can be so configured:
 *
 * _QueryConf_ = {
 *   phrase: string.
 *           Containing this string in either title, uri or tags.  Case
 *           insensitive.  Can use ^ and $ to match at beginning or end.
 *   host: string.
 *         Containing this string in the host.  Case insensitive.
 *         Can use ^ and $ to match at beginning or end.
 *   uri: string.
 *        Containing this string in the uri.  Case insensitive.
 *        Can use ^ and $ to match beginning or end.
 *   annotated: array of strings.
 *              With these annotations (Either page or item).
 *   bookmarked: object
 *   {
 *     tags: array of strings.
 *           Tagged with these tags.
 *     at: object
 *     {
 *       folder: number.
 *               Inside this folder. (non-recursive)
 *       position: number.
 *                 At this position. (relative to folder).
 *                 If undefined or null matches all children.
 *     }
 *     id: number.
 *         Bookmarked with this id.
 *     when: object
 *     {
 *       begin: optional Date object
 *              Bookmarks created after this time (included).
 *              Defaults to epoch.
 *       end: optional Date object 
 *            Bookmarks created before this time (included).
 *            Defaults to now.
 *     }
 *     modified: object
 *     {
 *       begin: optional Date object
 *              Bookmarks modified after this time (included).
 *              Defaults to epoch.
 *       end: optional Date object 
 *            Bookmarks modified before this time (included).
 *            Defaults to now.
 *     }
 *     onlyContainers: boolean.
 *                           Removes any non-container from results.
 *                           Default is false.
 *     excludeReadOnlyContainers: boolean.
 *                                Removes read only containers from results.
 *                                Default is false.
 *   }
 *   visited: object
 *   {
 *     count: object
 *            This is lazily based on visit_count, thus is not going to work
 *            for not counted transitions: embed, download, framed_link.
 *     {
 *       min: optional number.
 *            With more than this many visits.
 *            Defaults to 0.
 *       max: optional number.
 *            With less than this many visits.
 *            Defaults to inf.
 *     }
 *     transitions: array of transition types.
 *                  With at least one visit for each of these transitions.
 *     when: object
 *     {
 *       begin: optional Date object
 *              With visits after this time (included).
 *              Defaults to epoch.
 *       end: optional Date object 
 *            With visits before this time (included).
 *            Defaults to now.
 *     }
 *     excludeRedirectSources: boolean.
 *                             Removes redirects sources from results.
 *                             Default is false.
 *     excludeRedirectTargets: boolean.
 *                             Removes redirects targets from results.
 *                             Default is false.
 *     includeHidden: boolean.
 *                    Includes also pages marked as hidden.
 *                    Default is false.
 *     allVisits: boolean.
 *                Returns all visits ungrouped.
 *                Default is false, that means visits are grouped by uri.
 *   }
 *   sort: object
 *   {
 *     by: string.
 *         Either "none", "title", "time", "uri", "accessCount", "lastModified",
 *         "frecency".  Defaults to "none".
 *     dir: string.
 *          Either "asc" or "desc".  Defaults to "asc".
 *   }
 *   group: string.
 *          Either "tags", "containers", "days", "months", "years" or "domains".
 *          Defaults to "none".
 *          NOTE: Not yet implemented.
 *   limit: number.
 *          Maximum number of results to return.  Defaults to all results.
 *   merge: string.
 *          How to merge this query's results with others in the same request.
 *          Valid values:
 *          - "union": merge results from the 2 queries.
 *          - "except": exclude current results from the previous ones.
 *          - "intersect": only current results that are also in previous ones.
 * }
 * 
 * NOTE: In case of multiple queries, sort, group and limit of the first query
 * will be used for the global result.
 *
 *
 * The query returns an array of these simple ResultItem objects
 * ResultItem {
 *   pageId: the place_id of the page, useful for external linking of resources
 *   uri:
 *   title:
 *   host:
 *   accessCount: visit count for pages, aggregated or null for containers
 *   time: visit date, aggregated or null for containers
 *   icon: url of the icon
 *   sessionId: visit session or null if not available
 *   itemId: bookmark id or null if not bookmarked (see isBookmarked)
 *   isBookmarked: whether this is bookmarked or not
 *   dateAdded: bookmark creation Date() or null if not bookmarked
 *   lastModified bookmark modification Date() or null if not bookmarked
 *   parentId: bookmark folder id or null if not bookmarked
 *   tags: string of tags
 *   tagsArray: array of tags
 *   bookmarkIndex: position of the bookmark in his container or null
 *   frecency:
 *   visitId: id of the visit or null
 *   referringVisitId: id of the originating visit or null
 *   referringUri: uri of the originating visit or null
 *   transitionType: transition of this visit or null
 *   type: old container implementation type
 *   readableType: "bookmark", "container", "separator", "visit", "page"
 *   query: if this is a container will return a new PlacesQuery for contents
 * }
 * 
 */

// EXAMPLES (not all of them work, they are supposed to show the conf)

// Bookmarks toolbar.
new PlacesQuery({ bookmarked: {
                      folders: [PlacesUtils.bookmarksToolbarFolderId]
                  },
                  group: "container"
                });

// History menu.
new PlacesQuery({ visited: {},
                  sort: { by: "time", dir: "desc" },
                  limit: 10
                });

// Start-of-awesomebar (still missing adaptive, keywords, ...).
new PlacesQuery({ phrase: "someword",
                  sort: { by: "frecency", dir: "desc" },
                  limit: 12
                });

// Folder picker.
new PlacesQuery({ bookmarked: {
                      at: { folder: PlacesUIUtils.allBookmarksFolderId }
                      onlyContainers: true,
                      excludeReadOnlyContainers: true
                  },
                  group: "container"
                });

// Most recent bookmarks.
new PlacesQuery({ bookmarked: {},
                  sort: { by: "lastModified", dir: "desc"}
                  limit: 5
                });

// Most recent tags.
new PlacesQuery({ bookmarked: {},
                  group: "tags",
                  limit: 5
                });

// Left pane query.
new PlacesQuery({ bookmarked: { folders: [PlacesUIUtils.leftPaneFolderId] },
                  annotated: ["Places/OrganizerQuery"]
                  group: "containers"
                });

// Downloads
new PlacesQuery({ visited: {
                      transitions: [PlacesUtils.history.TRANSITION_DOWNLOAD],
                      includeHidden: true,
                      allVisits: true
                  },
                  sort: { by: "time", dir: "desc" }
                });

// Visited bookmarks containing either "foo" or "bar".
new PlacesQuery([
    { phrase: "foo",
      bookmarked: {},
      visited: {}
    },
    { phrase: "bar",
      bookmarked: {},
      visited: {}
    },
);


// Visited bookmarks containing "foo", but not "bar".
new PlacesQuery([
    { phrase: "foo",
      bookmarked: {},
      visited: {}
    },
    { phrase: "bar",
      bookmarked: {},
      visited: {},
      merge: "except"
    },
);

// Visited bookmarks containing "foo" and "bar".
new PlacesQuery([
    { phrase: "foo",
      bookmarked: {},
      visited: {}
    },
    { phrase: "bar",
      bookmarked: {},
      visited: {},
      merge: "intersect"
    },
);

Testing

  • Test plan
  • Most of this will be automated and will not need QA involvement
  • QA will keep an eye out for regressions during regular release cycles
  • QA will run a testday once this has landed
Retrieved from "https://wiki.mozilla.org/index.php?title=Firefox/Projects/PlacesQueryAPIRedesign&oldid=233455"