- Finished initial draft of user stories & dev plan for French toast, except for first run stuff and new "Explore" stuff
- Put together a draft roadmap based on the development plan: https://intranet.mozilla.org/Pancake/French_Toast_Roadmap
- Looked into some tools for managing/tracking -- we're still a small team so most of them seem overly complicated. Will start with a spreadsheet or wiki list & bugzilla and see how that goes.
- Had another great meeting with Cori -- expecting a user study proposal from her by end of day Tuesday
- Go through first two French Toast phases (A & B) with Stuart and Gordon to finalize plans.
- Get required Backend and Native items added to phases A & B
- Move French Toast roadmap & planning stuff to a public space (wiki.mo & open access)
- Look into building a simple set of semantic wiki tools for tracking our stuff on the wiki instead of using a google spreadsheet -- would be preferable, but more complicated/fiddly
- Think through various user tasks in French Toast -- diagram the flows (sketches) and see if there are any obvious flaws
- Brainstorm how to better communicate Pancake goings on internally (for the time being) and eventually externally
- Continue working through backlog of reading & longer-term thinking
- Worked on, mostly finished Pancake Brown Bag presentation
- Developed and rehearsed user testing script
- Scheduling user testing sessions
- User testing w/ prototype session
- Discussed "save for later" functionality with Deb, Stuart
- Ideation and wireframing for first-run, app dashboard and search
- Worked on visual concepts for French Toast
- Planned modeling and templating library changes with sfoster
- no work on pancake/french toast
- Concentration on Fennec/B2G
- Back from PTO (since last tuesday)
- Tried re-implementing the recommendation engine in Python. No good, it was way too slow for development. Computation was perhaps 50X slower than the Java version.
- Going to give the recommendation part of the algorithm another shot in Java, also starting to gear up for other French Toast components.
It turns out the problem I'm trying to solve is a pretty hard one to solve. Current solutions at this problem are domain-specific. That said, there are things we can try.
Here is what the recommendation engine does so far:
- Take a collection of documents, assume there exists K topics, find out how the word combinations in the documents fit in those K topics.
- For each document, find out how each document fits proportionally in each of those K topics.
Attempts at recommendation
- Given an input of a url, obtain the text of the url, making it document D and infer a topic distribution for that document.
- From the topic distribution, obtain a number of documents in the top N topics.
This does not work very well, since even if a document is a 50% match in topic A, it does not mean it is a good document to read or that it is similar to doc D.
Ongoing/Future attempts at recommendation
- Transform the intermediate result set into a query-able form using Locally Sensitive Hashing
- Cluster the documents into smaller sub-divisions and run known (but inefficient) similarity algorithms in the reduced candidate set.
- Use spatial algorithms to find nearest neighbouring documents. (hard to do given the high-dimensionality of the data. a reduction of dimensionality is required)
- Worked on typeahead. Dumped Cleo in favor of Elasticsearch
- Results live on http://ec2-50-19-67-63.compute-1.amazonaws.com:8081/
- Code on https://github.com/mozillapancake/typeahead-with-elasticsearch
- Working on minimal Android and iOS applications that can be used for French Toast development