Firefox/Input/2012Q3 Phase0 Reqs

Summary

We're doing a rewrite of input.mozilla.org. This covers Phase 0 which gets us to a minimum viable site that we can replace the existing site with.

Deadline: Done coding by end of 2012 Q3

Status of this document: First draft. Needs work.

Bug list: https://bugzilla.mozilla.org/showdependencytree.cgi?id=780626

Source documents

• Cheng's requirements doc: https://etherpad.mozilla.org/input-requirements-2012
• Mike's research on db requirements: https://etherpad.mozilla.org/kYwQx9Emjf
• Etherpad we used to talk about db requirements: https://etherpad.mozilla.org/secret-project-foo

Requirements

All requirements should be numbered R# and the number should never change. If we add new requirements, they get new numbers--we never reuse old numbers.

R1: Must support multiple products/channels each with unique feedback needs.

Phase 0:

We only need to support Firefox Desktop Stable and Firefox Mobile Stable.

For Firefox Desktop Stable, we'll collect the same data we're collecting in the current input.mozilla.org.

For Firefox Mobile Stable, we'll collect the same data we're collecting in the current input.mozilla.org plus manufacturer and device (if we can figure out where to get that from).

Future:

In the future (phase 1+), we'll want to collect data for other channels and other products. For example:

• Firefox desktop aurora
• Firefox desktop beta

For Aurora, we want to target feedback on specific features or aspects of Firefox (like performance or plugins). The input system needs to know where the feedback comes from (e.g. a start page snippet or unprompted by the user selecting a menu).

For Beta, users should be explicitly asked about changes they notice relative to the previous version of Firefox. Users should also be encouraged to give feedback repeatedly.

Thus "unique feedback needs" probably entails that different product/channel combinations need different feedback forms.

From Cheng:

For phase 0, I think our products are going to be Firefox desktop (all lumped together using the desktop UI) and Firefox mobile (all lumped together using the existing mobile UI).

In terms of knowing where the user comes from, we can just record URL parameters -- however, that's probably not a phase 0 requirement.

R2: Must have feedback page and thank you page.

Mushing a bunch of requirements into one. Needs to have a feedback page. We should lift that from the current site.

Submitting the feedback page brings you to the thank you page. We should lift that from the current site, too.

NIXED: R3: Must capture sufficient information to make feedback actionable.

"Sufficient" here is vague, but we're specifically talking about email address, urls, product version, operating system, locale, ...

Probably means we need to add a few more fields to the feedback page.

In the future, we'll also want about:support information, plugin data, ...

There are privacy implications here.

Phase 0:

We're not going to collect email addresses or about:support data. We're only collecting what we collected before. We'll deal with this in phase 1.

R4: Leaving feedback must be friendly and approachable.

We should get some UX help. We should meet WCAG 2.0 guidelines.

R5: Must be able to store significant amounts of feedback.

The ultimate goal is to use this site for feedback across Mozilla's product range and possibly also offer it as a service for Marketplace apps. Thus it needs to be able to handle large amounts of data both for storage as well as for searching.

TODO: Find out how much data we're currently storing and guess future needs.

Cheng: My target is to increase from current needs by 20x. We currently max out at about 2000 pieces of feedback per day.

R6: Analysts must be able to analyze data effectively.

"Effectively" is vague here, but the gist of it is that they need to be able to specify and see different slices of the data. One way to do this is to support Lucene query parser language for searches for analysts. Then, coupled with knowledge of the index, they should be able to do complex queries against the corpus.

They need to answer questions like:

• What are the top extensions for users who use the word "slow" (or maybe use that tag)?
• What are the top trending words used in conjunction with the word "youtube"?

Need to support these kinds of queries:

• Actual/filtered feedback in a date range (what are people saying about X this week) where X is general like "Firefox desktop" or specific like "negative feedback about Firefox desktop on windows 8 installs in Germany".

• Trending queries: Alerts would be awesome but generally: The top trending extensions/URLs/words/prefs in the last (timeframe) are compared to the previous (timeframe). I think we'd also want to be able to show what is trending even if it isn't significantly different than the previous period. This will show long standing "Big" issues.

• Straight-up comparisons: Users of X extension are more likely to use these words than users of Y extension.

• Graphing the above kinds of queries and making it easy to generate them on the fly will help engineers visualize the feedback and hence makes it more actionable without having to have a SUMO team member interpret it.

Cheng: Phase 0, let's just work on displaying and filtering/searching the feedback. Graphing by day can be a Phase 1 project; Alerts, reports, trending, metadata analysis can be all done as a future.

R7: Feedback page must work.

Need better requirement text here. The gist of it is that it should work unless we're having a site-wide meltdown.

Further, if ES is down, the site should still be usable.

Cheng: This is a performance and uptime thing. I'd like to see if we can make the following two promises: 1) A user will always be able to submit feedback. 2) Even if some of the analytics tools are down, we should always be able to read that feedback.

R8: Must support multiple locales.

Strings should be localized. We can use existing Mozilla processes for this.

Cheng: A lot of the existing strings are localized, I'd like to be able to migrate in all of that work so we don't duplicate it.

R9: Must have a mobile/b2g friendly interface.

This probably requires a set of mobile-centric templates.

Cheng: The B2G requirement is Phase 1 (end of Q4). We already have a mobile UI that we can lift. -- The only hard requirement is to have the feedback and thankyou pages be mobile friendly, the analysis/reading feedback pages can be desktop-only.

R10: Must either support or have redirects for urls currently being used.

input.mozilla.org/feedback (and #happy and #sad)

R11: Feedback form will have sentiment field which only has 'happy' and 'sad' options.

The old input site has "happy", "sad", and "idea" and the code has additional options as well--we'll only have "happy" and "sad" and the user must choose one of them.

Anti-requirements

Figured I'd throw these in here so I don't forget.

A-R1: We don't need to migrate existing production data.

EOM

Implementation thoughts

Surveys

See https://bugzilla.mozilla.org/show_bug.cgi?id=778377

Phase 0 has two feedback surveys: one for Firefox desktop and one for Firefox mobile.

For now, we're going to go with one table per survey shape.

When we add a new product/channel, we'll see if it has the same shape as another table we've got and if so, use that. If not, we'll figure out the best way to modify the existing system to meet the new needs.

We do want to minimize the amount of development work required to add a new product/channel, but work to minimize that work will be done in the future.

For now, adding a new product/channel will probably require:

1. developer figures out whether we can use an existing table or whether we need to create a new one
2. developer creates templates

Future thoughts:

If this happens a lot, then this is painful and we should work on scaffolding that reduces the work required. Maybe writing a ./manage.py command that fills out boilerplate or something.

We're not currently planning to build a generic survey hosting system.

Other things to think about

A note about the future: I'm hoping to build a separate project/product in conjunction with the Firefox engineering team where users are prompted for specific information (like "on a scale of 1-5 how would you rate your Firefox experience in terms of speed over the last month"). They'll be then asked if they want to provide more details and if so, get sent to input. But we'll want to be able to correlate those response as well as build the framework to support this kind of surveying. It's hugely future work but if that kind of plan influences how you design the system today, there you have it.

Maybe we should think about this as a metadata-enhanced general purpose survey system? Support different kinds of input fields. Support data coming from the url and http headers.

When doing analysis and running a search query, if it times out, the system could turn it into a task done later and the results emailed to the analyzer.

Email reports to subscribed people at specified intervals.

Email trend alerts.

On the thank you page after feedback, support suggestions from SUMO based on self-categorization or text analysis

For the mobile interface, you fill out your email address and it captures the metadata, then sends you a url you visit with your desktop browser to finish filling out the feedback---maybe this will help with getting better text from mobile users.

Automatic emails when users submit feedback

Automatic translation of feedback and/or aggregation to allow for community translation.

Spam controls (as needed).