Trigger suggester

Summary

Fjord has a suggestions framework that allows us to build providers that suggest links the user might be interested in based on the feedback the user has just left.

So far we have a single provider that queries SUMO for kb articles that might be relevant to the user's problems as indicated by their feedback.

There are a couple of additional use cases we'd like to cover:

1. Sees keywords in the feedback and asks the user to complete a survey for more information.
2. Sees keywords in the feedback and tells the user to go read our hot-fix workaround article.

This project covers implementing a trigger-rule based provider to handle these broken into several phases.

Potential use cases:

1. Identify video playback issues and solicit more information on specifics.
2. Identify webcompat issues and suggest user fill out a webcompat form.
3. Identify known hot-fix issue and send user to documentation on how to fix it.
4. Identify users with issues about a specific feature and send them to a survey with specific questions about their issues.

Status

• Project owner: Will Kahn-Greene
• Status: In-progress

History

• June 3rd, 2015: Landed the suggest framework and redirector for the Thank You page project
• June 16th, 2015: Wrote up bug 1175123 and over the course of the next week discussed it with User Advocacy and other folks
• June 30th, 2015: Turned it into a project.
• August 12th, 2015: Fleshed out Phase 1 after talking with stakeholders and mulling over requirements.
• September 9th, 2015: Finished critical mass of the new suggester and pushed it to production.
• September 25th, 2015: Tested the new system in production with a trigger rule for a broken addon.

Phase 1

Architecture

A suggestion is

This phase will focus on showing suggestions that have:

• title
• summary
• link url

Future phases can deal with different suggestion shapes.

Templated links

The link url will allow for templating with the following values:

• {LOCALE}
• {PRODUCT}
• {VERSION}

These values will be populated from the feedback response. If values are missing (e.g. the response has no version), then they will be empty strings.

Trigger rule criteria

Suggestion trigger criteria will have:

• one or more locales
• one or more products
• one or more product versions
• one or more key words/phrases
• have, don't have, don't care about url

These will be ORd inside a group (i.e. this locale OR that locale OR ...) and ANDd between groups (meets locales criteria AND meets products criteria AND ...).

Here are some examples of how I think it would work:

Example rule 1:

  locales=en-US
  products=Firefox
  keywords=rc4

This would match all feedback where the locale was "en-US" AND the product was "Firefox" AND the word "rc4" appeared in the description.

Example rule 2:

  locales=
  products=Firefox OS
  keywords=whatsapp

This would match all feedback from any locale AND the product was "Firefox OS" AND the word "whatsapp" appeared in the description.

Example rule 3:

  locales=es,pt
  products=Firefox,Firefox for Android
  keywords=word1,word2,word3,"this is a phrase"

This would match all feedback where the locale was "es" OR "pt" AND the product was "Firefox" OR "Firefox for Android" AND the word "word1", "word2", "word3" OR phrase "this is a phrase" appeared in the description.

Versions might be tricky since we don't always get version information. For now, we'll treat them like locales and products, but also allow for prefix matching with a syntax like "40*" which would match all versions that start with the string "40".

URL checking will be pretty basic for now and have three modes:

• don't care
• must have url
• must *NOT* have url

In a later phase, we can add domain-name matching or something like that.

Form to create/edit triggers

We'll build a form allowing analyzers to add, edit and disable triggers.

Tracking trigger usage

Further, we need to track:

• for a given trigger suggestion, how many people saw it?
• for a given trigger suggestion, how many people saw it and clicked on the link?

FIXME: Flesh this out.

Anti-goals

1. suggestions that don't fit the title/link/description shape
2. more complicated trigger criteria
3. trigger criteria involving other fields or tables (email address, remote troubleshooting, etc.)
4. translated text: If we want to create a trigger that has text to show in French, English and Spanish, we'll have to do three separate triggers with the appropriately translated text in the title/description slots; anything more than that requires a lot of stuff we don't have right now.

Nice-to-have

1. suggestion tester: Shows last x feedback that meet the trigger criteria. This would make it easier to build new triggers.

Requirements

1. implement trigger suggester provider which:
   1. looks at feedback response and all active triggers and figures out which suggestions to show
   2. posts GA event for each suggestion to show
   3. returns suggestions wrapped in appropriate redirector urls
2. implement trigger redirector provider which:
   1. converts redirects to urls
   2. posts GA event
   3. redirects user to outbound link
3. implement form to create, edit and disable triggers
4. tests for everything involved

Bugs

Tracker bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1198758

Full Query

ID	Summary	Priority	Status
1175123	keyword trigger suggestion provider	P3	RESOLVED
1198757	trigger suggester needs templated urls	--	RESOLVED
1198766	trigger rule tester	--	RESOLVED

3 Total; 0 Open (0%); 3 Resolved (100%); 0 Verified (0%);

Phase 2

Localize strings and open this up to non-en-US locales.

Can we open this up to non-Firefox desktop products as well? Do we need a setting in the Product model regarding whether to ping SUMO for relevant links and which SUMO product to search?

Future phases

Future phases will be figured out after the most recent phase is completed so we have time to act on what we've learned.

Future phases may include:

1. pulling solved questions (plus some other filters?) from SUMO?
2. pulling a mix of solved questions (plus some other filters?) and KB articles from SUMO?
3. get a kb article keyword -> article map for top 25 articles per locale and using that instead of doing lots of searches?