Firefox/Input/Feedback to Bug: Difference between revisions
(Created page with "= Feedback to Bug = == Summary == Sometimes users leave feedback on Input that describes a particular bug and is detailed enough to serve as the basis for a Bugzilla bug. We...") |
No edit summary |
||
| (4 intermediate revisions by the same user not shown) | |||
| Line 3: | Line 3: | ||
== Summary == | == Summary == | ||
Sometimes users leave feedback on Input that describes a particular bug and is detailed enough to serve as the basis for a Bugzilla bug. We want Input analyzers to be able to click a button on the feedback response view that opens up Bugzilla with a bunch of fields pre-filled with data from the feedback response. | Sometimes users leave feedback on Input that describes a particular bug and is detailed enough to serve as the basis for a Bugzilla bug. We want Input analyzers to be able to click a button on the feedback response view that opens up Bugzilla with a bunch of fields pre-filled with data from the feedback response. We also want analyzers to be able to find feedback that is considered to "likely" be a bug easily. | ||
== Status == | == Status == | ||
| Line 14: | Line 14: | ||
* 2011: [[Firefox/TellUsMore|Tell Us More]] project is created to supplement the sad feedback form with an option for users to file Bugzilla bugs directly. | * 2011: [[Firefox/TellUsMore|Tell Us More]] project is created to supplement the sad feedback form with an option for users to file Bugzilla bugs directly. | ||
* December, 2015: Osmose is briefed on the project at Mozlando. | * December, 2015: Osmose is briefed on the project at Mozlando. | ||
* December 18th, 2015: Osmose writes this | * December 18th, 2015: Osmose writes this initial project plan. | ||
== Phase 1 == | == Phase 1 == | ||
| Line 20: | Line 20: | ||
=== Summary === | === Summary === | ||
This phase will | This phase will include adding a very basic "File a Bug" button to feedback pages, as well as making available a listing of feedback considered to be possible good bugs. | ||
=== Requirements === | === Requirements === | ||
| Line 27: | Line 27: | ||
#* Opens up Bugzilla in a new tab with the fields pre-filled out based on the feedback. | #* Opens up Bugzilla in a new tab with the fields pre-filled out based on the feedback. | ||
#* Determines the correct product/component based on the Product the feedback was left in. | #* Determines the correct product/component based on the Product the feedback was left in. | ||
# Add a listing (or modify an existing listing) to show feedback considered to be good bugs. | |||
#* Optionally add the "File a Bug" link directly to this listing. | |||
=== Input to Bugzilla Mapping === | === Input to Bugzilla Mapping === | ||
| Line 48: | Line 50: | ||
|} | |} | ||
=== | === "Good Bug" Filter === | ||
We have an algorithm that analyzes feedback text and produces a "helpfulness" score measuring the likelihood of the feedback being useful or just hate speech. The "Good Bug" filter should take this and a few other things into account: | |||
* Does it have a high helpfulness score? | |||
* Does it have an email address associated with the feedback? | |||
* Did the user provide about:support information with the feedback? | |||
=== | === Measuring success === | ||
If we use a custom event in GA for when users click the file a bug button, we should be able to compare button clicks to views of the feedback page to measure engagement. | |||
==== Question: Is the button being used? ==== | ==== Question: Is the button being used? ==== | ||
| Line 64: | Line 70: | ||
total views of the feedback page | total views of the feedback page | ||
Our expectation is that the button won't be used very often, so we're not entirely sure what a goal is here. However, Phase 1 is low enough effort that measuring isn't too critical. | |||
==== Question: Is the "Good Bug" filter effective? ==== | |||
This is basically asking how often feedback that is marked as a "good bug" is successfully converted into a bug. If we wanted to be super-accurate, we'd test a few different algorithms and compare their performance, but that is probably overkill in this case. | |||
We could measure this: | |||
clicked on the file a bug button | |||
---------------------------------------- | |||
feedback was seen in a "good bug" filter | |||
Which would give us a rough estimate of how often feedback that has been filtered was actually converted into a bug. | |||
=== Bugs === | === Bugs === | ||
Main bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1233866 | |||
<!-- | <!-- | ||
<bugzilla> | <bugzilla> | ||
{ | { | ||
"blocks": " | "blocks": "1233866" | ||
} | } | ||
</bugzilla> | </bugzilla> | ||
| Line 84: | Line 102: | ||
Future phases may include: | Future phases may include: | ||
# Adding a new listing of feedback that shows feedback that are considered likely good bugs. | |||
# Searching bugzilla for bugs related to the feedback response. | # Searching bugzilla for bugs related to the feedback response. | ||
# Automatically generate the bug instead of pre-filling out the form. | # Automatically generate the bug instead of pre-filling out the form. | ||
Latest revision as of 18:59, 21 December 2015
Feedback to Bug
Summary
Sometimes users leave feedback on Input that describes a particular bug and is detailed enough to serve as the basis for a Bugzilla bug. We want Input analyzers to be able to click a button on the feedback response view that opens up Bugzilla with a bunch of fields pre-filled with data from the feedback response. We also want analyzers to be able to find feedback that is considered to "likely" be a bug easily.
Status
- Project owner: None
- Status: Planning
History
- 2011: Tell Us More project is created to supplement the sad feedback form with an option for users to file Bugzilla bugs directly.
- December, 2015: Osmose is briefed on the project at Mozlando.
- December 18th, 2015: Osmose writes this initial project plan.
Phase 1
Summary
This phase will include adding a very basic "File a Bug" button to feedback pages, as well as making available a listing of feedback considered to be possible good bugs.
Requirements
- Add a "File a Bug" button to the feedback response view that:
- Opens up Bugzilla in a new tab with the fields pre-filled out based on the feedback.
- Determines the correct product/component based on the Product the feedback was left in.
- Add a listing (or modify an existing listing) to show feedback considered to be good bugs.
- Optionally add the "File a Bug" link directly to this listing.
Input to Bugzilla Mapping
| Input Field | Bugzilla Field |
|---|---|
| Feedback | Summary |
| URL | URL |
| Technical Info | Link in Summary? See also? |
| Email address | None (we're supposed to keep these private!) |
"Good Bug" Filter
We have an algorithm that analyzes feedback text and produces a "helpfulness" score measuring the likelihood of the feedback being useful or just hate speech. The "Good Bug" filter should take this and a few other things into account:
- Does it have a high helpfulness score?
- Does it have an email address associated with the feedback?
- Did the user provide about:support information with the feedback?
Measuring success
If we use a custom event in GA for when users click the file a bug button, we should be able to compare button clicks to views of the feedback page to measure engagement.
Question: Is the button being used?
This is measured by:
clicked on the file a bug button -------------------------------- total views of the feedback page
Our expectation is that the button won't be used very often, so we're not entirely sure what a goal is here. However, Phase 1 is low enough effort that measuring isn't too critical.
Question: Is the "Good Bug" filter effective?
This is basically asking how often feedback that is marked as a "good bug" is successfully converted into a bug. If we wanted to be super-accurate, we'd test a few different algorithms and compare their performance, but that is probably overkill in this case.
We could measure this:
clicked on the file a bug button ---------------------------------------- feedback was seen in a "good bug" filter
Which would give us a rough estimate of how often feedback that has been filtered was actually converted into a bug.
Bugs
Main bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1233866
Future phases
Future phases will be figured out after the most recent phase is completed so we have time to act on what we've learned.
Future phases may include:
- Adding a new listing of feedback that shows feedback that are considered likely good bugs.
- Searching bugzilla for bugs related to the feedback response.
- Automatically generate the bug instead of pre-filling out the form.