Overview

Purpose

The purpose of this wiki is to serve as a general test plan for verifying that Fennec provide a feedback dialog when users turn off Content Notifications

Ownership

• Developer contact: X
• QA: Sorina Florean [:sorina]

Overall Status

[NOT LANDED] Nightly 50: NA

[NOT LANDED] Aurora 50: NA

[NOT LANDED] Beta 50: NA

Testing Summary

Scope of testing

The testing will be focus on

• verifying that users can provide feedback why they've turned off content notifications
• verifying that once feedback is submitted there is a chance to still receive a tip or hint or further information on the topic so that users might be able to change their option

Environments

Testing will be performed on following devices:

• Phones
  • 4.x
  • 5.x
  • 6.x
• Tablet
  • 4.x
  • 5.x
  • 6.x

Specifications

• As a Mozilla, we want to provide the user with a feedback channel to let us know why they purposely decided to opt-out of a feature
• As a Mozilla, we want to ask specific questions prompting this opt-out that will tell us why the user turned it off and if there is anything we can do to make the feature better
• As Mozilla, we want to collect and analysis the feedback about the opting out to make further product decision
• As Mozilla, we want to store this sentiment in the users profile so we can correlate this to further potential actions the user is doing, i.e. is she/he coming back to the future and turning it back on?

Requirements

• Include UI to start to showing and tracking heartbeat data inside Fennec
• Send a heartbeat ping with that data once the user turns off in settings, prompt them why they want to turn it off (canned answers)
• Content notifications as a our first run at Heartbeat with canned responses

User stories

• As a user, I want to provide feedback why I've turned off content notifications
• As a user, once I submitted my feedback would like to get a chance to receive a tip or hint or further information on the topic so that I might be able to change my mind (e.g. provide SUMO article)

Acceptance Criteria

• Heartbeat prompt shows up once user turns of content notifications under settings
• Heartbeat canned question and response is being presented to the user in an accessible way
• Heartbeat data is properly collected and accessible in re:dash

Success Criteria/KPIs

• Engagement rate: How many people turned off notification and did (not) react/communicates via Heartbeat prompt
• Product can decide what things are missing in the feature, what people frustrates about it and make product improvements, we could then run another heartbeat prompt on those users to check in and verify that the product has now become more useful to them
• Report that outlines most common reason users suppressed notification is X, Y, Z, which implies we need to improve the product in direction A,B,C (see Google doc table)

Additional Info

• Google doc with canned answers
  • https://bug1272719.bmoattachments.org/attachment.cgi?id=8752312

Testing details

Test Cases

• Test Case title

Testing days

Date: 2016-07-21

• Created Test Plan

Bug Work

• Meta bug
  • bug 1272717 - (heartbeat-content) [meta] Heartbeat for Content Notifications

1 Total; 0 Open (0%); 1 Resolved (100%); 0 Verified (0%);

Still issues

[NEW] Bug 1272719 - Add feedback dialog for turning off Content Notifications

Bug fixed

NA

Signoff

Criteria

• All test cases should be executed
• All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed

Results

Nightly testing NA

Aurora testing NA

Beta testing NA