QA/Fennec/LeanPlum Push Notification

From MozillaWiki
< QA‎ | Fennec
Jump to: navigation, search

Overview

Purpose

The purpose of this wiki is to serve as a general test plan for verifying that the user is receiving push notification

Ownership

Developer contact: Nevin Chen [:nechen]

QA: Bogdan Surd [:bogdan] Sorina Florean [:sorina]

Overall Status

[LANDED] Nightly 57: 21-08-2017

[LANDED] Beta 57:

Testing Summary

Scope of testing

  • The testing will be focus on verifying that user is receiving push notifications

Environments

Testing will be performed on following devices:

  • Phones
    • Android 4 -> Android 8
  • Tablet
    • Android 4 -> Android 7

Specifications

User stories

  • As Mozilla, we want to control the target users we push messages to by country and by locale
  • As Mozilla, we want to push in app messages to users based on event triggers
  • As a user, when I click on the push notification or in app message, I want to be taken directly to the page where I can take immediate actions
  • As Mozilla, we want to push notifications (if we chooses to) to users even when Firefox is not running

Acceptance Criteria

Testing details

Test Cases

Testing days

Date:2017-08-30

Date:2017-08-31

  • Ran Test Cases
    • Filed Bugs:
      • bug 1395548 - [LeanPlum] - Campaign A no notifications received
      • bug 1395544 - [LeanPlum] - Campaign B no push notifications received
      • bug 1395551 - [LeanPlum] - Campaign C no push notifications received
      • bug 1395552 - [LeanPlum] - Campaign D no push notifications received
      • bug 1395553 - [LeanPlum] - Campaign F no push notifications received

Date:2017-09-01

Date:2017-09-04

  • Ran Test Cases
    • Filed Bug:
      • bug 1396548 - [LeanPlum] - Push Notifications don't work if the user closes Fennec

Date:2017-09-13

  • Pre-Beta Sign-Off Test Cases Status: Green
    • Filed Bugs:
      • bug 1399388 - [LeanPlum]: Top Sites panel open instead of Bookmarks Panel
      • bug 1399470 - [LeanPlum] - Campaign B not received for Note 4

Date:2017-10-18

  • Pre-Release Sign-Off Test Cases Status: Green
    • No new issues found

Bug Work

Tracking bugs

  • Meta bug
Full Query
ID Priority Summary Status Assigned to
1351573 P1 Replace Leanplum moz.build integration with Gradle sub-module or external dependency RESOLVED
1351577 P1 Integrate LeanplumPushReceiver/LeanplumPushListenerService RESOLVED
1351581 P1 Limit Leanplum to specific regions (via Switchboard) RESOLVED
1351583 P1 Initialize Leanplum SDK and set required values early RESOLVED
1351585 P1 Code review of open-source Leanplum SDK RESOLVED Nevin Chen(Not active on Bugzilla)
1356517 P1 Implement a centralized Deep Link handling mechanism RESOLVED Nevin Chen(Not active on Bugzilla)
1358117 P1 Allow User to reset their LeanPlum Tracking (Android) RESOLVED Nevin Chen(Not active on Bugzilla)
1361664 P1 Create MVP Triggers and Deeplinks for Mobile Marketing Automation RESOLVED Nevin Chen(Not active on Bugzilla)
1365089 P1 Add releng build files containing Leanplum SDK private app tokens RESOLVED Nick Alexander :nalexander [he/him]
1367314 P2 Multiple notification overlaps if the situation fulfill more than one rule RESOLVED
1368970 P1 [LeanPlum] - Tapping on device back button doesn't dismiss the prompt RESOLVED Vlad Baicu
1368971 P2 [LeanPlum] - Save as PDF infinite loop RESOLVED Nevin Chen(Not active on Bugzilla)
1368995 -- [LeanPlum] - Prompt not shown when download a media content RESOLVED
1369006 P2 [LeanPlum] - Default Browser is still triggered even if Firefox is set as the default browser RESOLVED
1369013 P2 [LeanPlum] - Keyboard not displayed when going back from the Search Engine trigger RESOLVED
1369312 P3 Multiple notifications are overlapped while tapping power key RESOLVED
1370515 P2 [LeanPlum] - Promote Sync triggered on account creation/login RESOLVED Nevin Chen(Not active on Bugzilla)
1373174 -- (Leanplum) - Change user identification method and add the general documentation for Leanplum RESOLVED Nevin Chen(Not active on Bugzilla)
1375931 -- (Leanplum) Update Event RESOLVED Nevin Chen(Not active on Bugzilla)
1377022 -- Refine the document for Leanplum RESOLVED Nevin Chen(Not active on Bugzilla)
1383761 -- [LeanPlum] - Promote focus is not triggered when opening a new tab after opening a private tab RESOLVED Nevin Chen(Not active on Bugzilla)
1383762 -- [LeanPlum] Focus promotion triggered by unrelated actions VERIFIED Nevin Chen(Not active on Bugzilla)
1383765 -- [LeanPlum] Tapping the "+" icon to open a new tab, doesn't trigger the Focus promotion RESOLVED Nevin Chen(Not active on Bugzilla)
1383768 -- [LeanPlum] - The user is not taken to the sign up page. RESOLVED Nevin Chen(Not active on Bugzilla)
1385819 -- [LeanPlum] Focus promotion still displayed after installing Focus RESOLVED Nevin Chen(Not active on Bugzilla)
1385853 P1 [LeanPlum] - Default browser is still triggered if app was set as default from another app RESOLVED Petru-Mugurel Lingurar [:petru]
1386151 -- Enable Push Notification feature for Leanplum RESOLVED Nevin Chen(Not active on Bugzilla)
1392224 -- Make Fennec the default handler for Leanplum Push Notification RESOLVED Nevin Chen(Not active on Bugzilla)
1392542 -- [Leanplum] Add a new trigger for App Start RESOLVED Nevin Chen(Not active on Bugzilla)
1392545 P1 [Leanplum] Add user attribute for Pocket installation RESOLVED Nevin Chen(Not active on Bugzilla)
1394321 P5 Document incident response process for Android RESOLVED
1395544 -- [LeanPlum] - Campaign B no push notifications received RESOLVED Nevin Chen(Not active on Bugzilla)
1395548 -- [LeanPlum] - Campaign A no notifications received RESOLVED
1395551 -- [LeanPlum] - Campaign C no push notifications received RESOLVED Nevin Chen(Not active on Bugzilla)
1395552 -- [LeanPlum] - Campaign D no push notifications received RESOLVED Nevin Chen(Not active on Bugzilla)
1395553 -- [LeanPlum] - Campaign F no push notifications received RESOLVED Nevin Chen(Not active on Bugzilla)
1396548 P1 [LeanPlum] - Push Notifications don't work if the user closes Fennec VERIFIED Nevin Chen(Not active on Bugzilla)
1397188 -- [LeanPlum] - Push Notifications also received for active users VERIFIED Nevin Chen(Not active on Bugzilla)
1397208 P1 [LeanPlum] Push Notification don't work on Asus ZenPad and Huawei Honor RESOLVED Vlad Baicu
1397241 -- [LeanPlum] Push notification - Install Pocket displayed even if the app is already installed VERIFIED
1397245 P1 [LeanPlum] - Pocket installation page not opened using Google Play RESOLVED Vlad Baicu
1399388 P1 [LeanPlum]: Top Sites panel open instead of Bookmarks Panel RESOLVED Vlad Baicu
1399470 P2 [LeanPlum] - Campaign B not received for Note 4 RESOLVED
1438716 P1 Upgrade Leanplum SDK RESOLVED Andrei Lazar
1481965 -- (Leanplum) - There is not currently an event set up for the "Opened App" trigger RESOLVED Andrei Lazar

45 Total; 0 Open (0%); 41 Resolved (91.11%); 4 Verified (8.89%);


Signoff

Criteria

  • All test cases should be executed
  • All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed

Results

Nightly testing

  • feature looks stable and push notification are received by user

Beta testing

  • feature looks stable and push notification are received by user
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Fennec/LeanPlum_Push_Notification&oldid=1182466"