QA/Send Tab

From MozillaWiki
< QA
Jump to: navigation, search

Approvals Required / Received

The following individuals are required to/have approved this Test Plan:

Name Title Department Approval Date Method
RyanVM QA Reviewer Product Integrity n/a Email
Edouard Oger Software Engineer Engineering Email
Julie McCracken EPM Product Management Email

Revision History

This section describes the modifications that have been made to this wiki page. A new row has been completed each time the content of this document is updated (small corrections for typographical errors do not need to be recorded). The description of the modification contains the differences from the prior version, in terms of what sections were updated and to what extent.

Date Version Author Description
7/31/2017 1.0 Abe Masresha Created first draft
8/15/2017 1.1 Abe Masresha updated links and updated some sections
8/17/2017 1.2 Abe Masresha updated links and risk assesment section

Overview

Purpose

The purpose of this work is to make the send tab discoverable to non-Sync/FxA users.

Scope

  • Testing will focus on:
Stability of the browser with this work.
If send tab is discoverable to non-Sync/FxA users (Desktop only - at this time)
If user able to send links to other devices when signed-in to sync
  • How testing will be performed:
Manual
  • Automation is covered by the developer
[Link to automation test script will be provided here]

Ownership

Mozilla Development
Edouard Oger

QA Eng. Softvision Las Vegas
Abe Masresha

Testing summary

Scope of Testing

In Scope

This feature will be tested manually on Windows, Mac, and Linux operating systems and on iOS and Android devices.
Automation will be handled by the developer

Out of Scope

In fennec, the send tab may not be discoverable to non-Sync/FxA users as it is not implemented yet.

Requirements for testing

Test Environment Combinations

Windows 10 with iOS device
Windows 7 32bits with Linux OS
Linux with Android device
MacOSX 10.12 with Android device
Android with another Android device

Channel dependent settings (configs) and environment setups

Nightly

Firefox Nightly 56.0a1, no special configuration is required

Beta

Beta 56.0bx, no special configuration is required

Test Strategy

Risk Assessment and Coverage

ID Description / Threat Description Covered by Test Objective Magnitude Probability Priority Impact Score
RAC-1 The send tab feature may not send tab TO-1 3-High 2-Possible 3-High 18
RAC-2 The send tab may not be discoverable to non-Sync/FxA users (Desktop only - at this time) TO-2 3-High 2-Possible 3-High 18
RAC-3 The send tab feature may not implement as described in the mock. Mock TO-3 2-Moderate 2-Possible 2-Medium 8

Values:

  • Magnitude: 1- Low , 2-Moderate, 3-High
  • Probability: 1-Unlikely, 2-Possible, 3-Almost Certain
  • Priority: 1 - Low, 2-Medium, 3-High

Impact Score Breakdown:

  • An impact value of 1, 2, 3, 4 would describe an area which although should be covered there aren't expected any discoveries of critical issues.
  • An impact value of 6, 8, 9, 12 would describe an area in which we expect to find issues but those issues are not expected to be critical.
  • An impact value of 18 or 27 would describe an area on which it is likely to find issues and those issues to be critical or blockers.

Test Objectives

Ref Function Test Objective Evaluation Criteria Test Type RAC Owners
1 Unconfigured-Sync not loaded Check "Send page to Device" options within 10 seconds of the browser launch Check if "Not Connected to Sync" is displayed Manual RAC-3 Eng Team
2 Unconfigured-Sync loaded Check"Send Page to Device" options after 10 seconds of the browser launch Check if "Not Connected to Sync" is displayed Manual RAC-3 Eng Team
3 Signed-in - Sync not loaded Signed-in and check "Send page to Device" options within 10 seconds Check if "Syncing Devices ..." is displayed Manual RAC-3 Eng Team
4 Signed-in - Sync loaded Signed-in and check "Send page to Device" option after 10 seconds Check if your synced devices are listed Manual RAC-3 Eng Team
5 Signed-in but unverified If the user does not confirm email, sync should notify the user to verify Send tab feature notifies the user to verify email Manual RAC-3 Eng Team
6 Signed-in but password changed Signed-in to more than one devices and change password from one of them. Other device should prompt to verify or sign-in Check if user gets prompt from other device Manual RAC-3 Eng Team
7 Signed-in with only 1 device Signed-in with only 1 device and check if "No Devices Connected" is displayed "No Devices Connected" should be displayed Manual RAC-3 Eng Team
8 Signed-in (happy case) Check if user able to send tab using send tab feature Check if links are received by other devices Manual RAC1,RAC2 Eng Team
9 Signed-in / Unsendable link Some links like "about:support" are not sendable check some links are unsendable Manual RAC-3 Eng Team

Builds

This section should contain links for builds with the feature -

  • Links for Nightly build: use the latest official Nightly build
  • Links for Beta build: use the latest official beta build

Test Execution Schedule

The following table identifies the anticipated testing period available for test execution.

Project phase Start Date End Date
Start project 07/25/2017 IN PROGRESS
Study documentation/specs received from developers 07/25/2017 07/27/2017
QA - Test plan creation 7/31/2017 7/25/2017
QA - Test cases/Env preparation 7/25/2017 IN PROGRESS
QA - Nightly Testing 7/27/2017 7/31/2017
QA - Beta Testing IN PROGRESS IN PROGRESS
Release Date

Testing Tools

Detail the tools to be used for testing, for example see the following table:

Process Tool
Test plan creation Mozilla wiki
Test case creation TestRail
Test case execution TestRail
Bugs management Bugzilla

Status

Overview

Landed on Nightly on: 7/19/2017
Merged to Beta on: [date]

References

Testcases

Test Areas

Test Areas Covered Details
Multi-Process Enabled Yes The feature should work proprely in e10s browser
Add-ons
Testing with existing/popular addons yes Feature functionality will be tested with popular addons

Test suite

Full Test suite -Link to test rail
Smoke Test suite -Full test suite is used for pre-beta test cycle
Regression Test suite - Full test suite is used for pre-beta test cycle

Bug Work

Bug No Summary Status Firefox Version
1385440 Only one device receives the link when "Send to All Devices" is clicked VERIFIED FIXED 56
1385442 Android device is not listed in sync'd device list on mac RESOLVED DUPLICATE 56
1385451 in Android "send to other devices" is not displayed NEW 56
1385454 on Android, "verify your account" option is not available when the user changes password from other device NEW 56
1385458 Frozen spinner is shown when there are no devices to send tab to (iOS) NEW 56
1385442 In Android, Send Tab is not discoverable to non Sync/FxA users RESOLVED INVALID 56
1385447 All previously logged in devices are displayed in sync'd device list RESOLVED WORKSFORME 56

Sign off

Criteria

Checklist

  • All test cases should be executed
  • Has sufficient automated test coverage (as measured by code coverage tools)
  • All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)

Results

Nightly testing
List of OSes that will be covered by testing- listed in environments section

Merge to Beta Sign-off
List of OSes that will be covered by testing

  • Link for the tests run
    • Full Test suite

Checklist

Exit Criteria Status Notes/Details
Testing Prerequisites (specs, use cases) [DONE]
Testing Infrastructure setup [DONE]
Test Plan Creation [DONE]
Test Cases Creation [DONE]
Automation Coverage Will be covered by developer
Performance Testing Will be covered by developer
All Defects Logged [DONE] All found bugs are logged
Critical/Blockers Fixed and Verified IN PROGRESS Devs are fixing bugs
QA mid-Nightly Signoff [MISSED] Missed (test build and feature document were not avialable to QE)
QA pre-Beta Signoff [DONE] YELLOW (conditional sign-off) sent on 7/31/2017
QA Beta - Full Testing IN PROGRESS
QA pre-Release Signoff IN PROGRESS Email to be sent