Approvals Required / Received

The following individuals are required to/have approved this Test Plan:

Name	Title	Department	Approval Date	Method
	QA Manager	Product Integrity	Date	Email
	Software Engineer	Engineering	Date	Email
	EPM	Product Management	Date	Email

Revision History

This section describes the modifications that have been made to this wiki page. A new row has been completed each time the content of this document is updated (small corrections for typographical errors do not need to be recorded). The description of the modification contains the differences from the prior version, in terms of what sections were updated and to what extent.

Date	Version	Author	Description
24/05/2017	1.1	Mihai Boldan	Updated the Test Plan
13/06/2017	1.2	Mihai Boldan	Updated the Test Plan
10/07/2017	1.3	Mihai Boldan	mid-Nightly Sign off(YELLOW)
31/07/2017	1.4	Mihai Boldan	pre-Beta Sign off(YELLOW)

Overview

Purpose

This page details the testing that will be performed by Manual QA for the File read access restrictions sand boxing functionality. It defines the overall testing requirements and provides an integrated view of the project test activities. Its purpose is to document:

• Areas of risk
• What will be tested
• How testing will be performed
  • criteria
  • deliverables
  • ownership
  • schedule
• What data, if any, will be monitored

To help ensure the best possible release and long term maintenance of this feature.

Scope

This wiki details the testing that will be performed by the project team for the File Read Access Restrictions project. It defines the overall testing requirements and provides an integrated view of the project test activities. Its purpose is to document:

• What will be tested
  • Focus should be on the following:
    • Printing
    • Print to file
    • Clipboard interactions
    • file uploads to sites like google docs and photo sites
    • drag and drop of files into content
    • content file/page saves
    • add-ons:
      • WebExtensions
        • Popular WebExtensions still work
        • WebExtensions-developer use cases loading from home directory (i.e, about:debugging then load both packed and unpacked extensions)
      • Legacy
    • about:* pages are functional and links within access correctly
    • Flash (file picker)
    • Running browser outside of home directory, i.e., not installing it into proper location such as /Applications (on Mac), or Program Files (Windows)

• How testing will be performed
  • Manual

Ownership

Project Manager:
Jim Mathies (irc: jimm)
Development:
Bob Owen (irc: TBD)
Haik Aftandilian (irc: Haik)

Platform QA Team:
Tracy Walker (irc: TBD)
Mihai Boldan (irc: mboldan)
Bogdan Maris (irc: bogdan_maris)
Roxana Leitan (irc: RoxanaLeitan)
Ovidiu Boca (irc: Ovidiu)

Testing summary

Scope of Testing

In Scope

This section describes what is in scope for manual testing.

• Testing on most used OSs: Windows7x64, Windows10x64, Ubuntu16.04x64 and Mac 10.12
• Testing on different build structures:
  • Window 7x64 - Firefox build on 32-bit
  • Windows 10x64 - Firefox build on 64-bit
  • Mac OS X 10.12 - Firefox build on 64-bit
  • Ubuntu 16.04 - Firefox build on 64-bit
• Testing areas:
  • Printing
  • File uploading
  • Drag and Drop
  • Add-ons
  • Flash content
  • Accessibility
  • Virtual keyboard support
  • Content file/page saves
  • about:* pages
  • Interoperability
  • Downloads
• Test on different environments

Out of Scope

This section describes what is out of scope for manual testing.

• Testing on a high hardware variety
  • Separate test machines were configured for this feature
  • Only AMD 760G and ATI Radeon 3000 graphic cards available
• Testing on multiple Linux distribution systems
  • Testing will be performed only on Ubuntu OS

Requirements for testing

Environments

Full Testing will be performed using Beta build for Desktop with focus on platform specific issues as follows

• Windows
  • Roaming windows user
  • Running from a network drive
  • Profile on network drive
  • Profile on symbolic link or junction point
  • Windows Users directory on junction point
• Mac OS X
  • Profile in symlinked dir
  • ~ (home) in symlinked dir
• Linux
  • Profile in symlinked dir
  • .config in symlinked dir
  • ~ (home) in symlinked dir

Note: Read access restrictions on each platform will be slightly different. When creating test cases, any known differences should be called out in the test case.

Channel dependent settings (configs) and environment setups

Test Strategy

Risk Assessment and Coverage

ID	Description / Threat Description	Covered by Test Objective	Magnitude	Probability	Priority	Impact Score
RAC-1	Installing Firefox on a specific environment	Test Cases	3-High	2-Possible	3-High	18
RAC-2	Printing different file types from a specific environment	Test Cases	2-Moderate	2-Possible	2-Moderate	12
RAC-3	Install and run Legacy add-ons	Test Cases	2-Moderate,	1-Unlikely	1 - Low	2
RAC-4	Install and run Web extensions	Test Cases	3-High	3-Almost Certain	3-High	27
RAC-5	Uploading files on different websites	Test Cases	2-Moderate	2-Possible	3-High	12
RAC-6	Downloading different file types, using a specific environment	Test Cases	2-Moderate	2-Possible	3-High	12

Values:

• Magnitude: 1- Low , 2-Moderate, 3-High

• Probability: 1-Unlikely, 2-Possible, 3-Almost Certain

• Priority: 1 - Low, 2-Medium, 3-High

Impact Score Breakdown:

• An impact value of 1, 2, 3, 4 would describe an area which although should be covered there aren't expected any discoveries of critical issues.
• An impact value of 6, 8, 9, 12 would describe an area in which we expect to find issues but those issues are not expected to be critical.
• An impact value of 18 or 27 would describe an area on which it is likely to find issues and those issues to be critical or blockers.

Risk Analysis

Risk area	Requirement	Status
File Uploading	User has ability to upload files	TBD
Add-ons	File access properly restricted	TBD
about: pages	no regression in access of about: pages	TBD
3rd party hand-offs	TBD	TBD
Plugins (particularly Flash)	File access properly restricted	TBD
Profiles (user and Firefox) on "other" drives	File access properly restricted	TBD
Performance	No browser responsiveness regressions	TBD
Stability	No regressions in crash rate	TBD

Test Objectives

Criteria Description	Metric	Test results	Criteria Met?	QA Owner
Manual testing	Tests passed	-link to manual test results-	TBD (Date status updated)	Eng + Rel SV teams
Unit testing	Automated tests pass	perfherder	TBD (Date status updated)	TBD
Security	Security approves via review	-Link to security Analysis-	TBD (Date status updated)	TBD
Stability	Crash rate stable	-link to Stability Analysis-	TBD (Date status updated)	TBD

Builds

This section should contain links for builds with the feature -

• Links for Nightly builds
• Links for Beta builds - n/a

Test Execution Schedule

The following table identifies the anticipated testing period available for test execution.

Project phase	Start Date	End Date
Start project	Q1/2016	Q2/2017
QA - Test plan creation	Q2/2017	Q2/2017
QA - Test cases/Env preparation	22/05/2017	Q2/2017
QA - Nightly Testing	Q2/2017	Q2/2017
QA - Beta Testing	Q2/2017	Q3/2017
Release Date	FF56	2017-09-26

Testing Tools

Detail the tools to be used for testing, for example see the following table:

Process	Tool
Test plan creation	Mozilla wiki
Test case creation	TestRail/ Google docs
Test case execution	on Windows, on Mac OS X
Bugs management	Bugzilla

Status

Overview

Track the dates and build number where feature was released to Nightly - security.sandbox.content.level pref set to 3 by default from 56.0a1(22-07-2017)
Track the dates and build number where feature was merged to Release/Beta - TBD

References

• List and links for specs

 List and links for available specs - documents, user stories, specifications

• Meta bug

Testcases

Test Areas

Test Area	Covered	Notes
Multi-Process Enabled	Yes
Printing	Yes
Clipboard	Yes
Flash Plugin	Yes
Virtual Keyboard	Yes
Accessibility	Yes
Private Window	Yes
Install/Upgrade
Feature upgrades/downgrades data as expected	Yes
Does sync work	Yes
Enterprise		Profiles on network drives will be affected
Enterprise administration	Yes
Network proxies/autoconfig	Yes
ESR behavior changes	Yes
Data Monitoring
Temporary or permanent telemetry monitoring	Yes
Add-ons
Permissions	Yes
Testing with existing/popular addons	Yes
Web Compatibility		Focused on sites with file upload capability
Testing against target sites	Yes
Interoperability		Focus on third party hand-offs to/from Firefox
Common testing with other clients.	Yes

If it's not listed above, it is currently out of scope.

Test suite

Full Test suite - Test Rail
Smoke Test suite - TBD
Regression Test suite - TBD

Bug Work

Meta bug: TBD