This section describes the modifications that have been made to this wiki page. A new row has been completed each time the content of this document is updated (small corrections for typographical errors do not need to be recorded). The description of the modification contains the differences from the prior version, in terms of what sections were updated and to what extent.

Date	Version	Author	Description
09/19/2016	1.0	Alexandru Simonca Cristian Comorasu	Created first draft

Overview

Purpose

Detail the purpose of this document. For example:

• The test scope, focus areas and objectives
• The test responsibilities
• The test strategy for the levels and types of test for this release
• The entry and exit criteria
• The basis of the test estimates
• Any risks, issues, assumptions and test dependencies
• The test schedule and major milestones
• The test deliverables

Scope

This wiki details the testing that will be performed by the project team for the Restartless System Add-ons in Firefox. Its purpose is to document:

• What will be tested
• How testing will be performed

Ownership

Developer:
Robert Helmer (:rhelmer)

Leading QA efforts:
Alexandru Simonca (:asimonca)
Cristian Comorasu (:cristicomo)

Testing Summary

Scope of Testing

In Scope

The goal is for Firefox to update add-ons without the browser to restart in order to function properly.

Out of Scope

Testing functionality associated to specific system add-ons.

Requirements for testing

Environments

Platforms:

• Windows 7 x32
• Windows 8.1 x32
• Windows 10 x64
• Mac OS X 10.11
• Mac OX X 10.12.3
• Ubuntu 14.04 LTS x64
  

Update channels:

• release-sysaddon
• tbd

Test Strategy

Test Objectives

This section details the progression test objectives that will be covered.

Ref	Test Objective	Evaluation Criteria	Test Type	Owner
1	Restartlessly install an update to a sys add-on	“about:support” is accurate	Manual	Desktop Team
2	Restartlessly rollback to a sys add-on	“about:support” is accurate	Manual	Desktop Team
3	Restartlessly remove a sys add-on	“about:support” is accurate	Manual	Desktop Team
4	Restartlessly install System Addons - special cases	“about:support” is accurate	Manual	Desktop Team

Builds

This section should contain links for builds with the feature

• Link for Beta builds

Test Execution Schedule

The following table identifies the anticipated testing period available for test execution.

Project phase	Start Date	End Date
Start project	TBD	-
Study documentation/specs received from developers	TBD	TBD
QA - Test plan creation	TBD	TBD
QA - Test cases/Env preparation	TBD	TBD
QA - Nightly Testing	TBD	TBD
QA - Aurora Testing	TBD	TBD
QA - Beta Testing	TBD	TBD
Release Date	TBD	TBD

Testing Tools

Detail the tools to be used for testing, for example see the following table:

Process	Tool
Test plan creation	Mozilla wiki
Test case creation	TestRail
Test case execution	TestRail
Bugs Management	Bugzilla

Status

Overview

Track the date and build number when feature was merged to Aurora
Track the date and build number when feature was merged to Release/Beta

Risk Analysis

Identify the high-risk assumptions
Identify existing bugs on the feature with high risk
Identify if other areas are affected by the fix

Risk Areas	Mitigation
TBD	TBD

References

• Meta bug: 1204156 - Allow updated system add-ons to install without a restart
• Additional information can be found here

Testcases

• Summary of testing scenarios

Test areas

Test Areas	Covered	Details
Private Window	no
Multi-process enabled	yes
Multi-process disabled	no
Feature upgrades/downgrades data as expected	yes

Test Suite

Full Test Suite - Link with tests in TestRail

Bug Work

Tracking bug - meta bug

Sign Off

Criteria

• All test cases should be executed
• All blockers & critical must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)

Results

• Mid-Aurora sign-off
  • https://testrail.stage.mozaws.net/index.php?/plans/view/1737
• Pre-Beta sign-off
  • https://testrail.stage.mozaws.net/index.php?/plans/view/1962

Checklist

Exit Criteria	Status	Details
Testing Prerequisites (specs, use cases)	[DONE]
Testing Infrastructure setup
Test Plan Creation	[DONE]
Test Cases Creation	[DONE]
Full Functional Tests Execution	[DONE]
Automation Coverage	N/A
Performance Testing	N/A
All Defects Logged
Critical/Blockers Fixed and Verified	N/A
Daily Status Report (email/etherpad statuses/ gdoc with results)	N/A
Metrics/Telemetry	N/A
QA Signoff - Nightly Release	MISSED
QA Aurora - Full Testing	[DONE]
QA Signoff - Aurora Release	[DONE]
QA Beta - Full Testing	[DONE]
QA Signoff - Beta Release	[DONE]