Revision History

This section describes the modifications that have been made to this wiki page. A new row has been completed each time the content of this document is updated (small corrections for typographical errors do not need to be recorded). The description of the modification contains the differences from the prior version, in terms of what sections were updated and to what extent.

Date	Version	Author	Description
02/17/2016	1.0	Brindusa Tot	Created first draft
06/14/2016	2.0	Iulia Cristescu	Updated several TP sections

Overview

Purpose

Detail the purpose of this document. For example:

• The test scope, focus areas and objectives
• The test responsibilities
• The test strategy for the levels and types of test for this release
• The entry and exit criteria
• The basis of the test estimates
• Any risks, issues, assumptions and test dependencies
• The test schedule and major milestones
• The test deliverables

Scope

This wiki details the testing that will be performed by the project team for the Firefox Support for Windows Child Mode project. It defines the overall testing requirements and provides an integrated view of the project test activities. Its purpose is to document:

• What will be tested
• How testing will be performed

Ownership

Developer contacts: David Keeler (irc: keeler)

Engineering contact: Matt Wobensmith (irc: mwobensmith)(irc: matt)

Engineering Team - Nightly Testing:
Rares Bologa (irc: RaresB) - PM for QA team

Brindusa Tot (irc: brindusat) - leading QA efforts

Simona Badau (irc: simonab) - QA

Kamil Jozwiak (irc: kjozwiak) - QA
Desktop Team - Release Testing:
Cornel Ionce (irc: cornel_ionce) - QA

Iulia Cristescu (irc: IuliaC) - QA

Testing summary

Scope of Testing

In Scope

Child mode is a feature of Windows that allows a parent (administrator) to set up a child account in order to monitor internet activity. It does so by using an SSL certificate that allows the operating system to spy or “MiTM” the child’s connections. While this helps the parent keep track of the child’s activity, it presents a fundamental problem for the way Firefox works. Unlike other browsers, Firefox does not use the system’s SSL certificates to verify connections, but instead, relies on its own certificate store. Since this Microsoft certificate is not present here, Firefox will see these connections as untrusted, and the user of the child account will not be able to view secure sites. The solution is to import this Microsoft certificate only when Firefox detects that it is running in child mode. Be aware that the the Microsoft Family Safety feature is only working with other browsers, like Firefox, on Windows 8.1.

The testing efforts will be invested on the following areas:

• Setup environment - create child account, import Microsoft certificate for Firefox, update Firefox configs
• Verify Firefox works with parental controls for the account (child or administrator) enabled/disabled
• Verify parental controls works as on other browsers(IE/Edge) if enabled for the account(web filtering settings, logging of child actions on administator account )

Out of Scope

- Functionality of the Microsoft Family Safety Mode with all available features is not included in the testing efforts. Only Firefox support for the Microsoft Child Mode feature on Windows 8.1 is tested.

Requirements for testing

Environments

The Microsoft Family Safety feature is available on Windows 8, Windows 8.1 and Windows 10, but Microsoft offers support for Firefox only on Windows 8.1 (On Windows 8 there is no MitM certificate; on Windows 10 only the native browsers like Edge and IE are supported).

Testing was divided in 2 big areas and was done on the following OS-es:

• Child Mode Functionality (Firefox behavior on a child account)
  • on Windows 8.1
• Firefox Basic Functionality (Firefox behavior on a child account and on an admin/parent account having the preference security.family_safety.mode=2)
  • Windows 8
  • Windows 8.1
  • Windows 10

Other configuration/environmental setup needed:

• In Firefox configuration settings need to add: security.family_safety.mode with values:
  • 0 - disables the feature entirely
  • 1 - enables detection of Family Safety mode but doesn't import the root (mostly for gathering telemetry/diagnostics)
  • 2 - enables detection and imports the root

Child Mode Support Matrix:

Operating System	Child Mode Support	Loads Secure Sites [Default Build]	Access Secure Sites [Try Build with Feature]
Win 10 x64	NO (Edge Only)	PASS (x64 & x86 fx builds)	PASS (x64 & x86 fx builds)
Win 10 x86	NO (Edge Only)	PASS (x86 fx builds)	PASS (x86 fx builds)
Win 8 x64	YES	PASS (x64 & x86 fx builds)	PASS (x64 & x86 fx builds)
Win 8 x86	YES	PASS (x86 fx builds)	PASS (x86 fx builds)
Win 8.1 x64	YES	FAIL (x64 & x86 builds)	PASS (x64 & x86 fx builds)
Win 8.1 x86	YES	FAIL (x86 builds)	PASS (x86 fx builds)
Win 7 x64	YES (Win Essentials)	PASS (x64 & x86 fx builds)	PASS (x64 & x86 fx builds)
Win 7 x86	YES (Win Essentials)	PASS (x86 fx builds)	PASS (x86 fx builds)
Last Test/Check:	3/22/2016

Quick Note: Win 8.1 is the only Windows OS that uses a MITM certificate under Child Mode accounts

Test Strategy

Test Objectives

This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master. This could be documented in bullet form or in a table similar to the one below.

Ref	Function	Test Objective	Evaluation Criteria	Test Type	Owners
1	Setup environment	The objective the test is trying to demonstrate Firefox works after setting up Windows child mode account	Navigation is allowed	Manual	Eng Team & Desktop Team
2	Firefox with parental control disabled	The objective of test is trying to demonstrate Firefox works with parental control disabled	Navigation is allowed to any website, Firefox works normally on both child and administrator	Manual	Eng Team & Desktop Team
3	Firefox with parental control enabled, no other settings done	The objective of test is trying to demonstrate Firefox works with parental control enabled	Navigation is allowed, but Adult content is blocked. Other Firefox functions works normally on both child and administrator	Manual	Eng Team & Desktop Team
4	Firefox with parental control enabled, allow specified sites	The objective of test is trying to demonstrate Firefox works with parental control enabled	Navigation is allowed only on specified sites. Other Firefox functions works normally on both child and administrator	Manual	Eng Team & Desktop Team
5	Firefox with parental control enabled, block specified sites	The objective of test is trying to demonstrate Firefox works with parental control enabled	Navigation is allowed, but blocked sites and Adult content are not allowed. Other Firefox functions works normally on both child and administrator	Manual	Eng Team & Desktop Team
6	Firefox with parental control enabled, set up different Web Filtering Levels	The objective of test is trying to demonstrate Firefox works with parental control enabled	Navigation is allowed, depending on the Filtering level.	Manual	Eng Team & Desktop Team
7	Firefox with parental control enabled, set up Firefox as a restricted app	The objective of test is trying to demonstrate Firefox cannot be opened if is set as restricted	Firefox cannot be opened if is set as restricted	Manual	Eng Team & Desktop Team
8	Firefox with parental control enabled, Logging disabled/enabled	The objective of this test is to validate logging on child activity	As administrator of child account I am able to see child actions	Manual	Eng Team & Desktop Team
9	Multiple different child accounts	The objective of this test is to child mode settings are preserved only on setup child account	If there are multiple accounts, we don’t want enabling/disabling child mode in one of them to affect how Firefox behaves in a other account	Manual	Eng Team & Desktop Team

Builds

This section should contain links for builds with the feature -

• Try builds with the fix are available to the the TryBuild1, TryBuild2, TryBuild3
• Links for Nightly builds - available since April 3th 2016 link
• Links for Aurora builds - available since April 26, 2016 link
• Links for Beta builds - available since June 6, 2016 link

Test Execution Schedule

The following table identifies the anticipated testing period available for test execution.

Project phase	Start Date	End Date
Start project	02/16/2016
Study documentation/specs received from developers	02/16/2016	02/19/2016
QA - Test plan creation	02/17/2016	04/01/2016
QA - Test cases/Env preparation	02/19/2016	02/26/2016
QA - Nightly Testing	04/04/2016	04/08/2016
QA - Aurora Testing	04/26/2016; 08/01/2016	06/03/2016; 09/12/2016
QA - Beta Testing	06/08/2016; 09/12/2016	Disabled till Fx 50; 11/07/2016
Release Date

Testing Tools

Detail the tools to be used for testing, for example see the following table:

Process	Tool
Test plan creation	Mozilla wiki
Test case creation	Google docs
Test case execution	Google docs
Bugs management	Bugzilla

Status

Overview

The support for the Windows Child Mode feature was added in Nightly with Bug 1239166 which landed in 2016-04-02. First available in this build.

Track the dates and build number where feature was merged to Aurora - 48.0a2 (2016-04-26) 
Track the dates and build number where feature was merged to Release/Beta - 48.0b1 build1 (20160606200529)

The feature was disabled by default in Firefox 48 and enabled starting with Firefox 50.

Risk analysis

Microsoft future updates on Family Safety feature, might affect the way Firefox behaves under Child Mode accounts or even on Standard accounts. Insecurity, due to the fact that Microsoft offered support also, for windows 10, but in the meanwhile they dropped it.

References

• Specs available are:

 Available specs - link
 Child Mode Support Matrix - link
 Windows 8.1 upgrade - link
 Microsoft Child Mode updates - link
 Microsoft Family Safety portal - link

• Meta bug 1259463
• Implementation bug 1239166

Testcases

Overview

All the test cases are available at the link

Kamil's testing is available at the link

Windows Child Mode 48 sign-off test cases

Windows Child Mode 50 testsuite in Testrail (need Testrail account)

Windows Child Mode - Mid-Aurora Sign off (Fx50) testsuite in Testrail (need Testrail account)

Windows Child Mode - Pre-Beta Sign off (Fx50) testsuite in Testrail (need Testrail account)

Windows Child Mode - Pre-Release Sign off (Fx50) testsuite in Testrail (need Testrail account)

Test Areas

Test Areas	Covered	Details
Private Window	yes
Multi-Process Enabled	yes
Multi-process Disabled	no
Theme (high contrast)	yes
UI
Mouse-only operation	no
Keyboard-only operation	no
Display (HiDPI)	no
Interraction (scroll, zoom)	no
Usable with a screen reader	no	e.g. with NVDA
Usability and/or discoverability testing	yes	Is this feature user friendly
Help/Support
Help/support interface required	no	Make sure link to support/help page exist and is easy reachable.
Support documents planned(written)	no	Make sure support documents are written and are correct. Please see the reference section.
Install/Upgrade
Feature upgrades/downgrades data as expected	no
Does sync work across upgrades	no
Requires install testing	no	separate feature/application installation needed (not only Firefox)
Affects first-run or onboarding	no	Florin/Lawrence are investigating if there is a dedicated QA for this, or we should test? Should be an yes/no and if is yes should add in detail column the team/person assigned.
Does this affect partner builds? Partner build testing	no	yes/no options, add comment with details about who will lead testing
Enterprise		Raise up the topic to developers to see if they are expecting to work different on ESR builds
Enterprise administration	no
Network proxies/autoconfig	no
ESR behavior changes	no
Locked preferences	no
Data Monitoring
Temporary or permanent telemetry monitoring	no	List of error conditions to monitor
Telemetry correctness testing	no
Server integration testing	no
Offline and server failure testing	no
Load testing	no
Add-ons		If add-ons are available for testing feature, or is current feature will affect some add-ons, then API testing should be done for the add-on.
Addon API required?	no
Comprehensive API testing	no
Permissions	no
Testing with existing/popular addons	yes	10 top rated and 10 most popular add-ons from AMO.
Security		Security is in charge of Matt Wobensmith. We should contact his team to see if security testing is necessary for current feature.
3rd-party security review	no
Privilege escalation testing	no
Fuzzing	no
Web Compatibility		depends on the feature
Testing against target sites	yes	10 top alexa sites
Survey of many sites for compatibility
Interoperability		depends on the feature
Common protocol/data format with other software: specification available. Interop testing with other common clients or servers.	no
Coordinated testing/interop across the Firefoxes: Desktop, Android, iOS	no
Interaction of this feature with other browser features	no

Test suite

Full Test suite - link

Child Mode 48 sign-off test cases

Windows Child Mode - Mid-Aurora Sign off (Fx50) testsuite in Testrail (need Testrail account)

Windows Child Mode - Pre-Beta Sign off (Fx50) testsuite in Testrail (need Testrail account)

Windows Child Mode - Pre-Release Sign off (Fx50) testsuite in Testrail (need Testrail account)

Bug Work

Tracking bug – bug 1259463

Sign off

Criteria

Check list

• All test cases should be executed on Windows 8.1 - Child Mode functionality and Windows 8, 8.1 and 10 - Firefox functionality
• No Blockers should be acceptable at any point in Aurora, Beta, Release
• No Critical issues beyond Beta - low level crashes may be considered Major
• More details about the release criteria on each phase are available at link.

Results

Nightly testing

List of OSes that will be covered by testing

• Link for the tests run
  • Full Test suite - link
  • Regression Test suite, if needed/available
    

Aurora testing

Mid-Aurora Sign-off
List of OSes that will be covered by testing: Windows 8.1 - Child Mode Functionality and Windows 10 - Specific Windows 10 Child Mode Tests

• Link for the tests run
  

Pre-Beta Sign-off
List of OSes that will be covered by testing: Windows 10 - Specific Windows 10 Child Mode Tests and Windows 8, 8.1 and 10 - Firefox Basic Functionality

• Link for the tests run
  

The feature was disabled by default in Firefox 48 and enabled starting with Firefox 50.

Mid-Aurora Sign-off
List of OSes that will be covered by testing: Windows 8.1 - Smoke, Exploratory and Regression Testing

• Testrail Link for the tests run (need Testrail account)
  
• Link for the Manual Testing Summary
  

Pre-Beta Sign-off
List of OSes that will be covered by testing: Windows 8.1 and Windows 10 - Smoke and Exploratory Testing

• Link for the tests run (need Testrail account)
  
• Link for the Manual Testing Summary
  

Pre-Release Sign-off
List of OSes that will be covered by testing: Windows 8.1 and Windows 10 - Smoke and Exploratory Testing

• Link for the tests run (need Testrail account)
  

Checklist

Exit Criteria	Status	Notes/Details
Testing Prerequisites (specs, use cases)	[DONE]
Testing Infrastructure setup	[DONE]
Test Plan Creation	[DONE]
Test Cases Creation	[DONE]
Full Functional Tests Execution	[DONE]
Automation Coverage	TBD
Performance Testing	N/A
All Defects Logged	[DONE]
Critical/Blockers Fixed and Verified	[DONE]
Metrics/Telemetry	N/A
QA Signoff - Nightly Release	[DONE]	Conditional SignOff email sent on 04.14.2016
QA Aurora - Full Testing	[DONE]	(AT RISK) Sign off Email sent on 05/23/2016 (Green) Sign off Email sent on 08/26/2016
QA Signoff - Aurora Release	[DONE]	(Green) Sign off Email sent on 06/03/2016 (Green) Sign off Email sent on 09/15/2016
QA Beta - Full Testing	[DONE]
QA Signoff - Beta Release	[DONE]	(Yellow) Sign off Email sent on 10/31/2016