Approvals Required / Received

The following individuals are required to/have approved this Test Plan:

Name	Title	Department	Approval Date	Method
	QA Manager	Product Integrity	Date	Email
	Software Engineer	Engineering	Date	Email
	EPM	Product Management	Date	Email

Revision History

This section describes the modifications that have been made to this wiki page. A new row has been completed each time the content of this document is updated (small corrections for typographical errors do not need to be recorded). The description of the modification contains the differences from the prior version, in terms of what sections were updated and to what extent.

Date	Version	Author	Description
14/06/2017	1.0	Ciprian Georgiu	Created first draft
28/08/2017	1.1	Ciprian Georgiu	Updates

Overview

The feature's propose is to run the Network Monitor in the browser tab (Launchpad). The Launchpad makes it easy to build a developer tool for Firefox, Chrome, and Node. The Network Monitor uses webpack and several packages from devtools-core to run the Network Monitor as a normal web page. The Network Monitor uses Mozilla remote debugging protocol to fetch result and execute commands from Firefox browser.

Purpose

This document describes the following topics:

• the test scope, focus areas and objectives
• the test responsibilities
• the test strategy for the levels and types of test for this release
• the entry and exit criteria
• the basis of the test estimates
• any risks, issues, assumptions and test dependencies
• the test schedule and major milestones
• the test deliverables

Scope

This wiki details the testing that will be performed by the project team for the Netmonitor Phase II (Launchpad) project. It defines the overall testing requirements and provides an integrated view of the project test activities. Its purpose is to document what will be tested and how testing will be performed.

Ownership

Engineering contact(s):

• Marco Mucci (:MarcoM)
• Jan Honza Odvarko (:Honza)
• Ricky Chien
• Fred Lin

QA contact(s):

• Florin Mezei (:FlorinMezei) - Project Manager of the Release QA Team
• Andrei Vaida (:avaida) - Team lead of Release QA Team
• Ciprian Georgiu (:ciprian_georgiu) - leading QA efforts
• Emil Ghitta (:emilghitta) - leading QA efforts (peer)

Testing summary

Scope of Testing

In Scope

The Network Monitor (netmonitor) shows you all the network requests Firefox makes (for example, when a page is loaded or when an XMLHttpRequest is performed), how long each request takes, and details of each request. You can edit the method, query, header and resend the request as well. Read this MDN (https://developer.mozilla.org/en-US/docs/Tools/Network_Monitor) article to learn how to use the tool.

The testing effort for the Netmonitor Phase II (Launchpad) will be invested on the following areas:

• Netmonitor runs as intended in the browser tab (Launchpad) and the Launchpad interface is displayed when opening localhost:8000
• Devtools Launchpad communicates properly with Firefox (the remote debugging server) and list all opened tabs from Firefox
• the Toolbar is displayed well at the top of the main window and it provides all the functionality according to the existing requirements
• the feature lists all the network requests made in the course of loading the page and each request is displayed in its own row
• Netmonitor is cleared by default each time Firefox is navigating to a new page or the current page is releaded
• Netmonitor request columns has the intended behavior when toggling different columns
• Netmonitor request details open as intended a new pane and has the following pages: Headers, Cookies, Params, Response, Timings, Security (only for secure pages) Preview.
• Context menu and Submenu works as expected and the functionality is intact
• Hide/show request details panel button work well when Firefox is resized
• Statistics panel shows the correct information and it can be open from Performance analysis icon
• Netmonior works as intended when other themes are applied

Out of Scope

Testing will not check:

• how this feature works on a Hidpi configuration
• if this feature works with RTL locales
• whether the feature works well when Launchpad is loaded in Chrome
• Netmonitor opened in Devtools

Environments

• Operating systems:
  • Windows 10 x64 and Windows 7 x86
  • Mac OS X 10.11.6
  • Ubuntu 16.04 x64 LTS

• Devices: n/a
• Other configuration/environmental setup needed:
  • documentation on how to load Net Monitor in Launchpad: http://searchfox.org/mozilla-central/source/devtools/client/netmonitor/README.md

Channel dependent settings (configs) and environment setups

In order to run the Network Monitor in the browser tab, the following packages are required:

• node >= 6.9.x
• npm >= 3.x
• yarn >= 0.21.x
• Firefox either the released version or build from the source code.

Risk Assessment and Coverage

ID	Description / Threat Description	Covered by Test Objective	Magnitude	Probability	Priority	Impact Score
RAC-1	Netmonitor (Launchpad) Details panel (Header, Cookies, Params, Response etc.) are not showing the correct information for the selected request	TO-1, TO-2, TO-3, TO-4, TO-5, TO-6	3-High	3-Almost Certain	3-High	27
RAC-2	Filtering requests (by content type, by whether they are XMLHttpRequests or WebSocket requests, by URL, or by request properties) are not working correctly	TO-7	2-Moderate	3-Posibble	2-Medium	12
RAC-3	Context menu is not displayed when right clicking the list requests	TO-9	2-Moderate	2-Possible	3-High	12
RAC-4	Context menu and Submenu not saving the correct data for a selected request	TO-10, TO-9	2-Moderate	2-Possible	3-High	12
RAC-5	Netmonitor (sub)checkboxes cannot be accessed with mouse	TO-10	2-Moderate	1-Unlikely	3-High	6
RAC-6	Netmonitor may not run in Launchpad and the interface is not displayed		2-Moderate	1-Unlikely	3-High	6

Values:

• Magnitude: 1- Low , 2-Moderate, 3-High

• Probability: 1-Unlikely, 2-Possible, 3-Almost Certain

• Priority: 1 - Low, 2-Medium, 3-High

Impact Score Breakdown:

• An impact value of 1, 2, 3, 4 would describe an area which although should be covered there aren't expected any discoveries of critical issues.
• An impact value of 6, 8, 9, 12 would describe an area in which we expect to find issues but those issues are not expected to be critical.
• An impact value of 18 or 27 would describe an area on which it is likely to find issues and those issues to be critical or blockers.

Test Objectives

This section details the progression test objectives that will be covered. Please note that this is at a high level. For large projects, a suite of test cases would be created which would reference directly back to this master. This could be documented in bullet form or in a table similar to the one below.

Ref	Function	Evaluation Criteria	Test Objective	RAC	Test Type	Owners
TO-1	Netmonitor (Launchpad) Details panel (Header)	The tab lists basic information about the request: request URL, request method, remote IP address and port, status code with link to MDN doc (if available), the HTTP request and response headers that were sent and a button to edit and resend the request	Verifying that Detail panel (Header) displays the correct information for each request and if the request's method, URL, parameters, headers are resented.	RAC-1	Manual	Desktop Team
TO-2	Netmonitor (Launchpad) Details panel (Cookies)	The tab lists full details of any cookies sent with the request or response. As with headers, Cookies gives the option to filter the cookies.	Verifying that Detail panel (Cookies) displays the correct cookies for the selected request and check if filtering works as intended.	RAC-1	Manual	Desktop Team
TO-3	Netmonitor (Launchpad) Details panel (Params)	The tab displays the GET parameters and POST data of a request.	Verifying that the GET parameters and POST data of a request are displayed in the Params's panel.	RAC-1	Manual	Desktop Team
TO-4	Netmonitor (Launchpad) Details panel (Response)	The tab shows the corect response for the selected request type. (e.g.: HTML, JS, or CSS)	Verifying that response is shown as text for HTML, JS, or CSS, as an inspectable object for a JSON and a tab preview for a image.	RAC-1	Manual	Desktop Team
TO-5	Netmonitor (Launchpad) Details panel (Timings)	The tab breaks a network request down into the following subset of the stages defined in the HTTP Archive specification.	Verifying that the following specification: Blocked, DNS resolution, Connecting, Sending, Waiting, Receiving changes when a different request is clicked.	RAC-1	Manual	Desktop Team
TO-6	Netmonitor (Launchpad) Details panel (Security)	The tab is displayed if the site is being served over HTTPS.	Verifying that the tab contains details about the secure connection used including the protocol, the cipher suite, and certificate details.	RAC-1	Manual	Desktop Team
TO-7	Toolbar view buttons	The buttons filter the requests for each case: All, HTML, CSS, JS, XHR, Fonts, Images, Media, Flash, WS, Other and Filter Urls.	Verifying that all buttons works and requests are filtered for every scenarios.	RAC-2	Manual	Desktop Team
TO-8	Clear button	The button gives an option to clear all the requests at once.	Verifying that no requests are shown in Netmonitor after the button is pressed.	RAC-2	Manual	Desktop Team
TO-9	Context menu	Context-clicking on a row in the list displays the context menu and containing all options according to requests.	Verifying that functionality is intact and all the following options are displayed on context menu: Copy URL, Copy URL Parameters, Copy POST Data (only for POST requests), Copy as cURL, Copy Request Headers, Copy Response Headers, Copy Response, Copy Image as Data URI (only for images), Copy All As HAR, Save All As HAR, Save Image As (only for images), Edit and Resend, Open in New Tab, Start Performance Analysis for the page.	RAC-3	Manual	Desktop Team
TO-10	Submenu for Context menu	Submenu's options is opened form Context menu while hovering over Copy string.	Verifying that Submenu options works as intended as well as functionally. Copy URL, Copy URL Parameters, Copy POST Data, Copy as cURL etc, saves the correct data for each request.	RAC-4	Manual	Desktop Team
TO-11	Netmonitor checkbox	Netmonitor checkboxes gives the option to uncheck the columns as well as adding a new one.	Verifying that columns are added in netmonitor as intended and can be unchecked or reset to the default state.	RAC-5	Manual	Desktop Team

Builds

This section should contain links for builds with the feature:

• Links for latest Nightly builds
• Links for latest Beta builds

Test Execution Schedule

The following table identifies the anticipated testing period available for test execution.

Project phase	Start Date	End Date
Start project	6/12/2017	29/09/2017
Study documentation/specs received from developers	6/12/2017	6/13/2017
QA - Test plan creation	6/14/2017	6/14/2017
QA - Test cases/Env preparation	6/13/2017	6/13/2017
QA - Nightly Testing	7/7/2017	28/7/2017
QA - Beta Testing	08/22/2017	09/26/2017
Release Date	09/26/2017

Testing Tools

Detail the tools to be used for testing, for example see the following table:

Process	Tool
Test plan creation	Mozilla wiki
Test case creation	[1]/ Testrail
Test case execution	TestRail
Bugs management	Bugzilla

Overview

• Feature was released to Nightly: 55.0a1 (2017-04-10)
• Feature was merged to Beta: 56.0b1 (2017-08-07)
• Feature was merged to Release: TBD

References

• List and links for specs
  • run Netmonitor in Launchpad
  • MDN article

• Meta bug
  • bug 1348737 - (netmonitor-phaseII) [meta] Load Net Panel In Launchpad

Testcases

Test Areas

Test Areas	Covered	Details
Private Window	No
Multi-Process Enabled	Yes
Multi-process Disabled	Yes
Theme (high contrast)	No
UI
Mouse-only operation	Yes
Keyboard-only operation	No
Display (HiDPI)	No
Interaction (scroll, zoom)	No
Usable with a screen reader	No	e.g. with NVDA
Usability and/or discoverability testing	Yes	Is this feature user friendly
RTL build testing	No
Help/Support
Help/support interface required	Yes	Make sure link to support/help page exist and is easy reachable.
Support documents planned(written)	Yes	Make sure support documents are written and are correct.
Install/Upgrade
Feature upgrades/downgrades data as expected	No
Does sync work across upgrades	No
Requires install testing	No	separate feature/application installation needed (not only Firefox)
Affects first-run or onboarding	No	Florin/Lawrence are investigating if there is a dedicated QA for this, or we should test? Should be an yes/no and if is yes should add in detail column the team/person assigned.
Does this affect partner builds? Partner build testing	No	yes/no options, add comment with details about who will lead testing
Enterprise		Raise up the topic to developers to see if they are expecting to work different on ESR builds
Enterprise administration	No
Network proxies/autoconfig	No
ESR behavior changes	No
Locked preferences	No
Data Monitoring
Temporary or permanent telemetry monitoring	No	List of error conditions to monitor
Telemetry correctness testing	No
Server integration testing	No
Offline and server failure testing	No
Load testing	No
Add-ons		If add-ons are available for testing feature, or is current feature will affect some add-ons, then API testing should be done for the add-on.
Addon API required?	No
Comprehensive API testing	No
Permissions	No
Testing with existing/popular addons	No
Security		Security is in charge of Matt Wobensmith. We should contact his team to see if security testing is necessary for current feature.
3rd-party security review	No
Privilege escalation testing	No
Fuzzing	No
Web Compatibility		depends on the feature
Testing against target sites	No
Survey of many sites for compatibility	No
Interoperability		depends on the feature
Common protocol/data format with other software: specification available. Interop testing with other common clients or servers.	Yes
Coordinated testing/interop across the Firefoxes: Desktop, Android, iOS	No
Interaction of this feature with other browser features	No

Test suite

• Full Test suite - TestRail link
• Smoke Test suite - TestRail link
• Regression Test suite - n/a.

Bug Work

Meta bug: 1348737 - bug summary

Sign off

Criteria

Checklist

• All test cases should be executed
• Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan
• All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)

Results

Nightly testing

• List of OSes that will be covered by testing:
  
  • Windows 10 x64
  • Windows 7 x86
  • Ubuntu 16.04 x64
  • Mac OS X 10.11.6
• Link for the tests run - link

Merge to Beta Sign-Off

• List of OSes that will be covered by testing
  
  • Windows 10 x64
  • Windows 7 x86
  • Mac OS X 10.11.6
  • Ubuntu 16.04 x64
• Link for the tests run - link

Checklist

Exit Criteria	Status	Notes/Details
Testing Prerequisites (specs, use cases)	[DONE]
Testing Infrastructure setup	[DONE]
Test Plan Creation	[DONE]
Test Cases Creation	[DONE]
Full Functional Tests Execution	[DONE]
Automation Coverage	N/A
Performance Testing	[NOT APPLICABLE]
All Defects Logged	[DONE]
Critical/Blockers Fixed and Verified	[DONE]
Metrics/Telemetry	N/A
QA Signoff - Nightly Release	[DONE]	Email sent - YELLOW
QA Beta - Full Testing	[DONE]
QA Signoff - Beta Release	[DONE]	Email sent - GREEN