Symbol Exporting

From MozillaWiki
Jump to: navigation, search

QA/Symbols Exporting

Revision History

Date Version Author Description
11/10/2016 1.0 Abe Masresha Created first draft
11/10/2016 1.1 Grover Wimberly IV Fixed some wording in several sub-sections
1/13/2017 1.2 Abe Masresha Updated some links and added info on testing on android.



The goal of this work is to make Firefox crash less for release users.


This wiki details the testing that will be performed by the project team for the Symbol Exporting project. It defines the overall testing requirements and provides an integrated view of the project test activities. Its purpose is to document:

  • Testing will focus on:
Firefox launch and shutdown
Firefox child processes.
Sandboxing (content, GMP, plugin).
Exploratory testing for existing 3rd-party addons
Android basic functionality testing
  • How testing will be performed


Mozilla Development
Benjamin Smedberg

QA Eng. Softvision Las Vegas
Michelle ->Stefan - Team Lead ]
Abe Masresha - Test Lead
Grover Wimberly IV - Tester
Kanchan Kumari - Tester
Justin Williams - Tester

Testing summary

Scope of Testing

In Scope

This feature will be tested manually on Windows, OSX, Linux operating systems and Android devices.

Out of Scope

iOS Devices (Not affected by this work)
Automation will be handled by the developer (if needed)

Requirements for testing

Environments: Operating Systems

Windows 8.1
Windows 10
Windows 7
OSX 10.12

Firefox Channel and environment setups




Test Strategy

Test Objectives

Verify feature works correctly with Firefox launch and shutdowns
Verify feature works correctly with Firefox child processes
Verify feature works correctly with sandboxing (content, GMP, plugin)
Verify feature works fine with existing 3rd-party addons
Verify basic functionalities are working properly on Android

Test Execution Schedule

The following table identifies the anticipated testing period available for test execution.

Project phase Start Date End Date
Start project
Study documentation/specs received from developers 11/10/2016 11/10/2016
QA - Test plan creation 11/10/2016 11/10/2016
QA - Test cases/Env preparation 11/10/2016 11/16/2016
QA - tryBuild 11/15/2016
QA - Nightly 1/20/2017 1/20/2017

Testing Tools

Detail the tools to be used for testing, for example see the following table:

Process Tool
Test plan creation Mozilla wiki
Test case creation TestRail/ Google docs
Test case execution TestRail
Bugs management Bugzilla



Try Builds:



Testing on Android, retesting and bug verification: Nightly

Risk analysis


Test cases: gdoc or TestRail

Meta bug: Bug-1299187



Summary of testing scenarios

Test Areas

Test Areas Covered Details
Private Window Certain tests are performed in a private window.
Multi-Process Enabled Enabled by default
Multi-process Disabled To be tested
Theme (high contrast) Not applicable
Mouse-only operation To be tested
Keyboard-only operation To be tested
Display (HiDPI) Not applicable
Interraction (scroll, zoom) To be tested
Usable with a screen reader Not applicable
Usability and/or discoverability testing Is this feature user friendly
RTL build testing To be tested
Help/support interface required Make sure link to support/help page exist and is easy reachable.
Support documents planned(written) Make sure support documents are written and are correct.
Feature installs as expected To be tested
Does sync work across Install To be tested
Requires install testing To be tested
Affects first-run or onboarding Not applicable
Does this affect partner builds? Partner build testing Not applicable
Enterprise administration Not applicable
Network proxies/autoconfig Not applicable
ESR behavior changes Not applicable
Locked preferences Not applicable
Data Monitoring
Temporary or permanent telemetry monitoring List of error conditions to monitor
Telemetry correctness testing Not applicable
Server integration testing Not applicable
Offline and server failure testing Not applicable
Load testing Not applicable
Addon API required? Not applicable
Comprehensive API testing Not applicable
Permissions Not applicable
Testing with existing/popular addons To be tested
3rd-party security review Not applicable
Privilege escalation testing Not applicable
Fuzzing Not applicable
Web Compatibility depends on the feature
Testing against target sites To be Tested
Survey of many sites for compatibility Not applicable
Interoperability depends on the feature
Common protocol/data format with other software: specification available. Interop testing with other common clients or servers. Not Applicable
Coordinated testing/interop across the Firefoxes: Desktop, Android, iOS Desktop only
Interaction of this feature with other browser features Not Applicable

Test suite

Full Test suite - TestRail/ Google docs

Bug Work

Tracking bug – meta bug

Bug No Summary Status Firefox Verion
Bug 1318149 Crash Works for Me TryBuild/Nightly
Bug 1318175 Crash Works for Me TryBuild/Nightly
Bug 1317196 Netflix not working on Linux Known Issue Nightly
Bug 1318430 Norton Security Toolbar does not install in Firefox Known Issue Nightly
Bug 1317995 Firefox 50 64-bit Stable channel is unable to play Farmville 2 Facebook Flash game Known Issue Nightly
Bug 1308677 Flash warnings in Nightly are out of control Known Issue Nightly

Sign off


Check list

  • All test cases should be executed
  • Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan
  • All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)


Feature testing

  • Links for the tests run
    • Full Test suite: Link
    • Regression Test suite: [link] if needed/available


Exit Criteria Status Notes/Details
Testing Prerequisites (specs, use cases)
Testing Infrastructure setup
Test Plan Creation
Test Cases Creation
Full Functional Tests Execution
Automation Coverage
Performance Testing
All Defects Logged
Critical/Blockers Fixed and Verified
QA Signoff - Nightly Release
QA Aurora - Full Testing
QA Signoff - Aurora Release Email sent
QA Beta - Full Testing
QA Signoff - Beta Release