QA/Symbolication Server
Revision History
| Date | Version | Author | Description |
|---|---|---|---|
| 12/06/2016 | 1.0 | Abe Masresha | Created first draft |
Contents
Overview
Purpose
The Symbolication service (symbolapi.mozilla.org)is an API used for symbolicating Firefox stacks. It uses data from symbols.mozilla.org to matches PC addresses to modules in memory and looks up the corresponding function names in server-side symbol files. The service will be used to process crashes.
Scope
This wiki details the testing that will be performed by the project team for the Symbolication server project. It defines the overall testing requirements and provides an integrated view of the project test activities.
- How testing will be performed
- Manual
Ownership
Mozilla Development
Developer: Kirk Steuber
QA Eng. Softvision Las Vegas
Stefan Georgiev - Team Lead ]
Abe Masresha - Test Lead
Grover Wimberly IV - Tester
Kanchan Kumari - Tester
Justin Williams - Tester
Testing summary
Scope of Testing
In Scope
This feature will be tested manually on Windows, OSX and Linux operating systems.
Out of Scope
Android & iOS Devices
Automation will be handled by the developer
Requirements for testing
Environments: Operating Systems
Windows 8.1 Windows 10 Windows 7 Ubuntu/Linux OSX 10.12
Firefox Channel and environment setups
Build
[Link to build]
Test Strategy
Test Objectives
Test Execution Schedule
The following table identifies the anticipated testing period available for test execution.
| Project phase | Start Date | End Date |
|---|---|---|
| Start project | ||
| Study documentation/specs received from developers | 12/06/2016 | |
| QA - Test plan creation | 12/06/2016 | |
| QA - Test cases/Env preparation | ||
| QA - tryBuild/build |
Testing Tools
Detail the tools to be used for testing, for example see the following table:
| Process | Tool |
|---|---|
| Test plan creation | Mozilla wiki |
| Test case creation | [TestRail]/ [ Google docs] |
| Test case execution | [TestRail] |
| Bugs management | Bugzilla |
Status
Overview
Try Builds:
Mozilla wiki
[build link]
Risk analysis
References
Test cases: [link]
Meta bug: Bug-1316298
Testcases
Overview
Summary of testing scenarios
Test Areas
| Test Areas | Covered | Details |
|---|---|---|
| Private Window | Certain tests are performed in a private window. | |
| Multi-Process Enabled | Enabled by default | |
| Multi-process Disabled | To be tested | |
| Theme (high contrast) | Not applicable | |
| UI | ||
| Mouse-only operation | To be tested | |
| Keyboard-only operation | To be tested | |
| Display (HiDPI) | Not applicable | |
| Interraction (scroll, zoom) | To be tested | |
| Usable with a screen reader | Not applicable | |
| Usability and/or discoverability testing | Is this feature user friendly | |
| RTL build testing | To be tested | |
| Help/Support | ||
| Help/support interface required | Make sure link to support/help page exist and is easy reachable. | |
| Support documents planned(written) | Make sure support documents are written and are correct. | |
| Install/Upgrade | ||
| Feature installs as expected | To be tested | |
| Does sync work across Install | Not Applicable | |
| Requires install testing | To be tested | |
| Affects first-run or onboarding | Not applicable | |
| Does this affect partner builds? Partner build testing | Not applicable | |
| Enterprise | ||
| Enterprise administration | Not applicable | |
| Network proxies/autoconfig | Not applicable | |
| ESR behavior changes | Not applicable | |
| Locked preferences | Not applicable | |
| Data Monitoring | ||
| Temporary or permanent telemetry monitoring | List of error conditions to monitor | |
| Telemetry correctness testing | Not applicable | |
| Server integration testing | To be tested | |
| Offline and server failure testing | To be tested | |
| Load testing | Not applicable | |
| Add-ons | ||
| Addon API required? | Not applicable | |
| Comprehensive API testing | Not applicable | |
| Permissions | Not applicable | |
| Testing with existing/popular addons | To be tested | |
| Security | ||
| 3rd-party security review | Not applicable | |
| Privilege escalation testing | Not applicable | |
| Fuzzing | Not applicable | |
| Web Compatibility | depends on the feature | |
| Testing against target sites | To be Tested | |
| Survey of many sites for compatibility | Not applicable | |
| Interoperability | depends on the feature | |
| Not Applicable | ||
| Coordinated testing/interop across the Firefoxes: Desktop, Android, iOS | Desktop only | |
| Interaction of this feature with other browser features | Not Applicable |
Test suite
Full Test suite - [TestRail]
Bug Work
Tracking bug – bug 1316298
Bugs
| Bug No | Summary | Status | Firefox Verion |
|---|---|---|---|
| Bug | Build | ||
| Bug | Build |
</div>
Sign off
Criteria
Check list
- All test cases should be executed
- Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan
- All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)
Results
Feature testing
- Links for the tests run
- Daily Smoke: [link]
- Full Test suite: [Link]
- Regression Test suite: [link] if needed/available
Checklist
| Exit Criteria | Status | Notes/Details |
|---|---|---|
| Testing Prerequisites (specs, use cases) | ||
| Testing Infrastructure setup | ||
| Test Plan Creation | ||
| Test Cases Creation | ||
| Full Functional Tests Execution | ||
| Automation Coverage | ||
| Performance Testing | ||
| All Defects Logged | ||
| Critical/Blockers Fixed and Verified | ||
| Metrics/Telemetry | ||
| QA Signoff - Nightly Release | Email to be sent | |
| QA Aurora - Full Testing | ||
| QA Signoff - Aurora Release | Email to be sent | |
| QA Beta - Full Testing | ||
| QA Signoff - Beta Release | Email to be sent |
