QA/WebGL2
Revision History
This section describes the modifications that have been made to this wiki page. A new row has been completed each time the content of this document is updated (small corrections for typographical errors do not need to be recorded). The description of the modification contains the differences from the prior version, in terms of what sections were updated and to what extent.
Date | Version | Author | Description |
---|---|---|---|
2016/11/23 | 1.0 | Bogdan Maris | Created first draft |
2016/11/24 | 1.1 | Bogdan Maris | Updates |
Contents
Overview
Purpose
WebGL 2 is based on OpenGL ES 3.0, and brings with it many improvements and additions to help developers create stunning visuals on the Web. WebGL 2 will raise many restrictions and add new capabilities compared to WebGL 1. For example, while WebGL 1 only required support for being able to render using 8 textures at a time, WebGL 2 raises this minimum limit to 32.
(quote from Martin Best from Mozilla)
Scope
This wiki details the testing that will be performed for the WebGL 2.0 feature. It defines the overall testing requirements and provides an integrated view of the project test activities.
Ownership
Developer contact: Jeff Gilbert (:jgilbert on IRC)
QA Lead: Bogdan Maris mail, (:bogdan_maris on IRC)
Testing summary
Scope of Testing
In Scope
The goal is for Firefox to display the content for WebGL 2.0.
Out of Scope
- n/a
Requirements for testing
Environments
Testing will be performed on the following OSes (both x86 & x64 infrastructures):
- Windows 7
- Windows 10
- Ubuntu 16.04
- Mac OS X 10.11
- Mac OS X 10.12
Test Strategy
Test Objectives
This section details the progression test objectives that will be covered.
Ref | Function | Test Objective | Evaluation Criteria | Test Type |
---|---|---|---|---|
1 | WebGL 2.0 is enabled | Verify that the “webgl.enable-webgl2” is set to true in about:config | Observe the pref value | Manual |
2 | WebGL 2.0 functionality | Verify that WebGL 2.0 demos work | Visit demo | Manual |
3 | Private Browsing | Verify that the feature works while in private browsing | Visit demo | Manual |
4 | WebGL 2.0 is disabled | Set “webgl.enable-webgl2” to false | demo displays WebGL only and not WebGL 2.0 | Manual |
Builds
This section should contain links for builds with the feature -
Test Execution Schedule
The following table identifies the anticipated testing period available for test execution.
Project phase | Start Date | End Date |
---|---|---|
Start project | 2016/01/26 | - |
Study documentation/specs received | 2016/01/26 | 2016/11/24 |
QA - Test plan creation | 2016/01/27 | 2016/09/20 |
QA - Test cases/Env preparation | 2016/09/22 | 2016/10/11 |
QA - Nightly Testing | 2016/07/06 | 2016/07/11 |
QA - Aurora Testing | tbd | tbd |
QA - Beta Testing | 2017/01/11 | 2017/01/13 |
Release Date | tbd | tbd |
Testing Tools
Process | Tool |
---|---|
Test plan creation | Mozilla wiki |
Test case creation | TestRail |
Test case execution | TestRail |
Bugs management | Bugzilla |
Status
Overview
- Track the dates and build number where feature was released to Nightly
- Track the dates and build number where feature was merged to Aurora
- Track the dates and build number where feature was merged to Release/Beta
Risk analysis
Risk areas | Mitigation |
---|---|
External dependencies on WebGL 2.0 feature | WebGL 2.0 feature is dependent on the graphic card the user uses, it might not support this technology and also on the complexity of the demos tested. Some demos might be too demanding for the graphic card and might not be displayed properly |
References
- WebGL 2.0 demo in Firefox can be seen here.
- Test if your browser has WebGL 2.0: Here (If it says WebGL 2.0 it's working with WebGL 2.0).
- Main Wiki page for WebGL2 development.
- Original Firefox blogpost announcement of WebGL 2.0
Testcases
Overview
- Summary of testing scenarios
Test Areas
Test Areas | Covered | Details |
---|---|---|
Private Window | Yes | |
Multi-Process Enabled | Yes | |
Multi-process Disabled | Yes | |
Theme (high contrast) | No | |
UI | ||
Mouse-only operation | No | |
Keyboard-only operation | No | |
Display (HiDPI) | Yes | |
Interaction (scroll, zoom) | Yes | |
Usable with a screen reader | No | e.g. with NVDA |
Usability and/or discoverability testing | No | Is this feature user friendly |
Help/Support | ||
Help/support interface required | No | Make sure link to support/help page exist and is easy reachable. |
Support documents planned(written) | No | Make sure support documents are written and are correct. |
Install/Upgrade | ||
Feature upgrades/downgrades data as expected | No | |
Does sync work across upgrades | No | |
Requires install testing | No | |
Affects first-run or onboarding | No | |
Does this affect partner builds? Partner build testing | No | |
Enterprise | Raise up the topic to developers to see if they are expecting to work different on ESR builds | |
Enterprise administration | ||
Network proxies/autoconfig | ||
ESR behavior changes | ||
Locked preferences | ||
Data Monitoring | ||
Temporary or permanent telemetry monitoring | No | |
Telemetry correctness testing | No | |
Server integration testing | No | |
Offline and server failure testing | No | |
Load testing | No | |
Add-ons | If add-ons are available for testing feature, or is current feature will affect some add-ons, then API testing should be done for the add-on. | |
Addon API required? | ||
Comprehensive API testing | ||
Permissions | ||
Testing with existing/popular addons | Yes | |
Security | Security is in charge of Matt Wobensmith. We should contact his team to see if security testing is necessary for current feature. | |
3rd-party security review | ? | |
Privilege escalation testing | ? | |
Fuzzing | ? | |
Web Compatibility | depends on the feature | |
Testing against target sites | Yes | |
Survey of many sites for compatibility | No | |
Interoperability | depends on the feature | |
Common protocol/data format with other software: specification available. Interop testing with other common clients or servers. | ||
Coordinated testing/interop across the Firefoxes: Desktop, Android, iOS | ||
Interaction of this feature with other browser features | No |
Test suite
Bug Work
Meta bug - (webgl2) [meta WebGL2 Support]
Bug fix verification
Bug No | Summary | Status | Firefox Version |
---|---|---|---|
123 | bug summary | NEW | Nightly 45 |
Logged bugs
Bug No | Summary | Status | Firefox Version |
---|---|---|---|
Bug 1310219 | [10.12] Low FPS on WebGL 2.0 demo | NEW | Trunk |
Bug 1305035 | Crash in @ mozilla::WebGLTexture::CompressedTexImage | RESOLVED FIXED | Trunk |
Bug 1316533 | When printed, pages from National Geographic show a much larger logo instead of the normal one in Reader Mode | NEW | Trunk |
Bug 1316327 | Crash in mozilla::WebGLFramebuffer::BlitFramebuffer (Webgl2.0 regression) | NEW | Trunk |
Bug 1316569 | Crash in mozilla::WebGLSampler::SamplerParameter1f | RESOLVED DUPLICATE | Trunk |
Bug 1310247 | WebGL 2.0 demo not work having useTransformFeedback enabled | NEW | Trunk |
Bug 1316310 | WebGL2 transform_feedback demos are not displayed | RESOLVED DUPLICATE | Trunk |
Bug 1316294 | Wrong square size in webgl2 demo | NEW | Trunk |
Sign off
Criteria
Check list
- All test cases should be executed
- Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan
- All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)
Results
Nightly testing - 2016/09/23
- Link for the tests run
- Daily Smoke suite - N/A
- Full Test suite - TestRail
- Regression Test suite, if needed/available - N/A
Merge to Aurora Sign-off - 2016/09/23
- Link for the tests run -
- Full Test suite - TestRail
Mid Aurora Sign-off - 2016/10/19
- Link for the tests run -
- Full Test Suite - TestRail
Pre-Beta Sign-off - TBD
- Link for the tests run -
- Full Test Suite - TBD
Pre-Release Sign-off - TBD
- Link for the tests run -
- Full Test Suite - TBD
Checklist
Exit Criteria | Status | Notes/Details |
---|---|---|
Testing Prerequisites (specs, use cases) | [DONE] | |
Testing Infrastructure setup | No | |
Test Plan Creation | [DONE] | |
Test Cases Creation | [DONE] | |
Full Functional Tests Execution | [DONE] | |
Automation Coverage | ||
Performance Testing | ||
All Defects Logged | ||
Critical/Blockers Fixed and Verified | ||
Daily Status Report (email/etherpad statuses/ gdoc with results) | ||
Metrics/Telemetry | N/A | |
QA Signoff - Nightly Release | [DONE] | |
QA Aurora - Full Testing | [DONE] | |
QA Signoff - Aurora Release | [DONE] | |
QA Beta - Full Testing | ||
QA Signoff - Beta Release |