QA/WebExtensions
Contents
WebExtensions: Manual Testing
Ownership
Developer contacts:
QA:
Krupa Raj, :krupa on IRC - leading all QA efforts across different products
Victor Carciu, :VictorC on IRC - leading QA Softvision team
Vasilica Mihasca, :vasilica_mihasca on IRC; - platform owner on Firefox Client side testing
Valentina Peleskei, :ValentinaP on IRC; - peer for Firefox Client side testing
Madalin Cotetiu, :madalinc on IRC; - peer for Firefox Client side testing
Cosmin Badescu, :CosminB on IRC; - peer for Firefox Client side testing
Status
Overview
[LANDED] Nightly 48: first landed, Firefox 48 is the first release of Firefox where we are recommending people start to use WebExtension
[ON TRACK] Developer Edition 48: tbd
[ON TRACK] Beta 48: tbd
[ON TRACK] Release 48: tbd
Risk analysis
Event | Probability | Impact | Current situation | Mitigation steps |
---|---|---|---|---|
WebExtensions fails to be installed on Firefox | probable | high | Not able to test yet. | Periodical regression testing will cover this area. |
WebExtensions are successfully installed using an incompatible Firefox version | probable | high | Not able to test yet. | Periodical regression testing will cover this area. |
WebExtensions are successfully installed but breaks the browser | improbable | high | Not able to test yet. | Periodical regression testing will cover this area. |
WebExtensions are successfully installed but the expected functionality is not meet | probable | high | Not able to test yet. | Periodical regression testing will cover this area. |
WebExtensions are successfully installed but there are encountered issues in terms of UI or performance | probable | high | Not able to test yet. | Periodical regression testing will cover this area. |
Summary
Context
About WebExtensions:
- are small software programs that can modify and enhance the functionality of the browser
- are a zipped bundle of files—HTML, CSS, JavaScript, images, and anything else you need
- are essentially web pages, and they can use all the APIs that the browser provides to web pages, from XMLHttpRequest to JSON to HTML5.
Enabling the feature
WebExtensions installation will be available by default starting with Firefox 48 and there is no pref to control this.
AMO
WebExtensions will be able to be uploaded to addons.mozilla.org then reviewed, searched, installed and updated just like any other Addon Bug 1210037
References
- Meta Bug 1161828
- First release tracking bug Bug 1214433
Test cases
Desktop tests will focus on:
- Install a Webextension on Firefox
- Use available functionality
- Compare Firefox's behaviour with Chrome's behaviour
- Uninstall is clean and permanent
Coverage
WebExtensions test cases will be executed according to each OS's percentage of our ADi numbers.
- All the webextensions will be tested across the Main OSes:
[1] Windows: Windows 7, Windows 10, Windows XP, Windows 8.1
[2] Mac: Mac OS X 10.11, Mac OS X 10.10, Mac OS X 10.9.5, Mac OS X 10.6
[3] Ubuntu: 14.04 LTS, 12.04 LTS
- The popular webextensions will be also tested across the Secondary OSes:
[1] Windows: Windows 8, Windows Vista
[2] Mac: Mac OS X 10.7, Mac OS X 10.6
Overview
Targeted scenarios
Testing focused on basic functionality
Testing focused on interoperability
Testing based on recent bugs and regressions
Testing with e10s
Bug work
Tracking bug
meta: Bug 1161828
- depends on: bugzilla query
Bug triage +/- for verification
Bug fix verification
Logged bugs
Sign off
Criteria
- All the test cases were executed.
- All the blocker, critical, major bugs have been fixed.
Results
WebExtensions: Automation
Tracking Bug
Contacts
Dave Hunt, :davehunt on IRC
John Dorlus, :silne30 on IRC
Krupa Raj, :krupa on IRC
Planned coverage for Firefox 48
must-have coverage
- Install a webextension from AMO - being tracked at bug 1252288
- Install a webextension from an external source - being tracked at bug 1252295
- Uninstall a webextension - being tracked at bug 1252312
- Test API calls for tabs - being tracked at bug 1252701