Confirmed users
9,511
edits
QA/Jetpack/AddonBuilderTestPlan: Difference between revisions
no edit summary
No edit summary |
|||
| (48 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
== Introduction == | == Introduction == | ||
Add-on Builder is the quick and easy way to develop an extension for Firefox using common web technologies. | The Add-on Builder is the quick and easy way to develop an extension for Firefox using common web technologies. The Builder is essentially a web app that wraps the Add-on SDK for developers who do not possess extensive experience developing add-ons for Firefox or are not familiar with console-based development environments. Get started by clicking [https://builder.mozillalabs.com/ here]. | ||
== | == Release Planning and Implementation == | ||
== | The linked document below is used for planning and development of specs for upcoming releases of the Add-ons Builder. See the features link in QA strategy section for the list of major features that have been implemented to date. | ||
[https://docs.google.com/document/edit?id=1q680sqwBaJ-0uz25tbi_osLJfi4UIBnBFtWAUVOHg2E&hl=en# Add-on Builder Product Development Plan] | |||
== Criteria to ship == | |||
The Builder can be pushed to production when the manual and automated tests pass based on the following criteria: | |||
* All links on the website are working. There should be no 404 errors. | |||
* The add-ons and libraries are being saved. | |||
* We are able to write the code for an add-on inside the Skywriter editor and download it to Firefox to ensure it works as expected. | |||
A test Add-on and Library will each be created that possess all the possible attributes of a fully featured package. Each item included in the Add-on or Library will be entered into the Builder and tested in a stepwise process that mocks the route a developer would take in creating such a package. See the QA strategy section below for details on how and what to test. | |||
== QA Strategy == | |||
The Add-on builder will be tested on different platforms and environments. | The Add-on builder will be tested on different platforms and environments. | ||
{| class="standard-table" style="width: 100%;" | |||
|- | |||
| class="header" | Browsers | |||
| class="header" | Platforms | |||
|- | |||
| Latest Firefox release | |||
| Mac OS X 10.6 and higher<br>Windows XP, Vista and 7<br>Linux | |||
|} | |||
There are two types of testing for Add-on builder.<br> | |||
'''=== Manual: ===''' | |||
Manual testing includes running Smoke tests, BFTs and FFTs. They should pass in order to release Add-on builder. We will use spot checks for feature testing to ensure that when new features land, they do not regress any functionality in the application and behave as expected.<br> | |||
Following is the list of the features that are implemented in the Add-on builder and its test cases. | |||
* [https://wiki.mozilla.org/QA/Jetpack/AddonBuilderTestPlan/design Features] | |||
<br> | |||
* [https://wiki.mozilla.org/QA/Jetpack/AddonBuilderTestPlan/Testcases Test cases] | |||
'''=== Automation ===''' | |||
GitHub repository for tests: https://github.com/ayanshah/FlightDeck-selenium/ | |||
Being a web application, Add-on builder is a great candidate for automation tests. Here, we can basically test the flow of creating an add-on through Add-on builder. We can also test the creation of libraries as well. | Being a web application, Add-on builder is a great candidate for automation tests. Here, we can basically test the flow of creating an add-on through Add-on builder. We can also test the creation of libraries as well. | ||
The flow can be - | The flow can be - | ||
| Line 34: | Line 49: | ||
- Typing in text editor for module | - Typing in text editor for module | ||
- (Optional)Save | - (Optional)Save | ||
- (Optional)Add attachment(s) | - (Optional)Add/Remove attachment(s) | ||
- (Optional) | - (Optional)Save | ||
- (Optional)Add libraries | - (Optional)Add/Remove libraries | ||
- Edit add-on | - (Optional)Upload a package. | ||
- Edit add-on information | |||
- Save | - Save | ||
- Download add-on | - Download add-on | ||
| Line 43: | Line 59: | ||
This is a sample example for automation flow. We will have scripts to test different features like badging verification, notification verification etc. | This is a sample example for automation flow. We will have scripts to test different features like badging verification, notification verification etc. | ||
* Tools used for automation | |||
** Selenium Server - Python client | |||
== | == Schedule == | ||
{| class="standard-table" style="width: 100%;" | {| class="standard-table" style="width: 100%;" | ||
|- | |- | ||
| class="header" | Milestone | | class="header" | Milestone | ||
| class="header" | Release Date | | class="header" | Tags | ||
| class="header" | Planned Release Date | |||
| class="header" | QA Sign off | | class="header" | QA Sign off | ||
| class="header" | Status | | class="header" | Browser Status | ||
| class="header" | Platform Status | |||
|- | |- | ||
| [https://wiki.mozilla.org/ | | [https://wiki.mozilla.org/AMO/FlightDeck/0.9.2#Timeline 1.0a0.9.2] | ||
| | | Maintenance Release | ||
| | | April 7 | ||
| '''Mac''': | | April 7 | ||
| '''Firefox''':4.0 | |||
| '''Mac''':Pass<br> '''Windows''':Pass<br> '''Linux''':Pass<br> | |||
|- | |- | ||
| [https://wiki.mozilla.org/ | | [https://wiki.mozilla.org/AMO/FlightDeck/0.9.3#Timeline 1.0a0.9.3] | ||
| | | Feature Release | ||
| April 28 | |||
| | | | ||
| '''Mac''': | | '''Firefox''':4.0 | ||
| '''Mac''':<br> '''Windows''':<br> '''Linux''':<br> | |||
|} | |} | ||
== Testing results == | |||
* Flightdeck 1.0a7 | |||
** [https://wiki.mozilla.org/QA/Jetpack/AddonBuilderTestPlan/1.0a7stage Stage Results] | |||
* Flightdeck 1.0a8 | |||
** [https://wiki.mozilla.org/QA/Jetpack/AddonBuilderTestPlan/1.0a8stage Stage Results] | |||
* Flightdeck 1.0a9 Maintenance release | |||
** [https://wiki.mozilla.org/QA/Jetpack/AddonBuilderTestPlan/1.0a9stage_maintenance Stage results] | |||