QA/Jetpack/AddonBuilderTestPlan

Introduction

The Add-on Builder is the quick and easy way to develop an extension for Firefox using common web technologies. The Builder is essentially a web app that wraps the Add-on SDK for developers who do not possess extensive experience developing add-ons for Firefox or are not familiar with console-based development environments. Get started by clicking here.

Release Planning and Implementation

The linked document below is used for planning and development of specs for upcoming releases of the Add-ons Builder. See the features link in QA strategy section for the list of major features that have been implemented to date.

Add-on Builder Product Development Plan

Criteria to ship

The Builder can be pushed to production when the manual and automated tests pass based on the following criteria:

• All links on the website are working. There should be no 404 errors.
• The add-ons and libraries are being saved.
• We are able to write the code for an add-on inside the Skywriter editor and download it to Firefox to ensure it works as expected.

A test Add-on and Library will each be created that possess all the possible attributes of a fully featured package. Each item included in the Add-on or Library will be entered into the Builder and tested in a stepwise process that mocks the route a developer would take in creating such a package. See the QA strategy section below for details on how and what to test.

QA Strategy

The Add-on builder will be tested on different platforms and environments.

There are two types of testing for Add-on builder.

=== Manual: ===

Manual testing includes running Smoke tests, BFTs and FFTs. They should pass in order to release Add-on builder. We will use spot checks for feature testing to ensure that when new features land, they do not regress any functionality in the application and behave as expected.
Following is the list of the features that are implemented in the Add-on builder and its test cases.

• Features

• Test cases

=== Automation ===

GitHub repository for tests: https://github.com/ayanshah/FlightDeck-selenium/

Being a web application, Add-on builder is a great candidate for automation tests. Here, we can basically test the flow of creating an add-on through Add-on builder. We can also test the creation of libraries as well. The flow can be -

- Login
- Click on create Add-on
- Typing in text editor for module
- (Optional)Save
- (Optional)Add/Remove attachment(s)
- (Optional)Save
- (Optional)Add/Remove libraries
- (Optional)Upload a package.
- Edit add-on information
- Save
- Download add-on
- Install to browser

This is a sample example for automation flow. We will have scripts to test different features like badging verification, notification verification etc.

• Tools used for automation
  • Selenium Server - Python client

Schedule

Testing results

• Flightdeck 1.0a7
  • Stage Results

• Flightdeck 1.0a8
  • Stage Results

• Flightdeck 1.0a9 Maintenance release
  • Stage results