Apps/QA/OWA Extension
OWA Extension
| QA Lead: | Jason Smith |
| Lead Developer: | Mike Hanson |
Quick Start Guide
Here is a quick start guide to help you dive right in to Open Web Apps:
- Download and install the extension - Latest built can be found here
- Install apps - Sample apps can be found here
- View the Dashboard - Apps can be found by clicking the 'Apps' button at the bottom right of your browser or by going here
Once you have apps installed, you can launch them by clicking on them in the Dashboard. You can re-arrange the app icons and you can delete them from the Dashboard as well. You can also go to 'about:apps' (type this without the quotes in the URL bar) and launch apps there, view the manifest or delete apps.
Testing Initiative
Focus
- Extension (installing, deleting and launching apps)
- Dashboard
- App Sync
- Web Activities/Intents (not Q4)
The testing of the OWA extension will focus on the extension itself (for example, installation/deletion/compatibility and the mozApps.* api), on the dashboard (for example, adding/removing apps), on App Sync and on web activities. The idea is to determine the requirements/specifications of the extension and create manual test cases that will test the basics. Initially, the test cases will be subject to a great deal of changes depending on the requirements and will need to be kept up to date. This applies to the automation as well where keeping the automated test cases in sync with their corresponding manual test cases is essential. Further testing can be done using exploratory methods or a more advanced set of test cases. A proper way of getting apps that can be used for testing will be needed.
The basic iteration of the testing process will hopefully be as follows:
- Tester will install the latest build of the OWA extension
- Tester will verify any bugs that have been resolved on the latest extension build
- Tester will run automation tests to make sure no regressions have occured
- Tester will test using manual test cases and exploratory testing as well -- Once automation is in place, the tester can run the automation for the basic functionality testing
- Tester will file new bugs as necessary
- Tester will create new test cases as necessary and make sure the current test cases are up to date
NOTE: The extension will be available and will be needed at launch but eventually it will be built into the browser. Therefore, some of the test cases developed might not be needed anymore when this occurs.
Test Cases
A set of test cases is being created that focuses on manual testing strategies. The test cases document is actively being updated and can be found here. This document is split up into spreadsheets relating to a function/area to test. If there are any questions, comments or additions (other test cases you think are valuable but are not in the document) please feel free to contact David Clarke or Jason Smith.
There are quite a few questions that should be answered in order to have a good understanding of what the functionality of the extension and apps will be like. Answering these questions is crucial in having a solid set of manual test cases. The list of questions is being tracked here. Please feel free to update the document and add any other questions or answer some of the questions there already.
Native App Generation
Native app generation just means allowing the user to install apps natively on their machine. This is equivalent to installing a program or application. If you would like to check it out, please follow these steps (advanced) or simply install one of the extensions from here:
NOTE: Github links are out of date. Need to find links to point to mozilla central versions of the links.
- Install the addon-sdk (from github as the release version will not work) - https://github.com/mozilla/addon-sdk
- Download the oauthorizer package into the packages folder of the addon-sdk - https://github.com/mozilla/oauthorizer
- Clone the openwebapps project - https://github.com/mozilla/openwebapps
- In a terminal, cd into the addon-sdk folder and type: bin\activate (windows) or source bin/activate (linux)
- cd into openwebapps/addons/jetpack
- Type: cfx run - Note: Currently, using a release version of FF that is less than FF9 will cause an error relating to the icon. Therefore, either use FF9 (currently in beta) or use nightly or aurora by including the --binary command and passing in the path to firefox (.exe for windows) nightly/aurora (ex: cfx --binary="c:\Program Files\Nightly\firefox.exe" run).
Test Accounts
This will have test accounts that can be used for testing Open Web Apps.
Extension Builds
The latest builds for the OWA extension can be found here. The builds are created whenever a commit is made to the openwebapps github repo. The format is openwebapps-<commit id>-<date>.xpi. For automation, openwebapps.xpi within the 'latest' folder is used to avoid making changes to the addon.ini file for every test run. To build the extension yourself from the github repository, please do the following:
- git clone http://github.com/mozilla/openwebapps
- cd openwebapps
- make xpi (this will build the xpi file and place it in your current directory)
You can drag the xpi file to your Firefox window to install it. You can also host the file on the web and the user would just click on the xpi file to install it to their Firefox.
Bug Tracking/Reporting
This section will contain a list of all the bugs that we have filed or are tracking. Bugs are being logged in the Web Apps product in Bugzilla. Below is a quick link to the product to file a bug. There is also a tracking bug for the extension that is being used to track the progress of work and what needs to be done for an alpha release.
Bug Reporting Format:
Of course there are multiple ways of reporting a bug and what information to include or format to use can vary, this is just one way that is currently being used. Once you find a bug relating to Apps, you can do the following:
- Test the issue on all supported platforms
- Choose the right component under the Web Apps product in Bugzilla
- Choose the proper severity
- Select which OS the problem occurs for
- If you know who the issue should be assigned to, select the person from the 'Assignee' field. However, this is best left to the triage team if you are not sure.
- Enter a summary of the problem - Be specific and include all relevant information. For example, "Apps do not launch" is not a very effective summary. However, saying 'Apps do not launch from the Dashboard without having the apps extension installed" is a much better summary
- In the Description field:
- Enter a very detailed description of the problem and include any relevant information or observations
- Enter very detailed 'Steps to Reproduce' - This should be easy to follow and detailed enough that any person could reproduce the issue exactly as stated
- If possible, you should attach the xpi file you used to reproduce the problem. Alternately, you can say the following: 'Extension can be found at http://people.mozilla.com/~dclarke/openwebapps/Extension/ and look for <xpi file you used that contains the commit ID and the date>'
- Optionally, you could enter 'Alternate Steps to Reproduce' to show the different ways the problem occurs
- Enter the 'Actual Result' - Enter what happens right after the user performs the last step in the steps to reproduce
- Enter the 'Expected Result'
- Add any attachments as necessary
- This could include screenshots (highly recommended), videos or any files that you think are useful/helpful
Automation
The automation efforts will revolve around automating the manual test cases using mozmill. Since currently there is not much to work with, a basic skeleton framework is being looked into that can hopefully hold common API's that will be used when automating tthe test cases.
A demo test run was created with MozMill that installs an app, runs the app, closes the app and then deletes it. The patch for the demo test can be found here. Henrik Skupin is going to write a blog post about OWA automation and will include a demo that runs this script.
Test Server
An already existing nodejs server was modified to host test apps that we can use for load and smoke testing. To use the test server, please follow these steps (in linux):
Setup:
- git clone git://github.com/mozilla/openwebapps.git
- cd openwebapps
- Type: 'git submodule init' and then 'git submodule update'
- Make sure to have node installed - If not, you can get it from here. You can also follow one of the methods here.
Run:
- cd site/tests/
- Type: 'node run.js' - All necessary servers will start
- Launch the URL after 'Testing server started, to run tests go to:' and replace index.html with appstore.html
- A webpage will be shown that contains all the test apps
How It Works:
It's just a nodejs server that creates a separate server for each app that we have and hosts the necessary files within each server. The directory where the apps are stored is in site/tests/apps and each app has a folder within this directory.If you would like to add apps, all you have to do is add a folder within the apps directory and place the necessary files within that folder (please look at repo under site/tests/apps/ for examples). Please note the following:
- The manifest file must be called: manifest.webapp (this should be changed within the code to be more dynamic)
- If there is a problem with the page formatting, it's most likely because the icon pic used for the app is not compatible. The icon pic must be 128x128px in order to show up properly in the test server and in the Dashboard. You can use software such as gimp to adjust the pic size.
Documents and Links
QA Related
Firefox Store Apps Staging Server
Information/Tracking
WebRT - Useful information (can be used to create more test cases)
Explore
Infrastructure Requirements
Alternate Build Generation and Hosting
- A people.mozilla.com account has been created where the xpi file for the OWA extension will be placed as needed: mdabbagh@people.mozilla.com
- Developers or testers can make builds and place them there (for example, a bug is fixed and a quick check is needed using the developer's local code)
- Allows customization of builds
Getting Started:
To get started and to upload xpi files, you first need a people.mozilla.com account. Please follow this link in order to setup the account and to learn how to upload/download files.
If you only want to download the xpi files, please go here. If you would like an xpi file (extension) built, please send an email to mdabbagh@mozilla.com or ping mdabbagh in IRC (#qa, #openwebapps, #automation). You can also build the file yourself by following the instructions here (please use linux, Ubuntu for example, instead of Windows. It's much easier).
Marketplace instance managed by Al, limited access.