Apps/QA/Community Tasks
Overview
These are known tasks that we need help with on the Apps project with QA. To start on a task, fine one that sounds interesting and contact the listed mentor for that task.
Gecko vs. Webkit Evaluation
Mentors: Jason Smith and Aaron Train
Skills you will learn:
- <list skills here>
Summary:
<summarize here>
Subjective App Analysis
Mentor: Jason Smith
Skills you will learn:
- <list skills here>
Summary:
1. On an android device, download and install http://people.mozilla.org/~dclarke/openwebapps/Soup/latest/Soup.apk 2. On your android device, start an app titled "App Marketplace" 3. Login with a browser ID (if you don't have one with marketplace, use an account from https://etherpad.mozilla.org/testday-20120217) 4. Install an app titled "Sky Guide" (https://apps-preview.mozilla.org/en-US/apps/app/sky-guide) 5. Exploratory test the app by using the functionality provided by the app (e.g. if it has account management, create an account and login, click links)
Upon finishing testing of the app, provide the following information in this template and file bugs under Soup for bugs with Soup or Extension for bugs with the Extension. If you have any questions, please let me know.
Phone Quality Level: <Either excellent, good, fair, or poor>
Tablet Quality Level: <Either excellent, good, fair, or poor>
Web Desktop Quality Level: <Either excellent, good, fair, or poor>
Native Desktop Quality Level: <Either excellent, good, fair, or poor>
Test Account Required: <Does the app require a test account?>
Rationale:
<Why do you think the phone, tablet, web/native desktop quality level is at the level you specified? Such things to include are bugs in Soup/Extension preventing use of the app, problems with the app itself that the developer of the app should fix, whether a desktop site renders on a mobile device, etc.>
Action Items to Improve Quality for Developers of Soup and Extension:
<What should the developers of Soup and the Extension do to improve the quality of using the app? One example of this would be fixing certain bugs you have found with Soup and/or Extension>
Action Items to Improve Quality for Developer Who Created SkyGuide:
<What should the developers of the SkyGuide app do to improve the quality of using the app with the extension and/or Soup?>
1st Note: Apps preview marketplace will be going down sometime later in the week of March 4th. If it does go down and testing is still needed, please let me know, as David and I can add this app to a faker (allows websites to be used as apps) to allow the app to still be tested.
2nd Note: For specifying the quality level of the app, use the following specification:
Excellent
Cross-device support (works on all devices) Look and feel adapts to each device’s requirements (phone, tablet, desktop)
Good
App functionally works as expected (no functional errors on each device) Relatively usable across devices, even if look and feel doesn’t match each device’s requirements exactly (e.g. desktop only site, but functional on mobile, device screen-size) User responsiveness is good (pages load in a reasonable amount of time)
Fair
App partially works or is only partially usable (some clicks on app work, others generate incorrect behavior, rendering makes app hard to use) User responsiveness is not that great
Poor
App won’t render, won’t work entirely, unusable (doesn’t function correctly)
Soup Manual Testing
Mentor: Aaron Train
Skills you will learn:
- <list skills here>
Summary:
Run through the manual test cases for Soup listed here. These test cases span multiple sheets in this google document to cover testing soup installation, launching and running soup, installing apps, using the store, sync, purchasing apps, and other miscellaneous areas. After you run one or more test cases, send Aaron Train a message specifying what test case IDs you ran and the results of those test cases on the operating systems you ran the test cases on. Additionally, if an app was used during this case, specify what app was used. If you found any issues while testing, check to see if a bug was logged in the web apps bugzilla component. If there is no issue logged for an issue you found, log a bug here.
Develop New Soup Manual Test Cases
Mentor: Aaron Train
Skills you will learn:
- How to read an existing test cases document
- How to write a test case
- How to test an Android-based application
- How to capture missing test coverage in a system
Summary:
- Read the existing Soup test cases here
- Install the latest build of Soup here
- Play around with the Soup application and note any functionality areas that are not covered by existing test cases
- For the functionality areas not covered by test cases, construct test cases for them by providing the following:
- Summary: A short summary of what the test case is
- Steps to Perform: The steps to execute this test case
- Expected Results: The expected state of the system after running this test case
- Comments: Any additional information that needs to be specified
- Send these test cases created to Aaron Train for review
- If revisions are needed, make the revisions as needed and send it back for review
- For test cases passing review, Aaron Train will add them to the existing Soup test cases
Verify Fixed Bugs
Mentors: Jason Smith and Aaron Train
Skills you will learn:
- How to reproduce problems on past builds
- How to read an existing bug
- How to test the fixed bug in a desktop and android environment
- How to report test results on a bug verification
Summary:
- Use this bug query to find a bug to test
- Read the bug to understand the steps to reproduce the conditions to test
- Test the bug on an older build to verify you can see the bug (older builds are here)
- Test the bug on the most recent build to verify that it is fixed
- Send Jason Smith and Aaron Train a message containing the bug number and result