|
|
| (17 intermediate revisions by the same user not shown) |
| Line 1: |
Line 1: |
| = Test Plan OpenWebApps = | | = Platform Apps Testing = |
|
| |
|
| Author: David Clarke | | Author: David Clarke |
|
| |
|
| Date: 08/17/2011<br> | | Date: 05/30/2013 |
|
| |
|
| == Overview == | | == Overview == |
|
| |
|
| '''Open Webapps is a fairly overloaded term, but basically incorporates a few main pieces of infrastructure.''' | | '''Apps Based Testing Gaps''' |
|
| |
|
| *An App Store (based off of AMO)
| | == Automation == |
| **Paypal Chained API for payments
| | The immediate goal of this project was to identify areas of testing for Apps that have been missed by current automation methods. |
| *A set of Web API's that can be loosely defined as our App Cloud
| | The overview was completed as of 05/30/2013. The review process generally involved Jason Smith & Fabrice & Julien reviewing the strategy and evaluating these areas as being in need of being tested at the end of this effort. |
| *A browserID integration
| |
|
| |
|
| '''The devices that will interact with the system are initially to be'''
| | Below are the areas identified for testing / developing coverage. |
|
| |
|
| *Desktop (Windows / Mac) | | === Packaged Apps === |
| *Mobile (Android /iOS) | | * [P1] Install/launch/uninstall signed/unsigned packaged app |
| *Tablet (Android /iOS) | | * [P1] Install failure for packaged app not signed with type privileged (Security) |
| | * [P1] Install failure for packaged app with type certified (Security) |
| | * [P1] Cancel download of packaged app |
| | * [P1] Restart download of packaged app |
| | * [P>1] Download failure for packaged app for having a bad packaged app zip |
|
| |
|
| The purpose of this system is to serve as a way for HTML5 based apps to be distributed, across multiple devices. <br>
| | === Updated Apps === |
| | * [P1] Updated Manifest |
| | * [P1] Updated Manifest with preloaded appcache |
| | * [P1] General pieces of download portions of API on install/update |
| | * [P1] Web activities app install/update |
|
| |
|
| Community Involvement <br>
| | === Preloaded Appcache === |
| | * [P1] Install/launch/uninstall hosted app with appcache |
| | * [P>1] Download failure for hosted app with appcache for running out of space |
| | * [P1] Restart download of hosted app with appcache |
|
| |
|
| I see community involvement being a good way to do exploratory testing, gather usability feedback, as well as determine ease of use of the store / apps / widgets ..etc. The ability to test installs / usage patterns across multiple operating systems / tablets / phones will be very helpful.
| | === Hosted Apps === |
| | * [P1] Install/launch/uninstall hosted app |
| | * [P1] Install failure for hosted app with type as privileged or certified |
| | * [P1] Cancel download of hosted app with appcache |
| | * [P>1] Download failure for hosted app with appcache for having a bad appcache manifest |
|
| |
|
| == Desktop == | | == Bugs == |
| | | *{{Bug|880043}} Core DOM: App Mochitest: Install / Launch / Uninstall signed packaged app |
| === Windows - P1 ===
| | *{{Bug|880046}} Core DOM: App Install failure for unsigned privileged / signed certified packaged apps fail |
| | | *{{Bug|880053}} Core DOM: App Tests to exercise update app Apps Object api. |
| ==== Browsers w/o Addon ====
| | *{{Bug|880069}} Core DOM: App Changes to manifest trigger an update |
| {{Generic_TestCases_WebApp | | *{{Bug|880074}} Core DOM: App Mochitest: Install of hosted app with appcache |
| |os=Windows
| | *{{Bug|880076}} Core DOM: App Mochitest: Download failure for hosted app |
| |device=PC | |
| |browser = Firefox
| |
| }} | |
| | |
| {{Generic_TestCases_WebApp | |
| |os=Windows
| |
| |device=PC
| |
| |browser = Chrome | |
| }} | |
| | |
| {{Generic_TestCases_WebApp
| |
| |os=Windows
| |
| |device=PC
| |
| |browser = IE 8/9
| |
| }}
| |
| | |
| ==== Desktop App<br> ====
| |
| | |
| {| width="1109" cellspacing="1" cellpadding="1" border="1" class="wikitable" style=""
| |
| |-
| |
| | width="100" | '''Device<br>'''
| |
| | width="100" | '''PC'''<br>
| |
| | width="100" | '''Operating System<br>'''
| |
| | width="100" | '''OSX'''<br>
| |
| | width="100" | '''Browser'''
| |
| | width="100" | '''Firefox'''
| |
| |-
| |
| | width="200" | Test Case<br>
| |
| | width="200" | Steps<br>
| |
| | width="300" colspan="4" | Expected Result <br>
| |
| |-
| |
| | Run app from desktop, non-paid<br>
| |
| | Install a non paid app to the desktop run the app from desktop<br>
| |
| | colspan="4" | application should run as expected
| |
| |-
| |
| | Run app from desktop, paid<br>
| |
| | Install a paid app to the desktop
| |
| run the app from the desktop<br>
| |
| | |
| | colspan="4" | application should run as expected<br>
| |
| |-
| |
| | App refunded, launch from Desktop <br>
| |
| | Install a paid app to the desktop
| |
| Issue a refund<br> run the paid app from the desktop<br>
| |
| | |
| | colspan="4" | application should not run, possibly forward to the store<br>
| |
| |-
| |
| | Verify installation of apps work across multiple versions of windows<br>
| |
| | Try to install the app on different versions of windows. <br>
| |
| | colspan="4" | Verify install / launch works as expected<br>
| |
| |-
| |
| | uninstall firefox, verify apps don't launch
| |
| | Uninstall firefox, click on an installed app
| |
| | colspan="4" | App should fail to launch ?
| |
| |}
| |
| | |
| <br>
| |
| | |
| === Mac - P1 ===
| |
| | |
| ==== Use Cases ====
| |
| | |
| {{Generic_TestCases_WebApp | |
| |os=Mac
| |
| |device=PC
| |
| |browser = Firefox | |
| }} | |
| | |
| {{Generic_TestCases_WebApp | |
| |os=Mac
| |
| |device=PC
| |
| |browser = Safari | |
| }} | |
| | |
| <br> {{Generic_TestCases_WebApp
| |
| |os=Mac
| |
| |device=PC | |
| |browser = Chrome
| |
| }} | |
| | |
| ==== Desktop App<br> ====
| |
| | |
| {| width="1109" cellspacing="1" cellpadding="1" border="1" style="" class="wikitable"
| |
| |-
| |
| | width="100" | '''Device<br>'''
| |
| | width="100" | '''PC'''<br>
| |
| | width="100" | '''Operating System<br>'''
| |
| | width="100" | '''OSX'''<br>
| |
| | width="100" | '''Browser'''
| |
| | width="100" | '''Firefox'''
| |
| |-
| |
| | width="200" | Test Case<br>
| |
| | width="200" | Steps<br>
| |
| | width="300" colspan="4" | Expected Result <br>
| |
| |-
| |
| | Run app from desktop, non-paid<br>
| |
| | Install a non paid app to the desktop run the app from desktop<br>
| |
| | colspan="4" | application should run as expected
| |
| |-
| |
| | Run app from desktop, paid<br>
| |
| | Install a paid app to the desktop
| |
| run the app from the desktop<br>
| |
| | |
| | colspan="4" | application should run as expected<br>
| |
| |-
| |
| | App refunded, launch from Desktop <br>
| |
| | Install a paid app to the desktop
| |
| Issue a refund<br> run the paid app from the desktop<br>
| |
| | |
| | colspan="4" | application should not run, possibly forward to the store<br>
| |
| |-
| |
| | Verify installation of apps work across multiple versions of osx (*community*)<br>
| |
| | Try to install the app on different versions of osx. <br>
| |
| | colspan="4" | Verify install / launch works as expected<br>
| |
| |-
| |
| | uninstall firefox, verify apps don't launch
| |
| | Uninstall firefox, click on an installed app
| |
| | colspan="4" | App should fail to launch ?
| |
| |}
| |
| | |
| === Linux -P2 ===
| |
| | |
| == Mobile ==
| |
| | |
| == Tablet ==
| |
| | |
| == Cross Device Testing ==
| |
| | |
| == Store ==
| |
| | |
| == Automation ==
| |
| | |
| Marketplace: Verify payment flows / expectations from the store through the store in terms of log in / log out.
| |
| Developer upload / configure app for store.
| |
| | |
| == Long Term Plan ==
| |