Features/Mobile/webapps/TestPlan

From MozillaWiki
< Features‎ | Mobile‎ | webapps
Jump to: navigation, search

Overview - Web Apps Integration in Firefox for Android

Feature Release Target Dev Lead QA Lead Dev Status QA Status Health
Web Apps Integration in Firefox for Android Firefox 16 Mark Finkle Aaron Train, Jason Smith In Progress In Planning OK

Summary

In essence, the goal of web application support in mobile is to expose web apps to users of Firefox for Android. Users should be able to access and manage their apps have ability discover new apps and to have a seamless online apps experience.

Web applications are essentially websites built in web technologies (e.g, HTML, CSS, JavaScript) that users can interact with through a 'chrome-less' window in a native environment (e.g, Android). These web applications intent to act exactly like native applications on the operating system. For a website to become a web application, a developer creates an app manifest for their website and hosts it on an origin (e.g, www.yourhost.com) where the website is located. Then, the app developer adds this manifest to a store (e.g, Firefox Marketplace) to allow the app to be installed on ones machine or mobile device. Users can then go to this store to install any aforementioned application to their native device; manage ones application, launch ones application and or remove it.

For paid applications, users will have to pay for the application through PayPal. Upon making the application payment, a user's device receives a receipt to prove that the application is paid for. A user can then install the paid for application to their device. Upon attempt to install, a confirmation will appear confirming the installation to ones device; upon installation, a user should understand where the application was installed to (Android).

After applications are installed, users can then run these applications in their native environment in a chromeless shell both online and offline. If the application is paid, validation will need to take place using the receipt for the application to ensure that the user paid for the application. On Android, applications are typically ran as shortcuts on the Android home-screen or through the Application dashboard (currently accessible via the chrome URL: about:apps). When an application is launched, a chromeless window starts using Firefox's WebApp mode under the hood with the web application running in the shell. Within the shell, users can interact within the application such as logging in, clicking links within the origin of the application, play a game, and more. Within the shell itself, users have a menu to allow basic applications such as quitting.

If the users no longer wants the application installed on ones device, they can uninstall the application from within Firefox. On Android; uninstalling currently occurs by visiting the application dashboard, selecting the application and invoking a context menu on it with the option to 'Uninstall'. Upon uninstalling an application, all locally stored data that is previously created during installation should be cleared.

Testing Scope

Major Features

For our rounds of testing feature focused testing will focus on the following areas:

  • Web Apps Integration
    • Installing Web Apps
    • Launching Web Apps
    • Using Web Apps
    • Uninstalling Web Apps

Software Qualities

The following software qualities that need to be analyzed are (ordered by priority):

  1. Security
  2. Availability & Resilience
  3. L10N
  4. Location
  5. Performance & Scalability

Edge Cases

The following special edge cases need to be taken into account when testing the major parts of this feature:

  • Browser crashes
  • Loss of internet connection (e.g. offline mode)
  • App state changes (i.e. free to paid, paid to free)
  • Invalid or no receipts for paid apps

Testing Strategy

Akin to desktop there will be a similar and or shared testing strategy. Please see this testing strategy.

Sign Off Criteria

Aurora

  • No bugs found with flagged as Aurora blockers from Aurora test cases
  • Aurora-level (Phase 1) functional test cases pass
  • Aurora-level (Phase 1) tier 1 app scenario test cases pass
  • No unresolved bugs with Aurora blockers that cause Aurora-level test failures
  • Software Quality Requirements (?)

Beta

  • Tier 1 app scenario aurora and beta test cases all pass
  • Aurora and Beta-level (Phase 1) functional test cases pass
  • No unresolved bugs with a severity level of major or higher that cause Aurora or Beta test cases to fail
  • No regressions in Firefox linked to the features code changes detected with a severity of major or higher
  • Software Quality Requirements (?)

Release

  • No bugs found with a severity level of major or higher in test cases
  • All major functional test cases pass
  • Tier 1 app scenario test cases pass
  • No unresolved bugs with a severity level of major or higher that causes tests to fail
  • Software Quality Requirements (?)

Infrastucture Requirements

Test Cases and Results

Test Cases

  • Manual test cases can be found in Moztrap: Web App Integration - Smoke Tests
  • Manual test cases can be found in the Spreadsheet: Google Spreadsheet
  • Exploratory test cases of tier 1 apps can be found in the Spreadsheet: <link>
  • Tier 1 app scenario tests can be found in the Spreadsheet: <link>

Results

Bugs

Open Bugs

ID Summary Priority Status
1114631 [parity] Detect "web app capable" sites using meta mobile-web-app-capable or meta application-name -- NEW
1234558 Use icon from app manifest for "Add to home screen" -- NEW
1237045 Allow user to edit home screen shortcut name -- NEW
1239358 "minimal-ui" display mode should be supported -- NEW
1242786 consider deleting stale downloaded synthetic APKs -- NEW
1242787 Delete Gecko profiles associated with apps installed by web runtime -- NEW
1249318 homescreen shortcut opens new tab if Firefox isn't currently running -- NEW

7 Total; 7 Open (100%); 0 Resolved (0%); 0 Verified (0%);


Open Bugs: WebRT : Blocking 1

ID Summary Priority Status
738545 Provide opt-in installation of Marketplace Application upon loading of Firefox for Android at a common Firefox for Android location (about:home?) P1 RESOLVED
741621 Mozilla Marketplace activity should open for marketplace URLs P1 VERIFIED
766094 Disable telemetry prompt on chrome-less application launch P1 VERIFIED
766260 Default icon for Webapps pointing to invalid icons P1 VERIFIED
766275 Default context menu for webapps P1 VERIFIED
766389 Add snippet on home screen to advertise the Marketplace P1 VERIFIED
766802 Clicking target=blank links in a web app should load those links in the browser P1 VERIFIED
766808 Cannot launch apps with icons using a relative url P1 VERIFIED
769840 Change the maximum number of apps allowed to be installed in firefox for android to some larger number P1 VERIFIED
771915 Show the origin of the page in the chrome of the app when it is different from webapp's origin P1 VERIFIED
772162 Disable webapps support on FF 15 P1 VERIFIED
779490 Unable to install an application with unicode characters (UTF-8) within the name P1 VERIFIED
781060 All apps should use the same webapp database P1 VERIFIED
781061 System Notifications are broken in webapps P1 VERIFIED
785496 Marketplace app installs 4 shortcuts on dekstop P1 VERIFIED
786058 Disable webapps support on FF 16 P1 VERIFIED
786826 java.lang.NoSuchMethodError: android.content.SharedPreferences$Editor.apply at org.mozilla.gecko.WebAppAllocator.findAndAllocateIndex(WebAppAllocator.java) on Froyo P1 RESOLVED
787188 Hold off on about:home marketplace promo for now P1 RESOLVED
791612 "other origin" bar persists after pressing Back button to return to same-origin page P1 VERIFIED
793750 After starting up the Marketplace app through FF Android in Open with app, then returning to FF Android and reaccessing marketplace.mozilla.org on a new tab, nothing happens P1 VERIFIED
799221 Turn off the marketplace as an app for now -- VERIFIED

21 Total; 0 Open (0%); 3 Resolved (14.29%); 18 Verified (85.71%);


Open Bugs

ID Summary Priority Status

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);


Open Bugs: WebRT : Blocking ?

ID Summary Priority Status
776600 Implement the "orientation" property of the app manifest for web apps on Android P1 RESOLVED
784468 Launching an application from the installation success notification from within an application opens the browser instead -- RESOLVED
790392 launching Marketplace from Android notification displays persistent white screen -- VERIFIED

3 Total; 0 Open (0%); 2 Resolved (66.67%); 1 Verified (33.33%);


Open Bugs

ID Summary Priority Status

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);


Open Bugs by Severity

Bugs Needing Verification by Severity

References

EtherPads

Open Questions