Personal tools

Features/Mobile/webapps/TestPlan

From MozillaWiki

Jump to: navigation, search

Contents

Overview - Web Apps Integration in Firefox for Android

Feature Release Target Dev Lead QA Lead Dev Status QA Status Health
Web Apps Integration in Firefox for Android Firefox 16 Mark Finkle Aaron Train, Jason Smith In Progress In Planning OK

Summary

In essence, the goal of web application support in mobile is to expose web apps to users of Firefox for Android. Users should be able to access and manage their apps have ability discover new apps and to have a seamless online apps experience.

Web applications are essentially websites built in web technologies (e.g, HTML, CSS, JavaScript) that users can interact with through a 'chrome-less' window in a native environment (e.g, Android). These web applications intent to act exactly like native applications on the operating system. For a website to become a web application, a developer creates an app manifest for their website and hosts it on an origin (e.g, www.yourhost.com) where the website is located. Then, the app developer adds this manifest to a store (e.g, Firefox Marketplace) to allow the app to be installed on ones machine or mobile device. Users can then go to this store to install any aforementioned application to their native device; manage ones application, launch ones application and or remove it.

For paid applications, users will have to pay for the application through PayPal. Upon making the application payment, a user's device receives a receipt to prove that the application is paid for. A user can then install the paid for application to their device. Upon attempt to install, a confirmation will appear confirming the installation to ones device; upon installation, a user should understand where the application was installed to (Android).

After applications are installed, users can then run these applications in their native environment in a chromeless shell both online and offline. If the application is paid, validation will need to take place using the receipt for the application to ensure that the user paid for the application. On Android, applications are typically ran as shortcuts on the Android home-screen or through the Application dashboard (currently accessible via the chrome URL: about:apps). When an application is launched, a chromeless window starts using Firefox's WebApp mode under the hood with the web application running in the shell. Within the shell, users can interact within the application such as logging in, clicking links within the origin of the application, play a game, and more. Within the shell itself, users have a menu to allow basic applications such as quitting.

If the users no longer wants the application installed on ones device, they can uninstall the application from within Firefox. On Android; uninstalling currently occurs by visiting the application dashboard, selecting the application and invoking a context menu on it with the option to 'Uninstall'. Upon uninstalling an application, all locally stored data that is previously created during installation should be cleared.

Testing Scope

Major Features

For our rounds of testing feature focused testing will focus on the following areas:

  • Web Apps Integration
    • Installing Web Apps
    • Launching Web Apps
    • Using Web Apps
    • Uninstalling Web Apps

Software Qualities

The following software qualities that need to be analyzed are (ordered by priority):

  1. Security
  2. Availability & Resilience
  3. L10N
  4. Location
  5. Performance & Scalability

Edge Cases

The following special edge cases need to be taken into account when testing the major parts of this feature:

  • Browser crashes
  • Loss of internet connection (e.g. offline mode)
  • App state changes (i.e. free to paid, paid to free)
  • Invalid or no receipts for paid apps

Testing Strategy

Akin to desktop there will be a similar and or shared testing strategy. Please see this testing strategy.

Sign Off Criteria

Aurora

  • No bugs found with flagged as Aurora blockers from Aurora test cases
  • Aurora-level (Phase 1) functional test cases pass
  • Aurora-level (Phase 1) tier 1 app scenario test cases pass
  • No unresolved bugs with Aurora blockers that cause Aurora-level test failures
  • Software Quality Requirements (?)

Beta

  • Tier 1 app scenario aurora and beta test cases all pass
  • Aurora and Beta-level (Phase 1) functional test cases pass
  • No unresolved bugs with a severity level of major or higher that cause Aurora or Beta test cases to fail
  • No regressions in Firefox linked to the features code changes detected with a severity of major or higher
  • Software Quality Requirements (?)

Release

  • No bugs found with a severity level of major or higher in test cases
  • All major functional test cases pass
  • Tier 1 app scenario test cases pass
  • No unresolved bugs with a severity level of major or higher that causes tests to fail
  • Software Quality Requirements (?)

Infrastucture Requirements

Test Cases and Results

Test Cases

  • Manual test cases can be found in Moztrap: Web App Integration - Smoke Tests
  • Manual test cases can be found in the Spreadsheet: Google Spreadsheet
  • Exploratory test cases of tier 1 apps can be found in the Spreadsheet: <link>
  • Tier 1 app scenario tests can be found in the Spreadsheet: <link>

Results

Bugs

Open Bugs

ID Summary Priority Status
892288 Remove ability to trust user-added certificates for code signing -- NEW
957061 translate Android intents to web activities -- NEW
839821 Allow passing 'name' and 'icon' with the URL as WEBAPP intent -- NEW
863034 Make org.mozilla.gecko/WebApp transparent to minimize the flash when launching a URL based webapp -- NEW
909917 Unable to launch app from about:apps -- NEW
915274 Disable the webapps actor in Firefox for Android until it's properly supported -- NEW
923819 Getting to "My Apps" quickly -- NEW
925087 Appcached web apps not being cached on at least Firefox for Android -- NEW
940605 Provide a text selection action mode for webapps -- NEW
959529 crash in __libc_android_abort | dlfree | free | _ZdlPv -- NEW
959609 crash in __libc_android_abort | dlfree | free | libGLESv2_adreno200.so@0x65b6f -- NEW
999991 Notification text if no updates for installed web apps available too long/cropped in some languages -- NEW
897755 A default profile may not be set properly in createProfileDir -- NEW
957065 uninstall synthetic APK when web page with privileges calls mozApps.mgmt.uninstall() P1 NEW
957068 check that G+ package installer gives non-null results in ApkResources.getInstallerPackageName P1 NEW
986085 User-hand-holding to check the Unknown Sources checkbox. P1 NEW
970200 let users restrict webapp updates to wifi connections P1 NEW
978143 Cancelling APK installation does not trigger error callback P1 NEW
863037 Create a group of reusable, temporary "slots" for URL based webapps P1 NEW
863048 Read app name and splash color from URL launcher Intent and use to create a splash screen P1 NEW
865647 Dungeon Fury fails to load with InvalidStateError P1 NEW
957063 notify user and clean up when app install/uninstall process fails P1 NEW
960584 notify user of download progress indication when fetching APK P1 NEW
965282 Bad experience with no Firefox browser installed P1 NEW
970061 Tapping update notification does nothing when browser not running P1 NEW
973114 Cut The Rope opens in portrait mode despite manifest.orientation property P1 NEW
973942 Crash on application launch when installed via Guest Browsing P1 NEW
976340 Installation fails with java.io.FileNotFoundException when using Google Install and Verify option P1 NEW
985604 Fosnews crashes aurora (Fx30) when launching after installation P1 NEW
985614 Packaged app installation does not trigger `onsuccess` callback P1 NEW
986222 add app's icon to Home screen on installation P1 NEW
988978 Updated APKs downloaded, but not installed. P1 NEW
988980 during upgrade of open webapps, error: "an existing package by the same name with a conflicting signature is already installed" P1 NEW
989086 Web App permission granted/denied settings (and other app state) are preserved across uninstall/install cycle, with no obvious way how to clear them P1 NEW
989655 Audio from Web Apps is paused/disabled when screen is turned off (e.g. with SoundCloud, News Podcasts) P1 NEW
991394 previously-installed apps stop running P1 NEW
994982 add allocated but unregistered app to registry on launch P1 NEW
995803 crash in java.lang.IllegalStateException: Already registered Webapps:Preinstall at org.mozilla.gecko.EventDispatcher.registerListener(EventDispatcher.java) P1 NEW
970203 animate webapp update "checking" and "downloading" notification icons P2 NEW
987795 [Honeycomb] launcher crash in Android's createIconBitmap on application install (com.android.launcher) P2 NEW
741624 Custom message for offline/unreachable apps. P2 NEW
766299 Pop-up windows outside of the main app window need to be shown as so, along with ability to close them P2 NEW
776677 Catch errors thrown from Webapps.js and show to user P2 NEW
784108 share credentials with apps P2 NEW
790401 Dashboard not listing newly installed/registered applications — browser re-launch required P2 NEW
792651 Web Apps should have the ability to specify storage requirements in the permissions property P2 NEW
818205 Add support for Google Play campaign parameters to Firefox for Android P2 NEW
851119 Horoscopo app is not correctly displayed if it is opened in landscape mode P2 NEW
851854 Disable GlobalHistory when running a WebApp P2 NEW
851855 Disable BrowserProvider when running a WebApp P2 NEW
854328 Robocop: Add support for WebApp testing using Robocop P2 NEW
859660 install hosted app that loads appcache, cut network, start app -- app fails with "Firefox can't establish a connection" P2 NEW
866777 Implement support for the fullscreen app manifest property P2 NEW
879662 Bug 872987 - [WebPayment] [Android] Expose ICC info to the payment provider P2 NEW
889037 Flip on b2g device api's when apps request them in their manifest P2 NEW
898901 Execute test_signed_apps-marketplace.js on Android P2 NEW
957060 migrate non-APK-installed apps to synthetic APKs P2 NEW
970131 Takes about a minute of inactivity before Android requests installation on old device P2 NEW
982548 "Check for Updates" opens Marketplace after previously opening Marketplace via "Browse the Firefox Marketplace" P2 NEW
983688 Re-use existing 'no updates available' notification on check for updates P2 NEW
985515 Unable to install web apps on LG Slider P2 NEW
986958 WeatherFox is unable to get my location, when installed as open web app via Firefox Marketplace on Android P2 NEW
987687 crash in java.lang.NullPointerException: at org.mozilla.gecko.webapp.ApkResources.getManifestUrl(ApkResources.java) P2 NEW
989057 Crash in java.lang.SecurityException: Permission Denial: reading org.mozilla.gecko.db.BrowserProvider uri content://org.mozilla.fennec.db.browser/flags?profile=webapp0 P2 NEW
990139 allow users to ignore an available update P2 NEW
994983 merge UninstallListener.doUninstall and EventListener.uninstallWebapp P2 NEW
997644 After Fennec update to a version with Synthetic APKs, the updated apps can not be launched from about:apps P2 NEW
960703 Manifest icon not used for synthesized APK in Android 2.3 P3 NEW
766300 Style doorhangers for WebApps P3 NEW
767376 Support registering webapps as protocol handlers P3 NEW
778279 Add support of installing of multiple apps off of the same origin for the android web runtime P3 NEW
783790 Launching a web app from about:apps persists a not-so-good-looking grey background on the app icon on about:apps P3 NEW
783792 Nit - overuse of dead space on right side of the about:apps UI with multiple apps installed P3 NEW
785932 No way to clear doorhanger settings set in a web app P3 NEW
790461 Integration of add-ons and webapps P3 NEW
792876 Add support for multiple orientations in the app manifest for web apps on ff android P3 NEW
793783 Add the name of the app being loaded in the splashscreen during initial startup of the app P3 NEW
817691 Reinstall a web app should refresh the browser cache P3 NEW
826780 Splash screen background color is not right for open webapp P3 NEW
852685 Move DIRECTORY_NAME from Webapps.jsm to AppUtils.jsm P3 NEW
872942 "Planet defense" can't be used P3 NEW
883121 Replace inefficient call to navigator.mozApps.mgmt.getAll(); on startup of a Webapp. P3 NEW
934482 Android runtime should expose API to "share a picture" P3 NEW
957058 implement WebappOSUtils.isLaunchable for Android P3 NEW
965284 Leaked receiver on application swipe off P3 NEW
967208 StrictMode violation in file:// Uri exposed through Intent.getData() in startActivityForResult P3 NEW
967331 installing packaged app fails with "JSONException: Value undefined… cannot be converted to JSONObject" P3 NEW
970210 display determinate progress meter while downloading webapp updates P3 NEW
979907 log webapp messages with log level P3 NEW
980018 Visible access to non web app context functionality P3 NEW
986962 GeckoEventDispatcher dispatchEvent no listeners for various events P3 NEW
987790 Installed web apps should prefer their source app P3 NEW
988615 launching sideloaded app on emulator throws NullPointerException at InstallHelper.copyApplicationZipFile P3 NEW
990274 make "Web App" generic name for legacy shortcut localizable P3 NEW
990959 remove or reuse initial tab when app is ready to load P3 NEW
994984 sideloaded/directly-installed APKs show wrong splash screen color on firstrun P3 NEW
996068 manifest/minifest mismatch causes packaged app to hang on launch P3 NEW

Open; Resolved; Total (0% complete)


Open Bugs: WebRT : Blocking 1

ID Summary Priority Status
787188 Hold off on about:home marketplace promo for now P1 RESOLVED
738545 Provide opt-in installation of Marketplace Application upon loading of Firefox for Android at a common Firefox for Android location (about:home?) P1 RESOLVED
786826 java.lang.NoSuchMethodError: android.content.SharedPreferences$Editor.apply at org.mozilla.gecko.WebAppAllocator.findAndAllocateIndex(WebAppAllocator.java) on Froyo P1 RESOLVED
799221 Turn off the marketplace as an app for now -- VERIFIED
766389 Add snippet on home screen to advertise the Marketplace P1 VERIFIED
766802 Clicking target=blank links in a web app should load those links in the browser P1 VERIFIED
781060 All apps should use the same webapp database P1 VERIFIED
785496 Marketplace app installs 4 shortcuts on dekstop P1 VERIFIED
786058 Disable webapps support on FF 16 P1 VERIFIED
741621 Mozilla Marketplace activity should open for marketplace URLs P1 VERIFIED
766094 Disable telemetry prompt on chrome-less application launch P1 VERIFIED
766260 Default icon for Webapps pointing to invalid icons P1 VERIFIED
766275 Default context menu for webapps P1 VERIFIED
766808 Cannot launch apps with icons using a relative url P1 VERIFIED
769840 Change the maximum number of apps allowed to be installed in firefox for android to some larger number P1 VERIFIED
771915 Show the origin of the page in the chrome of the app when it is different from webapp's origin P1 VERIFIED
772162 Disable webapps support on FF 15 P1 VERIFIED
779490 Unable to install an application with unicode characters (UTF-8) within the name P1 VERIFIED
781061 System Notifications are broken in webapps P1 VERIFIED
791612 "other origin" bar persists after pressing Back button to return to same-origin page P1 VERIFIED
793750 After starting up the Marketplace app through FF Android in Open with app, then returning to FF Android and reaccessing marketplace.mozilla.org on a new tab, nothing happens P1 VERIFIED

Open; Resolved; Total (14.29% complete)


Open Bugs

ID Summary Priority Status

Open; Resolved; Total (0% complete)


Open Bugs: WebRT : Blocking ?

ID Summary Priority Status
784468 Launching an application from the installation success notification from within an application opens the browser instead -- RESOLVED
776600 Implement the "orientation" property of the app manifest for web apps on Android P1 RESOLVED
790392 launching Marketplace from Android notification displays persistent white screen -- VERIFIED

Open; Resolved; Total (66.67% complete)


Open Bugs

ID Summary Priority Status

Open; Resolved; Total (0% complete)


Open Bugs by Severity

Bugs Needing Verification by Severity

References

EtherPads

Open Questions