QA/Fennec/Support manifest display-standalone display mode

Last updated: 2017/04/05

Overview

Purpose

The purpose of this wiki is to serve as a general test plan for verifying that users of Android mobile devices can install and use a progressive web app, as a standalone app.

Ownership

Developer contact:

QA: Oana Horvath (:oanah)

Overall Status

[LANDED] Nightly: In progress

[NOT LANDED] Aurora: n/a

[NOT LANDED] Beta: n/a

Testing Summary

Scope of testing

The testing will focus on verifying on supported devices that progressive web apps (pwa) can be installed as homescreen shortcuts and are opened as standalone apps, with an individual entry in the task manager.

Environments

Testing will be performed on the following Android devices:

• Phones
  • 5.x
  • 6.x
  • 7.x
• Tablet
  • 5.x
  • 6.x
  • 7.x
• Android 4 devices, or anything under API 21 level, will not be supported.

Specifications

• bug 1285858 - Support manifest display: standalone display mode
• More details about PWAs: https://developers.google.com/web/progressive-web-apps/
• A list of PWAs for testing: https://pwa.rocks/

User stories

• As a user, I want to be able to download a web app so I can launch it from the homescreen, to load instantly, regardless of the network state, without the need of an app store.

Acceptance Criteria

• All [Test cases] have passed and all bugs were fixed.

Testing details

Test Cases

• TestRail test cases
• Google sheet - test cases

Testing days

Date: 03-20-2017

• Started research and studying the documentation.
• Started creating the Test Plan
• Started writing Test Cases

Date: 03-21-2017

• Documentation research and writing test cases

Date: 03-24-2017

• Documentation research and writing test cases

Date: 03-21-2017

• Completed the Test Plan
• Writing Test Cases

Date: 03-30-2017

• Writing Test Cases
• Tested on Nightly 55 with LG G4 (Android 5.1)
• Logged bug: bug 1352099 - [PWA] Web Apps don't always have a title and icon in the task switcher entry

Date: 03-31-2017

• Verified bugs:
  • Bug 1347154 - Cannot install twitter as a PWA
  • Bug 1347616 - Handle invalid manifests
• Logged bug:
  • Bug 1352378 - [PWA] Closing the Nightly task crashes if it has web app tabs opened
• Investigated:
  • Bug 1347585 - Swiping to kill a custom tab leaves the tab open in Fennec

Date: 04-03-2017

• Writing test cases
• Tested on Nightly 55 with Motorola Nexus 6 (Android 7.0)
• Logged bug: Bug 1352991 - Pages loaded in Nightly show in the web app

Date: 04-05-2017

• Verified bugs:
  • bug 1352726 - Flipboard is not given a title
  • bug 1337341 - Handle the user navigating out the scope of the webapp
  • bug 1352099 - [PWA] Web Apps don't always have a title and icon in the task switcher entry
• Logged bug:
  • bug 1353714 - Crash @ java.lang.NoClassDefFoundError: android.app.ActivityManager$TaskDescription at org.mozilla.gecko.webapps.WebAppActivity.loadManifest(WebAppActivity.java)

Bug Work

Tracking bugs

• bugs to track the feature

Full Query

ID	Summary	Priority	Status
1234558	Use icon/title from app manifest for "Add to home screen"	--	VERIFIED
1300570	[meta] Separate normal / custom / web app tabs	--	RESOLVED
1321320	Ability to determine whether a url is within an installed app scope	--	VERIFIED
1336355	Launched manifest bookmarks should have their own taskswitcher entry	--	VERIFIED
1337341	Handle the user navigating out the scope of the webapp	--	VERIFIED
1346542	Back button navigation and web content copy/paste broken in restored GeckoApp instance	P1	VERIFIED
1347154	Cannot install twitter as a PWA	--	VERIFIED
1347165	Change context menu "open in new tab" to "open in Firefox tab" when not in browser	P2	RESOLVED
1347585	Swiping to kill a custom tab leaves the tab open in Fennec	P2	RESOLVED
1347605	Web Apps opened from homescreen should be singletons	--	VERIFIED
1347611	Web Apps (and custom tabs) flash previously opened content	P2	VERIFIED
1347616	Handle invalid manifests	--	VERIFIED
1348686	Implement ability to change a tab's tab type	P2	RESOLVED
1348794	Back button does not work in standalone mode	--	RESOLVED
1350236	Pull to refresh for standalone webapps	--	RESOLVED
1351169	Crash in java.lang.NullPointerException: Null native pointer at org.mozilla.gecko.mozglue.GeckoLoader.nativeRun(Native Method)	P1	VERIFIED
1351554	PWA: Collect information on the number of Firefox launches using PWA standalone mode	P1	RESOLVED
1351555	PWA: Collect information on the number of progressive web apps that’s added to homescreen	P1	RESOLVED
1351556	PWA: Collect information on the number of users using PWA standalone mode	P1	RESOLVED
1351739	System for switching between different kinds of GeckoApp	P1	RESOLVED
1351834	Set correct tab type when launching WepApp activity from external intent	--	RESOLVED
1352004	Doorhangers missing background	P1	VERIFIED
1352099	[PWA] Web Apps don't always have a title and icon in the task switcher entry	P1	VERIFIED
1352378	[PWA] Closing the Nightly task crashes if it has web app tabs opened	--	RESOLVED
1352394	Crash in java.lang.NullPointerException: Attempt to invoke virtual method ''java.lang.String org.mozilla.gecko.Tab.getURL()'' on a null object reference at org.mozilla.gecko.webapps.WebAppActivity.onNewIntent(WebAppActivity.java)	P1	RESOLVED
1352433	First run tour in browser and some settings reset after opening a web app	P1	VERIFIED
1352726	Flipboard is not given a title	--	VERIFIED
1352991	Pages loaded in Nightly show in the web app	P1	RESOLVED

28 Total; 0 Open (0%); 14 Resolved (50%); 14 Verified (50%);

Signoff

Criteria

• All [Test cases] have passed and all major bugs are fixed.

Results

Nightly testing

• n/a

Aurora testing

• n/a

Beta testing

• n/a