QA/Fennec/Support manifest display-standalone display mode
From MozillaWiki
Last updated: 2017/06/21
Contents
- 1 Overview
- 2 Testing Summary
- 3 Testing details
- 3.1 Test Cases
- 3.2 Testing days
- 3.2.1 Date: 03-20-2017
- 3.2.2 Date: 03-21-2017
- 3.2.3 Date: 03-24-2017
- 3.2.4 Date: 03-21-2017
- 3.2.5 Date: 03-30-2017
- 3.2.6 Date: 03-31-2017
- 3.2.7 Date: 04-03-2017
- 3.2.8 Date: 04-05-2017
- 3.2.9 Date: 04-10-2017
- 3.2.10 Date: 04-11-2017
- 3.2.11 Date: 04-12-2017
- 3.2.12 Date: 04-13-2017
- 3.2.13 Date: 04-18-2017
- 3.2.14 Date: 04-19-2017
- 3.2.15 Date: 04-24-2017
- 3.2.16 Date: 04-26-2017
- 3.2.17 Date: 04-27-2017
- 3.2.18 Date: 05-04-2017
- 3.2.19 Date: 05-08-2017
- 3.2.20 Date: 05-10-2017
- 3.2.21 Date: 05-11-2017
- 3.2.22 Date: 05-12-2017
- 3.2.23 Date: 05-15-2017
- 3.2.24 Date: 05-17-2017
- 3.2.25 Date: 05-18-2017
- 3.2.26 Date: 05-19-2017
- 3.2.27 Date: 05-25-2017
- 3.2.28 Date: 05-26-2017
- 4 Bug Work
- 5 Signoff
Overview
Purpose
The purpose of this wiki is to serve as a general test plan for verifying that users of Android mobile devices can install and use a progressive web app, as a standalone app.
Ownership
Developer contact:
Overall Status
[LANDED] Nightly: In progress
[NOT LANDED] Aurora: n/a
[NOT LANDED] Beta: n/a
Testing Summary
Scope of testing
The testing will focus on verifying on supported devices that progressive web apps (pwa) can be installed as homescreen shortcuts and are opened as standalone apps, with an individual entry in the task manager.
Environments
Testing will be performed on the following Android devices:
- Phones
- 4.x
- 5.x
- 6.x
- 7.x
- Tablet
- 5.x
- 6.x
- 7.x
Specifications
- bug 1285858 - Support manifest display: standalone display mode
- More details about PWAs: https://developers.google.com/web/progressive-web-apps/
- A list of PWAs for testing: https://pwa.rocks/
User stories
- As a user, I want to be able to download a web app so I can launch it from the homescreen, to load instantly, regardless of the network state, without the need of an app store.
Acceptance Criteria
- All [Test cases] have passed and all bugs were fixed.
Testing details
Test Cases
Testing days
Date: 03-20-2017
- Started research and studying the documentation.
- Started creating the Test Plan
- Started writing Test Cases
Date: 03-21-2017
- Documentation research and writing test cases
Date: 03-24-2017
- Documentation research and writing test cases
Date: 03-21-2017
- Completed the Test Plan
- Writing Test Cases
Date: 03-30-2017
- Writing Test Cases
- Tested on Nightly 55 with LG G4 (Android 5.1)
- Logged bug: bug 1352099 - [PWA] Web Apps don't always have a title and icon in the task switcher entry
Date: 03-31-2017
- Verified bugs:
- Bug 1347154 - Cannot install twitter as a PWA
- Bug 1347616 - Handle invalid manifests
- Logged bug:
- Bug 1352378 - [PWA] Closing the Nightly task crashes if it has web app tabs opened
- Investigated:
- Bug 1347585 - Swiping to kill a custom tab leaves the tab open in Fennec
Date: 04-03-2017
- Writing test cases
- Tested on Nightly 55 with Motorola Nexus 6 (Android 7.0)
- Logged bug: Bug 1352991 - Pages loaded in Nightly show in the web app
Date: 04-05-2017
- Verified bugs:
- bug 1352726 - Flipboard is not given a title
- bug 1337341 - Handle the user navigating out the scope of the webapp
- bug 1352099 - [PWA] Web Apps don't always have a title and icon in the task switcher entry
- Logged bug:
- bug 1353714 - Crash @ java.lang.NoClassDefFoundError: android.app.ActivityManager$TaskDescription at org.mozilla.gecko.webapps.WebAppActivity.loadManifest(WebAppActivity.java)
Date: 04-10-2017
- Added test cases and verified on Nightly 55 with LG G4 (Android 5.1).
- Verified bug:
- Bug 1353714 - Crash @ java.lang.NoClassDefFoundError: android.app.ActivityManager$TaskDescription at org.mozilla.gecko.webapps.WebAppActivity.loadManifest(WebAppActivity.java)
- Logged bugs:
- Bug 1355094 - WebApps - Remove context menu items that are not needed
- Bug 1355098 - WebApps - 'Add to homescreen' from the bookmarks/history panels doesn't install the shortcut as a standalone app
Date: 04-11-2017
- Exploratory Testing on latest Nightly 55.0a1
- Logged bugs:
- Bug 1355513 - Allow access to microphone permissions not displayed in web apps
Date: 04-12-2017
- Exploratory testing on Nightly 55
- Verified bugs:
- Bug 1352433 - First run tour in browser and some settings reset after opening a web app
- Investigated:
- Bug 1336355 - Launched manifest bookmarks should have their own taskswitcher entry
- Bug 1355513 - Allow access to microphone and camera permissions not displayed in web apps
- Added a comment on: Bug 1352099 - [PWA] Web Apps don't always have a title and icon in the task switcher entry
- Logged bugs:
- Bug 1355824 - Browser opens a new tab for web apps already running - tabs tray shows wrong info
- Bug 1355847 - Web apps - Share selected text doesn't work on Android 6+
- Bug 1355852 - Web Apps - Android 4 & 5 Text selection toolbar not displayed in web apps
Date: 04-13-2017
- Exploratory testing on latest Nightly 55.0a1
- Research for relevant add-ons for web apps
Date: 04-18-2017
- Exploratory testing on latest Nightly 55.0a1
- Research for relevant add-ons for web apps. The two add-ons finally selected are:
- Logged bugs:
- Bug 1357443 - Web Apps - Can't download files from web apps using Nightly
- Bug 1357438 - Web apps - Login autocomplete isn't working
Date: 04-19-2017
- Exploratory testing on latest Nightly 55.0a1
- Logged bug:
- Bug 1357779 - Web Apps - Some webapps: the external url remains displayed
Date: 04-24-2017
- Exploratory testing on latest Nightly 55.0a1
Date: 04-26-2017
Functional and exploratory testing on latest Nightly 55.0a1, with Prestigio Grace X5 (Android 4.4.2)
Date: 04-27-2017
Exploratory testing on latest Nightly 55.0a1 Verified bugs:
- Bug 1355676 - Crash in java.lang.NullPointerException: Attempt to invoke virtual method android.graphics.Bitmap org.mozilla.gecko.icons.decoders.LoadFaviconResult.getBestBitmap(int) on a null object reference at org.mozilla.gecko.BrowserApp.handleMessage(Browser...
Date: 05-04-2017
Verified bugs:
- Bug 1355824 - Browser opens a new tab for web apps already running - tabs tray shows wrong info
- Bug 1352991 - Pages loaded in Nightly show in the web app
- Bug 1353857 - Fix handling of entering/leaving Web App
- Bug 1355852 - Web Apps - Android 4 & 5 Text selection toolbar not displayed in web apps
- Bug 1357443 - Web Apps - Can't download files from web apps using Nightly
Date: 05-08-2017
Full testing on Nightly, with:
- Motorola Motorola Nexus 6 (Android 7.0)
- Prestigio Grace X5 (Android 4.4.2)
Logged bugs:
- Bug 1363049 - Web Apps - Web apps are reopened when closing a tab opened from them
- Bug 1363052 - Web Apps - Re-launching a web app with an external page opened, resets is to the homepage
Date: 05-10-2017
Full testing on Nightly, with:
- Lenovo A536 (Android 4.4.2)
- Huawei Honor 5X (Android 5.1.1)
Verified bugs:
- Bug 1234558 - Use icon/title from app manifest for "Add to home screen"
- Bug 1360658 - Snack bars not showing up in WebAppActivity
Date: 05-11-2017
Full testing on Nightly, with:
- Lenovo Yoga Tablet 2 (Android 4.4.2)
Investigated:
- Bug 1360455 - Web Apps - snapdrop.net cannot be installed as pwa on Huawei/Xiaomi devices
Logged:
- Bug 1364044 - Web Apps - (Tablets) Tabs appear in the browser when there are multiple webapps opened
- Bug 1364052 - Web Apps - Some webapps show the internal page url
Date: 05-12-2017
Full testing on Nightly, with:
- Lenovo Yoga Tablet 2 (Android 4.4.2)
Date: 05-15-2017
Full testing on Nightly, with
- HTC Nexus 9 (Android 7.1.1)
- Huawei Honor 5X (Android 5.1.1)
Date: 05-17-2017
Full testing on Nightly, with LG G4 (Android 6)
- Verified:
- Bug 1357438 - Web apps - Login autocomplete isn't working
- Updated:
- Bug 1364044 - Web Apps - Tabs appear in the browser when there are multiple webapps opened
Date: 05-18-2017
Smoke testing on Nightly, with LG G4 (Android 6.0) Verified Bug 1363359 - standalone manifest install replaces Nightly in Touchwiz Sent the mid-Nightly feature sign-off email.
Date: 05-19-2017
Added PWA feature test cases in TestRail: https://testrail.stage.mozaws.net/index.php?/suites/view/254&group_by=cases:section_id&group_id=3341&group_order=asc
Date: 05-25-2017
Verified: Bug 1363049 - Web Apps - Web apps are reopened when closing a tab opened from them
Date: 05-26-2017
- Full testing on Asus ZenPad 8.0 Z380KL (Android 6.0.1)
- Logged bug: Bug 1368024 - Web Apps - Some web apps have an URL instead of title
Bug Work
Tracking bugs
- Bugs to track the feature
- Meta bug: bug 1285858 - Support manifest display: standalone display mode
57 Total; 1 Open (1.75%); 32 Resolved (56.14%); 24 Verified (42.11%);
Signoff
Criteria
- All [Test cases] have passed and all major bugs are fixed.
- All P1s are fixed.
Results
Nightly testing
- n/a
Aurora testing
- n/a
Beta testing
- n/a
