QA/Fennec/CustomTab on Fennec
From MozillaWiki
Contents
- 1 Overview
- 2 Testing Summary
- 3 Testing details
- 3.1 Test Cases
- 3.2 Testing days
- 3.2.1 Date: 2017-03-02
- 3.2.2 Date: 2017-03-30
- 3.2.3 Date: 2017-04-03
- 3.2.4 Date: 2017-04-05
- 3.2.5 Date: 2017-04-14
- 3.2.6 Date: 2017-04-19
- 3.2.7 Date: 2017-05-08
- 3.2.8 Date: 2017-05-17
- 3.2.9 Date: 2017-05-22
- 3.2.10 Date: 2017-05-29
- 3.2.11 Date: 2017-08-07
- 3.2.12 Date: 2017-08-09
- 3.2.13 Date: 2017-08-28
- 3.2.14 Date: 2017-08-31
- 3.2.15 Date: 2017-10-09
- 3.2.16 Date: 2017-10-23
- 3.2.17 Date: 2017-10-26
- 3.2.18 Date: 2018-02-28
- 4 Bug Work
- 5 Signoff
Overview
Purpose
The purpose of this wiki is to serve as a general test plan for verifying that Custom Tabs are supported in Fennec
Ownership
Developer contact: Dylan Roeh [:droeh]
Overall Status
[LANDED] Nightly 57:
[LANDED] Beta 57
Testing Summary
Scope of testing
The testing will be focus on:
- verifying that users can use Custom Tabs in Fennec
- verifying that a toolbar color was set
- verifying that the toolbar back button for custom tabs will close the tab and finish the activity, allowing to getting back to the app that launched the custom tab
- verifying that my ad-blocker add-on from my Fennec has the same functionality for my custom tabs
- verifying that the web-page is reload when users scrolled down the web-page
- verifying that there is 3 different types of displaying site info in URL Bar
- verifying that progress bar is displayed for loading page
Environments
Testing will be performed on following devices:
- Phones
- 5.x?
- 6.x
- 7.x
- Tablet
- 6.x
- 7.x
Specifications
- Toolbar: https://bug1279278.bmoattachments.org/attachment.cgi?id=8792338
- https://drive.google.com/drive/folders/0B4dMhI4hp32OUkNmYlRlSmVDQm8
- Other features that help users browse the web, like saved passwords, auto-fill, Tap to Search, and Sync, are also available in custom tabs.
Screenshot:
User stories
- As a user, I want to be able to use custom tabs on Fennec
- As a user, I want if I set Firefox for Android as the default browser to also use Firefox when 3rd party apps use custom tabs
- As a user, I expect links that I open from an app using custom tabs will share cookie jar and permissions from the full Firefox for Android browser so I don't have to log in to sites that I already connected to, or granted permissions to
Acceptance Criteria
- 3-party app can edit background color of custom tab, and add functions onto custom tab, such as use red background and add a “Share” button
- UI customization:Toolbar color, action button, custom menu items, custom in/out animations, bottom toolbar
- Synchronized AutoComplete across devices for better form completion.
Testing details
Test Cases
- Test Case Custom Tab Testrail
- https://docs.google.com/spreadsheets/d/1Op8Yqctmh4p7UwkN0I7q8uf1GoyXXnQ90E-oq4o5mCE/edit#gid=0
Testing days
Date: 2017-03-02
- Created Test Plan
Date: 2017-03-30
- Created Test Cases
- Exploratory testing on latest Nightly (2017-03-30) with Nexus 6 (Android 7.1.1)
- New bugs:
- bug 1352104 - Back arrow and three dots menu doesn't have the same color
- bug 1352108 - Nightly custom tab display a blank page when device orientation is changed
- For enabling this feature: Fennec Menu -> Settings -> Advanced -> Experimental features -> Custom Tabs (turn on)
Date: 2017-04-03
- Commented on:
- bug 1329149 - Custom tabs: allow sharing of AutoComplete from the full Firefox for Android browser for better form completion
- bug 1351605 - Custom tabs: Title becomes blank after resume
- Verified P1 bugs:
- bug 1337238 - Custom Tabs: progress bar on the custom bar for loading page
- bug 1332546 - [CustomTab] 3 different types of displaying site info in URL bar should be designed in custom tab while there is only one in chrome
- bug 1325174 - Custom tabs toolbar back button should close the tab and finish the activity
- bug 1348716 - Copy link in CustomTabsActivity by long-clicking in Title
- bug 1346542 - Back button navigation and web content copy/paste broken in restored GeckoApp instance
- bug 1329143 - Custom tabs: support custom toolbar color
- New bugs during verifying/testing:
- bug 1352993 - Custom Tab: Action bar is not displayed when selecting a word
- bug 1352999 - Custom Tab: Same page/URL is opened in different tabs in Nightly
- bug 1353019 - Custom tabs: reload/refresh button from menu doesn't change to "x"
Date: 2017-04-05
- Exploratory testing with LG G4 (Android 5.1) and HTC Desire 820 (Android 6.0.1)
- Verified bugs:
- bug 1315937 - Custom tabs: Support custom menu items
- bug 1329148 - Custom tabs: allow sharing of cookie jar and permissions from the full Firefox for Android browser so users don't have to log in to sites they are already connected to, or granted permissions to
- New bug:
- bug 1353732 - Custom Tab: Issues while switching between Nightly and apps that supports custom tabs
Date: 2017-04-14
- SignOff report for 55:
- Exploratory and Test Cases with HTC 10 (Android 7.0) and Huawei MediaPad M2 (Android 5.1.1).
- TestRail: https://testrail.stage.mozaws.net/index.php?/runs/view/3757&group_by=cases:section_id&group_id=2563&group_order=asc
- New bugs:
- bug 1356545 - CustomTab toolbar stay displayed when scroll down
- bug 1356549 - Custom Tab:: forms are not focused
- bug 1356560 - Custom Tab: doorhanger is not correctly displayed
Verified bugs:
- bug 1352104 - Back arrow and three dots menu doesn't have the same color
- bug 1352108 - Nightly custom tab display a blank page when device orientation is changed
Date: 2017-04-19
- Exploratory Testing with Huawei Honor (Android 5.1.1)
- New Bugs:
- bug 1357778 - Custom Tab: Fennc launched with the same URL as CustomTab
- bug 1357780 - Custom Tab: Different color for Share option from menu
- bug 1357781 - Custom Tab: Edit Site Settings option not implemented
Date: 2017-05-08
- Exploratory Testing with Huawei Honor (Android 5.1.1), Honor 8 (Android 6.0), Asus ZenPad 8(Android 6.0.1)
- New Bug:
- bug 1363043 - Custom Tab: ActionBar with different colors
- Verified bugs:
- bug 1353019 - Custom tabs: reload/refresh button from menu doesn't change to "x"
- bug 1351605 - Custom tabs: Title becomes blank after resume
- bug 1352108 - Nightly custom tab display a blank page when device orientation is changed
- bug 1352993 - Custom Tab: Action bar is not displayed when selecting a word
Date: 2017-05-17
- Exploratory Testing with Nexus 9 (Android 7.1.1), HTC Desire 820(Android 6.0.1)
- Verified bugs:
- bug 1350198 - Custom tabs: Cannot paste strings to input element
- bug 1329145 - Custom tabs: support custom enter and exit animation
- bug 1337771 - Custom tabs: Action button not working for some apps
- bug 1329144 - Custom tabs: support custom action button
- bug 1352352 - Custom tabs: open link in context menu does not work well
- Commented on:
- bug 1329149 - Custom tabs: allow sharing of AutoComplete from the full Firefox for Android browser for better form completion
- bug 1355735 - Custom tabs: Menu says "Open in Android-System" if no default browser is set
- bug 1357778 - Custom Tab: Fennc launched with the same URL as CustomTab
Date: 2017-05-22
- Exploratory Testing with Huawei Honor (Android 5.1.1)
- Finished Test Cases
- New Bugs:
- bug 1366770 - Custom Tab: Toolbar still displayed if a video is fullscreen
- bug 1366782 - Custom Tab: about:home launched when CustomTab activity is closed
Date: 2017-05-29
- Exploratory Testing and TCs with Asus ZenPad 8 (Android 6.0.1) - Gmail, Yahnac, News&Weather, Chromer apps
- Verified bugs:
- bug 1366648 - Custom tabs: reload button has wrong color
- bug 1357781 - Custom Tab: Edit Site Settings option not implemented
Date: 2017-08-07
- Reviewing TCs for Custom Tab - GeckoView - feature in Nightly 57
- Exploratory testing with Nexus 5 (Android 6.0.1) on latest Nightly (08-07)
- New bug 1388058 - CustomTab: Security level of the site not displayed
Date: 2017-08-09
- Exploratory testing with HTC 10 (Android 7.0) on latest Nightly
- New bug 1388734 - CustomTab: download is not working
- New bug 1388724 - CustomTab: doorhanger not displayed
Date: 2017-08-28
- Exploratory testing with Asus ZenPad 8 (Android 6.0.1) on latest Nightly
- Verified bugs:
- bug 1366770 - Custom Tab: Toolbar still displayed if a video is fullscreen
- bug 1388058 - CustomTab: Security level of the site not displayed
Date: 2017-08-31
- Mid-Nightly signOff report, status: YELLOW for 57
- New bugs found:
- bug 1395570 - CustomTab: media notification not displayed
- bug 1395573 - CustomTab: "Edit site settings" not displayed
- bug 1395577 - CustomTab: Autocomplete not working
- bug 1395582 - CustomTab: context menu not working
- bug 1395557 - Customtab: security level of site not displayed same as Fennec
Date: 2017-10-09
- Test cases and exploratory testing on 57 Beta 5
- Huawei MediaPad M2 (Android 5.1.1) - https://testrail.stage.mozaws.net/index.php?/runs/view/6575&group_by=cases:section_id&group_order=asc&group_id=2563
- Google Pixel (Android 8.0) - https://testrail.stage.mozaws.net/index.php?/runs/view/6574&group_by=cases:section_id&group_order=asc&group_id=2563
- Samsung Galaxy Tab 3 (Android 7.0) - https://testrail.stage.mozaws.net/index.php?/runs/view/6567&group_by=cases:section_id&group_order=asc&group_id=2563
- Reopened bug 1356560 - Custom Tab: doorhanger is not correctly displayed
- Verified bug 1400243 - CustomTab: no doorhanger displayed when log in to a page
Date: 2017-10-23
- Verified bugs:
- bug 1300574 - Back button doesn't navigate back a page in custom tabs
- bug 1365868 - Support minimal context menu functionality in GeckoView-based custom tabs
- bug 1388724 - CustomTab: doorhanger not displayed
- bug 1315348 - Rotating the device while viewing a custom tab loses toolbar color
Date: 2017-10-26
- Pre-Release Firefox 57 Sign off report, Status: GREEN
Date: 2018-02-28
- Ran test cases on Nightly 60.0a1 with Samsung Galaxy S8(Android 7.0): https://testrail.stage.mozaws.net/index.php?/runs/view/8591&group_by=cases:section_id&group_order=asc&group_id=2563
- Commented/verified bugs:
- bug 1393128 - Custom tabs: make downloads work
- bug 1353732 - Custom Tab: Issues while switching between Nightly and apps that supports custom tabs
- bug 1360699 - Fennec Custom Tab doesn't close after pressing back or close buttons.
- bug 1440592 - Crash in java.lang.IllegalStateException: Unexpected new session at org.mozilla.gecko.customtabs.CustomTabsActivity.onNewSession(CustomTabsActivity.java)
Bug Work
- Meta bug
- bug 1208655 - (customtabs) [meta] Make Fennec be an Android "custom tab" ecosystem participant
83 Total; 0 Open (0%); 57 Resolved (68.67%); 26 Verified (31.33%);
- bug 1356346 - (customtabs_geckoview) Overhaul CustomTabsActivity to use GeckoView rather than extending GeckoApp
17 Total; 0 Open (0%); 12 Resolved (70.59%); 5 Verified (29.41%);
Still issues
Bug fixed
Signoff
Criteria
- All test cases should be executed
- All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed
Results
Nightly testing
- feature stable
Beta testing
- feature stable
