«  Fenix RC 3.2.0-beta.3 | Fenix RC 4.0.0-beta.1 | Fenix RC 4.0.0-beta.2 »

Firefox Preview 4.0.0-beta.1 Test Plan

• Release date: February 24th, 2020
• QA Contact: Oana Horvath; SV QA team
• Slack: Oana;
• Build: Firefox Preview 4.0.0-beta.1
• TestRail test plan: Testrail

Features

Full functional Testing

Full Functional Tests are a more comprehensive set of manually covered tests that cover functional testing of features that have landed.

QA Assignment

Results

Telemetry

• https://testrail.stage.mozaws.net/index.php?/runs/view/25916

Phone

• https://testrail.stage.mozaws.net/index.php?/runs/view/25914

Tablet

• https://testrail.stage.mozaws.net/index.php?/runs/view/25913

Bug verification

List of issues that needed QA:

• 4807 - [Bug] Deleting opened tabs, exits Settings-Delete browsing data and returns to homescreen - VERIFIED AS FIXED
• 6981 - [Bug] (regression) Google (and other engines) missing in other languages - VERIFIED AS FIXED
• 4205 - [Bug] Lastpass showing mozilla credentials for website HTTP Basic Authentication box - RE-OPENED
• 8449 - [Bug] Delete browsing data UI clipping - VERIFIED AS FIXED
• 8268 - [Bug] Update snackbar for saving to top sites - VERIFIED AS FIXED
• 7957 - Enable Login management preference - VERIFIED AS FIXED
• 7796 - Turn on Logins for all users (not just Nightly) - VERIFIED AS FIXED
• 5695 - Add "Search" to the text selection context menu - VERIFIED AS FIXED
• 8354 - [Bug] Open links in private tab is not working - VERIFIED - WITH ISSUES
• 8056 - Mitigate impact of Experiments.initialize on startup, if impact is non-trivial - VERIFIED AS FIXED
• 8172 - Create a Fenix Leanplum Attribute that tells us the app is Fenix - VERIFIED AS FIXED
• 1741 - Remove second line on search suggestions - VERIFIED AS FIXED
• 8514 - CWTS preferences should look disabled when disabled - VERIFIED AS FIXED
• 6479 - Custom ETP: Choose which trackers and scripts to block - Still in verification process
• 6479 - Ability to disable third-party data collection - Still in verification process
• 7295 - Send a custom ping for `first install - Still in verification process

L10N

Light testing on featured locales to ensure strings fit the space allotted and there are no entity errors from missing or corrupt localizations.

• https://testrail.stage.mozaws.net/index.php?/runs/view/25915

A11y

• Accesibility scanner test run: https://testrail.stage.mozaws.net/index.php?/runs/view/25946

Issues

New isues

Major:

• 8580 - [Bug] Crash on returning from a new private tab
• 8637 - [Bug] Crash when opening link from custom tab in Fenix (open links in private previously set)

• 8565 - [Bug] Cookie option from Custom is reset every time you close the ETP panel
• 8535 - [Bug] Private mode homescreen is not updated after opening an external link
• 8582 - [Bug] Sending tabs to multiple devices is not working
• 8540 - [Bug]Disabling all options from Custom ETP still track all your browser
• 8569 - [Bug] Adding youtube/reddit search engine from the pre-defined list doesn't work
• 8571 - [Bug] History isn't sorted chronologically

Other:

• 8626 - [Bug] Camera & mic still running if you close a tab, until the undo snackbar is hidden
• 8534 - [Bug] Double-tap needed to switch the Open links in private tab toggle
• 8622 - [Bug] Tapping around "+" on Add-ons manager page opens the details of the add-on page
• 8583 - [Bug] Crash when accessing about:crashcontent
• 8570 - [Bug] Disabling uBlock and quickly opening the Settings section crashes Fenix
• 8575 - [Bug] Tablets keyboard doesn't hide, covering the save login prompt
• 8577 - [Bug] Logins prompt icons aren't properly displayed while using the Light theme
• 8567 - [Bug] Can't create new bookmarks folder due to #6974
• 8578 - [Bug] Different size icons for "Not supported yet" addons
• 8581 - [Bug] [a11y] Onboarding UI - The selected enabled options are not recognized as activated
• 8523 - [Bug] ETP options from onboarding screen can be selected when ETP is turned off
• 8524 - [Bug] Red buttons on Android 5 devices
• 8529 - [Bug]Tracking content menu from Custom ETP is not displayed
• 8627 - [Bug] [a11y] ETP radio buttons labels on the onboarding screen incorrect reading order
• 8629 - [Bug][a11] Save to collection button touch target is too small
• 8623 - [Bug][a11y]Sign In-The description for Lockwise autofill popup search through app is incorrect
• 8625 - [Bug][a11y] Default search engine list - The selected enabled options are not recognized as activated
• 8635 - [Bug][a11y]Error Page-Information is being repeated from "Address Not Found" section
• 8636 - [Bug] [a11y] Downloads - Elements not recognized through TalkBack
• 8638 - [Bug] [a11y] Addons - Lag for a couple of seconds when interchanging buttons
• 8640 - [Bug] [a11y] Addons - "Successfully enabled uBlock Origin" is not recognized by Talkback

Known issues

Major:

• 7540 - [Bug] Backgrounded custom tab is lost when opening Fenix
• 5636 - [Bug] Blocked media autoplay doesn't work properly
• 8386 - [Bug] Launching the app with private shortcut will keep the PB theme on normal browsing too
• 7507 - [Bug] Autoplay is not working for a muted video
• 8531 - [Bug] Deleted history still show up as search suggestions
• 8117 - [Bug] Crash when accessing any of the links from "About" from the uBlock Origin's page

Other:

• 7094 - Don't prompt user to login if they select "Never Save" in password prompt
• 5769 - [Bug] talky.io - Cam & mic access denied when the mic is granted permission
• 5478 - [Bug] Tabs turn into about:blank pages
• 8465 - [Bug]Improperly displayed Enhanced Tracking Protection CFR after rotation
• 6589 - Empty bookmarks library view looks buggy
• 6076 - [Bug]Improperly displayed folder icon in edit mode using the dark theme
• 5010 - [Bug]Add undo option when deleting a bookmark from edit mode
• 6970 - [Bug]History and Bookmark icons flicker when using multi-select feature
• 6975 - [Bug]Bookmarks folder can't be renamed (following #6974)
• 6974 - [Bug]Current Bookmark folder shouldn't be displayed as a root folder option while editing
• 6977 - [Bug]Deleted Bookmarks folder reappears after deleting it
• 5298 - [Bug] The same page appears in the History section multiple times
• 4484 - [Bug]Flicker when deleting History items
• 4432 - [Bug]Delete all history or individual history items snackbar missing
• 6079 - [Bug]Open tabs section flickers when opening multiple history items in new tabs
• 4178 - [Bug] Sharing multiple tabs to another browser, only opens 1 tab
• 7953 - [Bug] Error 0/0 displayed when double tapping on Find in page toolbar
• 8436 - [Bug] Sync isnt syncing bookmarks correctly
• 8108 - [Bug] The same site can be added multiple times to home screen
• 8480 - [Bug]Pocket Top Articles top site is present for locales different than EN
• 7230 - [Bug] Option to open external links with private Firefox missing in the new version
• 2725 - [Bug] Undo snackbar is not displayed when removing a tab from a collection
• 7331 - [Bug] Google images, maps and deezer fail with unknown protocol
• 3869 - [Bug] [Bug] Fenix not setting default search engine properly
• 6907 - [Bug] Changing Locale Does Not Update Search Engines
• 7385 - [Bug] Unclear when the "Reader view" toggle is available after pageload
• 3207 - [Bug] Location errors until you check the doorhanger's "don't ask again" box
• 6251 - [Bug] Auto-sign in doesn't work on Beta
• 7684 - [Bug] The error messages have different colors in three-dot menu
• 7723 - [Bug] The account sync doesn't start automatically when the app is brought to the foreground
• 7718 - [Bug] [a11y] Consider making the "x" button from the ETP pop-up larger
• 8139 - [Bug]Improperly displayed uBlock icons using the Light theme
• 3585 - [Bug] [a11y] Three-dot menu is not read