QA/FennecESR
From MozillaWiki
< QA
Last updated: 2019/05/24
Overview
Purpose
The purpose of this wiki is to serve as a general test plan for testing Fennec ESR Nightly and Beta builds.
Scope
- Builds should look and act like regular Nightly/Beta builds, with a few exceptions.
QA: *Fennec QA (slack: mobilesv)
Overall Status
- Done
Testing Summary
Scope of testing
- As part of the work to move Fennec to ESR, we need to verify that Nightly builds coming from a Beta-level tree function as expected, as this will be the first time we've ever done such a thing.
Environments
Testing will be performed on the following devices:
- Phones
- Samsung Galaxy S8 (Android 9)
- Sony Xperia Z5 (Android 7.0.0)
- Google Pixel 3XL (Android 9)
- Huawei P9 Lite (Android 6.0)
- Tablets
- Nexus 9 (Android 7.1.1)
- x86 - Xiaomi Mi Pad 2 (Android 5.1)
- Samsung Galaxy Tab S3 (Android 8)
- Xiaomi Mi Pad 2 (Android 5.1)
- Lenovo Yoga Tablet 2 (Android 4.4.2)
Acceptance Criteria
- All tests should pass.
Testing details
Test Cases
Nightly and Beta:
Testing days
Date: 2019-05-10
- Created Test Plan and Test Runs
- Nightly
- Testrail Armv7 - tester: abodea
- Testrail AArch64 - tester: ebalazs
- Testrail x86 - tester: sdeiac
- Testrail x86-64 - tester: sflorean
- Beta
- Testrail Armv7 - tester: drus
- Testrail AArch64 - tester: mlobontiuroman
- Testrail x86 - tester: sdeiac
- Testrail x86-64 - tester: sflorean
Date: 2019-05-24
- Nightly
- Testrail api16 - tester: sdeiac
- Testrail AArch64 - tester: lapahidean
- Testrail x86 - tester: ebalazs
- Testrail x86-64 - tester: sflorean
- Testrail api16 - tester: drus
Tracking bugs
- Bugs to track the feature
- bug 1523402 - [meta] Move Fennec to ESR
48 Total; 0 Open (0%); 47 Resolved (97.92%); 1 Verified (2.08%);
Signoff
Criteria
- All Test cases should pass.
- All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/QA).
Results
Nightly testing
Date 10-05-2019
- Fennec-ESR - Armv7 - Nightly 68.0a1 on Google Pixel 3XL (Android 9)
- Fennec-ESR - AArch64 - Nightly 68.0a1 on Samsung Galaxy Tab S3 (Android 8.0)
- Fennec-ESR - x86 - Nightly 68.0a1 on Lenovo Yoga Tablet 2 (Android 4.4.2)
- Fennec-ESR - x86-64 - Nightly 68.0a1 on Xiaomi Mi Pad 2 (Android 5.1)
Beta testing
Date 10-05-2019
- Fennec-ESR - Armv7 - Beta 68.0b1 on Sony Xperia Z5 (Android 7.0.0)
- Fennec-ESR - AArch64 - Beta 68.0b1 on Samsung Galaxy S8 (Android 9)
- Fennec-ESR - x86 - Beta 68.0b1 on Lenovo Yoga Tablet 2 (Android 4.4.2)
- Fennec-ESR - x86-64 - Beta 68.0b1 on Xiaomi MiPad 2 (Android 5.1)
Issues
- New:
- bug 1550719 - Cannot update official Beta/Nightly builds with Fennec-ESR builds
- bug 1551442 - [ESR builds][Tablet] Make default browser option not displayed on top of the Settings Menu
Example Bug:
**{{Bug|1305969}} - Fennec sometimes get stuck in a state where there is a perma-bar of white on the bottom of the screen