QA/Fennec/Klar+GeckoView
Overview
Purpose
The purpose of this wiki is to serve as a general test plan for verifying that switching the current Klar browser to Klar+GeckoView builds creates no visual and performance issues.
Ownership
Developer contact: Emily Kager
QA:
Overall Status
In development. Hoping to ship the new Klar+GeckoView by the end of Q2.
Testing Summary
Scope of testing
The testing will focus on:
- One-time smoke test of the basic browser functionality (menus/etc) of a Klar+GeckoView build.
- Visual comparison the top N mobile websites running in Klar+GeckoView vs Klar+WebView vs Fennec (looking for any Gecko webcompat differences).
- Some manual performance comparisons of Klar+GeckoView vs Klar+WebView. For example: Speedometer and approximate "stopwatch time" of app startup and page load times.
Environments
Testing will be performed on the following devices:
- Google Pixel 2 (Android 8.0.1)
- Samsung Galaxy S8 (Android 7.0)
At this point, we will not test on tablets nor x86 devices.
Specifications
Switching Klar from Chromium WebView to GeckoView should be an invisible change. Klar currently uses Chromium's WebView. We're switching Klar to use Firefox for Android's Gecko code (aka "GeckoView") and want to make test that nothing breaks in the new "Klar+GeckoView" browser.
User stories
As a user, switching from the standard Klar to Klar with GeckoView builds should not affect me.
Acceptance Criteria
No visible change in UI and performance. Nothing breaks in the new "Klar+GeckoView" browser.
Testing details
Test Cases
Testing days
Date:02-14-2018
- [Oana] Research, documentation, test cases and started testing.
Date:02-15-2018
- [Oana] Research and manual performance tests
Date:02-19-2018
- [Oana] Manual performance tests
- Functional testing:
- [Oana] Samsung Galaxy S8 (Android 7.0)
- [Ioana] Pixel 2 (Android 8.1.0)
- Logged issue:
Date:02-20-2018
- [Ioana] logged issue:
Date:02-21-2018
[Oana] logged issues:
- Google maps get location should be inactive
- Security level indicator missing for insecure connections
Date:03-02-2018
[Oana]
- Full functional tests on Samsung Galaxy Note 4 (Android 5.0.1)
- Logged:
- Verified:
Date:03-05-2018
[Oana] logged issues:
- Klar+Gecko: Crash when opening a drop-down menu
- Klar+GeckoView: fails to display a date picker
- Klar+GevkoView fails to open certain links
Date:03-05-2018
[Oana]
- Functional Tests - Klar+GeckoView on Xiaomi Mi Pad 2 (Android 5.1) - x86 architecture
- Klar+GeckoView: Search results should be displayed in the selected language
- With WIFI off, Klar redirects to other apps
- Klar+GeckoView: Can't add new search engines
Bug Work
Meta bug
Issues:
Bugs fixed
Signoff
Criteria
- All test cases should be executed
- All blockers, critical issues must be fixed and verified or have an agreed-upon timeline for being fixed.
- We'll want a more thorough pre-Beta/Release QA sign-off then, but for now we just need an initial test pass to find the big bugs.