Overview

Purpose

The purpose of this wiki is to serve as a general test plan for landing an `about:compat` page to display active sitepatches, and allow site authors to disable them.

Ownership

Developer contact: Mike Taylor [:miketaylr]

QA: Eliza Balazs

Overall Status

[LANDED] Nightly 68

[LANDED] Beta 68

Testing Summary

Timeline

• bug 1539918 - Tracking bug for about:compat to ride the trains in 68

QA Calendar

• Feature complete - May 03
• QA pre-merge Regression Testing - May 10 (Nightly sign-off)
• Pre release preliminary status reports - June 14
• Pre-Release feature Sign-offs - Jun 28 (Beta sign-off)

Scope of testing

The testing will focus on verifying the "about:compat" page that displays the active sitepatches, and allows the site authors to disable them.

Environments

Testing will be performed on the following Android devices:

• any configuration, we don’t have differences,
• latest version of Android: Android 9

Languages covered

• First run - Fennec 68 Nightly
  • English US

Specifications

• bug 1539918 - Tracking bug for about:compat to ride the trains in 68
• Documentation:
  • https://github.com/mozilla/webcompat-team-okrs/issues/53
  • https://docs.google.com/document/d/15nFMMwvuiuPBOqJ-jd8H_z5rWRdpX7AvdL-NaVVzaSQ/edit

User stories

• Desktop and Mobile users on Nightly (and then DevEdition) should be able to navigate to about:compat to see which UA overrides and interventions are enabled.
• Users should be able to disable them via about:compat (either globally, or individually).
• Ideally the content on this page is populated by the webcompat system addon.

Acceptance Criteria

• All Test cases have passed.

Testing details

Test Cases

• TestRail - Tracking bug for about:compat to ride the trains in 68
• https://docs.google.com/spreadsheets/d/1FIEf7K7OctJyzVEoJH0Al8RQkoHx0a_gOFcqWC951xQ/edit#gid=0

Testing days

Date: 03-15-2019

• Created Test Plan

Date: 03-26-2019

• Created Test Cases

Date: 04-04-2019

• Testing on latest Nightly 68.0a1 (2019-04-04) on the following devices:
  • OnePlus 5T(Android 9)
  • Samsung Galaxy S8 (Android 9)
  • Huawei MediaPad M2 (Android 5.1.1)

Date: 04-11-2019

• Testing on latest Nightly 68.0a1 (2019-04-11) on the following device:
  • OnePlus 5T(Android 9)

Date: 04-22-2019

• Verified bugs on latest Nightly 68.0a1 (2019-04-22):
  • bug 1541418 - The "about:compat" page UI is different in portrait and landscape mode
  • bug 1545081 - Unable to toggle interventions that have the same BZ issue as reference
  • bug 1543983 - Pop-up security level indicator is missing on the "about:compat" page
  • bug 1541019 - "More information:" links should open in a new tab all the time

Date: 04-24-2019

• Testing on latest Nightly 68.0a1 (2019-04-23) on the following device:
  • OnePlus 5T(Android 9)

Date: 04-25-2019

• We Finished our second run and sent out our Test Report

Date: 05-22-2019

• Bug Verification and we will do a Full Testing Run in BETA

Date: 05-23-2019

• Testing on latest Beta 68.0b3 with the following device:
  • Samsung Galaxy S8 (Android 9)

Date: 06-07-2019

• Bug Verification and testing on Beta 68.0b7 with the following device:
  • OnePlus 5T (Android 9)

Date: 06-12-2019

• Preliminary status sent

Date: 06-26-2019

• Run on Beta 68.0b13 before the Sign-off

Date: 06-27-2019

• The Official pre-release Sign-off was sent GREEN

Bug Work

Tracking bugs

• bugs to track the feature
  • bug 1539918 - Tracking bug for about:compat to ride the trains in 68

Full Query

ID	Summary	Priority	Status
1541019	"More information:" links should open in a new tab all the time	P3	VERIFIED
1541418	The "about:compat" page UI is different in portrait and landscape mode	P3	VERIFIED
1543968	Problem loading page when restarting Fennec, from the "about:compat" page	P3	RESOLVED
1543983	Pop-up security level indicator is missing on the "about:compat" page	P1	VERIFIED
1545081	Unable to toggle interventions that have the same BZ issue as reference	P1	VERIFIED
1553739	"The address isn't valid" is displayed when accessing the "about:compat" page on Fennec	P3	RESOLVED

6 Total; 0 Open (0%); 2 Resolved (33.33%); 4 Verified (66.67%);

Signoff

Criteria

• All Test cases have passed.

Results

Nightly testing

• All tests have passed on Nightly 68.0a1 (2019-04-23). No new/known issues found while testing this feature.

Beta testing

• All tests have passed on Beta 68.0b13. No new/known issues found while testing this feature.