QA/Fennec/Stylo

From MozillaWiki
< QA‎ | Fennec
Jump to: navigation, search

Overview

Purpose

The purpose of this wiki is to serve as a general test plan for verifying that

Ownership

Developer contact: Makoto Kato QA: Ioana Chiorean [:ioanachiorean]

Overall Status

  • [LANDED] Nightly 59: GREEN
  • [NOT LANDED] Beta 59:

Testing Summary

Scope of testing

  • The testing will be focus on verifying that user can navigate and browse without issues on Quantum pages

Environments

Testing will be performed on following devices:

  • Phones: Android 4 -> Android 8
  • Tablet: Android 4 -> Android 7

Specifications

  • Prerequisites :
  • Ensure the following about:config pref is true:
    • “layout.css.servo.enabled” = TRUE
    • Ensure that about:support says: “Stylo: true”.
    • Ensure that https://arewestyloyet.rs/ says "Yes, you are using Stylo!" and "Yes, this page is styled by Stylo!"
Bookmarks Hint

User stories

  • As a user, when I navigate to and browser the most used pages I expect a smooth experience and no bad behavior.
  • As a user, comparing the pages for each use case in Firefox with Stylo enabled and disabled, i should not see any difference. They should look the same.

Testing details

Test Cases

Testing days

Date:2017-11-15

  • Feature submitted and assigned

Date:2017-12-23

  • Testing done on Pixel XL - Android 8.0 - without documentation ( using Chris' notes)

Date:2018-01-10

  • Test Plan Creation

Bug Work

Tracking bugs

Full Query
ID Priority Summary Status Resolution Assigned to
1397764 -- Allow to set BINDGEN_CFLAGS from moz.confgure (or autoconf) RESOLVED FIXED Makoto Kato [:m_kato]
1399337 P2 Rust code should be compiled with THUMB2 as Android/arm's default RESOLVED FIXED Makoto Kato [:m_kato]
1401845 -- Add common android config to ServoBindings.toml RESOLVED FIXED Makoto Kato [:m_kato]
1403074 P4 stylo: crashed [@ core::sync::atomic::atomic_add<usize>] on Android/arm debug build RESOLVED WORKSFORME
1403766 -- Enable thread local arenas on Android RESOLVED FIXED Mike Hommey [:glandium]
1404178 P3 Fix some reftest conditions for Stylo on Android RESOLVED FIXED Makoto Kato [:m_kato]
1404545 P4 stylo: support Fennec's "Honor system font sizes" setting RESOLVED FIXED Hiroyuki Ikezoe (:hiro)
1405548 P3 stylo: getComputedValue might not flush style with stylo RESOLVED FIXED Brian Birtles (:birtles, Feb 26~ parental leave)
1405550 P3 stylo: Adjust assertion count of crash tests for stylo on Android RESOLVED FIXED Makoto Kato [:m_kato]
1406290 -- Support styloVsGecko tests on Fennec/android RESOLVED WONTFIX
1407525 P3 stylo: layout/reftests/webkit-box/webkit-display-values-1.html is always failure on Android/stylo RESOLVED FIXED
1408300 P3 make nsComputedDOMStyle use Servo code for property serialization NEW Emilio Cobos Álvarez (:emilio)
1408847 -- nsLookAndFeel::GetFontImpl should return Roboto RESOLVED FIXED Makoto Kato [:m_kato]
1410336 -- 321402-3.xul and 421203-3.xul are passed on android/stylo RESOLVED WORKSFORME Makoto Kato [:m_kato]
1411522 P3 Skip layout/style/crashtests/439184-1.html on debug build of Android/stylo RESOLVED FIXED Makoto Kato [:m_kato]
1411802 -- Build stylo on Android as default RESOLVED FIXED Makoto Kato [:m_kato]
1417356 -- stylo: Some web extension xpcshell tests on Android are failure with stylo RESOLVED FIXED Makoto Kato [:m_kato]
1420369 P2 stylo: 6.21 - 6.54% remote-tsvg / remote-twitter (android-6-0-armv8-api16, android-7-1-armv8-api16) regression on push db56323cd08f4883e4824199b441a3141be655e5 (Thu Nov 23 2017) NEW Makoto Kato [:m_kato]
1432354 P3 stylo: disable stylo on Android build RESOLVED FIXED Makoto Kato [:m_kato]

19 Total; 2 Open (10.53%); 17 Resolved (89.47%); 0 Verified (0%);


Signoff

Criteria

  • All test cases should be executed
  • All blockers, critical must be fixed and verified or have an agreed-upon timeline for being fixed

Results

Nightly testing

  • There were no blockers found while testing this feature.
  • All pages suggested were tested

Beta testing NA

FAQ

  • Are there any standard smoke tests that Softvision runs for Fennec QA sign-off that we should run in addition to the Quantum top sites?
    • The smoke Test Suit contains a test for "Popular Websites" that checks behavior on: www.yahoo.com, facebook.com, amazon.com and ebay.com in both portrait and landscape mode.
    • Several other suites from the BFT runs test others like cnn.com, maps.google.com, espn.go.com, msn.com, youtube.com etc
  • On which Android devices does Softvision test Fennec?