QA/IntersectionObserver

From MozillaWiki
< QA
Revision as of 18:07, 15 February 2017 by Twalker (talk | contribs)
Jump to navigation Jump to search

Overview

Purpose

This wiki details the testing that will be performed by QA for the Intersection Observer API. It defines the overall testing requirements and provides an integrated view of the project test activities. Its purpose is to document:

  • Areas of risk
  • What will be tested
  • How testing will be performed
    • criteria
    • deliverables
    • ownership
    • schedule

To help ensure the best possible release and long term maintenance of this feature.

Risk Analysis

Risk area Requirement Status
Ad intensive web pages No regression in interaction performance TBD
Security Fuzz testing TBD
Stability Crash rate TBD
Add-ons Breaks ad-block type add-ons?? TBD

Testing Strategy

Scope

Test Areas Covered
Private Window Yes
Multi-Process Enabled Yes
Multi-process Disabled Yes
Stability Yes
UI
Display (HiDPI) Yes
Interraction (scroll, zoom) Yes
Add-ons
Interferes with ad-blocking Add-ons? Yes
Security
Fuzzing Yes
Web Compatibility
Testing against target sites Yes

If it's not listed above, it is currently out of scope.

Test Strategy

Objectives

Criteria Description Metric without IO API IO API value Criteria Met? QA Owner
Manual testing Interaction performance (test results) (test results) TBD (Date status updated) Tracy
Unit testing Automated tests (test results) (test results) TBD (Date status updated) Tobias
Fuzz testing Automated tests (test results) (test results) TBD (Date status updated) Fuzzing team (tbd)
Stability Crash reports telemetry/crash stats no notable stability regression TBD (Date status updated) Tobias

Testcases

Testing should happen on Ad-intensive websites. In general news sites tent to be rich of Ads. Therefore the top news sites on Alexa are a great source for suitable websites.

[1] The top 500 news sites on the web

List of known sites using the Intersection Observer API:

Possible test scenarios:

  • Open a single news site. Wait till it is fully loaded to make sure all Ads are properly initialized. If there is content to scroll, scroll up and down a couple of times. Click on some articles to trigger a refresh.
  • Open a multiple news sites in different tabs. Wait till every page is fully loaded to make sure all Ads are properly initialized. Switch between tabs from time too time. Close and open new tabs randomly. If there is content to scroll, scroll up and down a couple of times. Click on some articles to trigger a refresh.
  • Repeat scenarios above but let the browser untouched for a longer period of time. Then proceed.

Environments

Full Testing will be performed using Nightly for Desktop builds on:

  • Windows
  • Mac OS X
  • Linux

Full Testing will be performed using Nightly for Mobile builds on:

  • Android

Channel dependent settings (configs) and environment setups

  • In Nightly (and DevEd when available)- dom.IntersectionObserver.enabled needs to be set to "true" in about:config
  • If any kind of Ad-blocker is installed/in use it needs to be uninstalled/disabled

Project Information

Ownership

Developer contacts: Tobias Schneider
QA: Tracy Walker
QA Peer: TBD

Builds

This section should contain links for builds with the feature

Schedule

The following table identifies the anticipated testing period available for test execution.

Project phase Start Date End Date
Start project
Study documentation/specs received from developers
QA - Test plan creation
QA - Test cases/Env preparation
QA - Nightly Testing
QA - Aurora Testing
QA - Beta Testing
Release Date

References

  • List and links for specs
    • [2] Intersection Observer Spec
    • [3] Intersection Observer API
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/IntersectionObserver&oldid=1163044"