QA/WebVR

Overview

Purpose

• The purpose of this wiki is to serve as a general test plan for the WebVR API.
• A separate test plan will cover the testing for A-Frame.

Ownership

Developer contacts: Kearwood Kip Gilbert
QA: Naoki Hirata (:nhirata_), Cornel Ionce (:cornel_ionce)

Testing Summary

Scope of testing

In Scope

The current testing scope is to ensure that the WebVR API:

1. does not have regressions in performance or latency
2. is stable
3. is reliable ( visibility )

Requirements for testing

Environments

Testing will be performed on following OSes (x64 infrastructures):

• Windows 10

Hardware

The minimum system requirements for Windows OS are:

• Graphics
  • GTX 970 equivalent or greater
  • AMD 290 equivalent or greater
  • Quadro k600 or newer
• CPU
  • Intel i5-4590 equivalent or greater
  • AMD FX 8320 or greater
• 8GB+ RAM

Devices

• Oculus Development Kit 2 with positional tracker
  • Note: soon to change to CV1

Test Strategy

Test Objectives

Following table contains main features to be tested:

Ref	Function	Test Objective	Evaluation Criteria
1	Basic scenarios	TBD	TBD

Test Types

Type of Testing	Required (Yes/No)	Owners	Details/Comments
Manual	Yes	Moz QA Rel team
Automation
Performance	No
Security	No
Usability	No
Metrics/Telemetry	TBD

Builds

This section should contain links for builds with the feature -

Nightly builds with can be found at link

Testing Tools

Detail the tools to be used for testing, for example see the following table:

Process	Tool
Test plan creation	Mozilla wiki
Test case creation	Google docs
Test case execution	Google docs
Bugs management	Bugzilla

Status

Overview

'[LANDED] Nightly 46: first landed on - 2016.01.05

Risk analysis

- No Risks found at the moment

References

Meta: Bug 1218482 - Enable WebVR By Default on all platforms

Test cases

Test suites

Full Test Suite - can be reached at this link

Bugs Work

Tracking bug: Meta bug 1218482

Bug fix verification

Bug No	Summary	Status	Firefox Version
1218482	[webvr] Enable WebVR By Default on all platforms	Verified - Fixed	Firefox 46
1182048	Make webvr and e10s work together	Verified - Fixed	Firefox 46

Logged bugs

Sign off

Criteria

• All test cases should be executed
• All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)

Results

Merge to Aurora Sign-off

• [DONE] mid-Aurora 46 (2016-02-15).
  • There were no blockers found while testing this feature.
  • Manual testing covered Windows 10 (x64) and Mac OS X 10.10.5.

Merge to Beta Sign-off

• [BLOCKED] Pre-beta 46 (2016-03-04).
  • Crash 1239055 is severely affecting the feature (considered a blocker for the release).
  • Manual testing covered Windows 10 (x64) on two machines.