QA/Fennec/Play HLS

From MozillaWiki
< QA‎ | Fennec
Jump to navigation Jump to search

Overview

Purpose

The purpose of this wiki is to serve as a general test plan for verifying that HLS video can be correctly played in any given condition.

Ownership

Developer contact: X

QA: :Ninu

Overall Status

[LANDED] Nightly 56.0a1

[LANDED] Nightly 59.0a1

[LANDED] Beta 59: 1/23/2018

Testing Summary

Scope of testing

The testing will be focus on

  • making the HLS video watching experience good
  • making sure that the implementation of HLS video controls don't impact other video playing type

Environments

Testing will be performed on following devices:

  • Phones
    • From Android 4.4 to 7.1
  • Tablet
    • From Android 4.4 to 7.1

Specifications

  • HLS Player Landscape
  • HLS Player Portrait

User stories

  • As a user, I want to be able to play HLS videos without glitches and streaming problems, while the other app features are not impacted by a playing video.

Acceptance Criteria

Testing details

Test Cases

Step 

1. Go to goo.gl/LLxEY8 and play the HLS video
R: The video plays correctly and the audio is correctly played

Step 

1. Go to goo.gl/LLxEY8 and play the HLS video in landscape/portrait
R: The video plays correctly and the audio is correctly played
2. Change the device's orientation 
R: The video keeps playing and no interruptions or glitches are triggered

Step 

1. Go to goo.gl/LLxEY8 and play the HLS video
R: The video plays correctly and the audio is correctly played
2. Try the video-player media controls
R: All buttons response is correctly applied in the video progress

Step 

1. Go to goo.gl/LLxEY8 and play the HLS video
R: The video plays correctly and the audio is correctly played
2. Close HLS video player and go to youtube, play a video
R: Any other video player is not impacted by the HLS video player previous use

Step 

Pre-Condition:  Make sure Auto-Play is ON
1. Go to goo.gl/LLxEY8 and play the HLS video
R: The video starts Auto-playing

Step 

Pre-Condition:  Make sure Auto-Play is OFF
1. Go to goo.gl/LLxEY8 and play the HLS video
R: The video should not start Auto-playing

Testing days

Date:05-09-2016

  • Exploratory testing with focus on the Aurora build and creating a test plan for the feature

Date:07-09-2016

  • Functional testing with focus on both Aurora and Nightly builds

Logged issues:

  • HLS video player overlaps other tabs once played - bug 1301043
  • A misleading error is received before playing an HLS video - bug 1301053
  • A HLS video cannot be played twice without reloading the whole page - bug 1301055

Date:08-09-2016

  • Creating test cases and exploratory testing on both Aurora and Nightly builds

Logged issues:

  • HLS videos automatically start playing when auto-play is OFF - bug 1301326

Date:03-10-2016

  • Functional testing with focus on all: Beta Aurora and Nightly builds

Logged issues:

  • Switching tabs will stop the HLS player - bug 1307074
  • HLS Player breaks other video types in other tabs - bug 1307076
  • Interrupting an HLS video with Task manager will lead to incorrect video resume - bug 1307078

Date:22-06-2017

  • Exploratory testing with focus on links from duplicated bugs and GitHub issues with devices Asus ZenPad 8 (Android 6.0.1) and LG G4 (Android 5.1), build Nightly 56.0a1 from 2017/06/21
  • New bugs found during testing
    • bug 1375478 - HLS: "Full-screen" button not displayed in the right position
    • bug 1375474 - Video doesn't play on observator.tv

Date:03-07-2017

  • TCs and exploratory testing with devices Asus ZenPad 8 (Android 6.0.1) on build Nightly 56.0a1 from 2017/07/03
    • Prerequisite: "media.mediasource.enabled" set to false
  • New bugs found during testing
    • bug 1377873 - [Fennec][HLS] Issues when changing device orientation
    • bug 1377891 - [Fennec][HLS] Issues with videos from twitch.tv

Date:06-07-2017

  • TCs and exploratory testing for Mid-Nightly Signoff on latest Nightly build
    • Prerequisite: "media.mediasource.enabled" set to false

Date:31-07-2017

Date:22-11-2017

  • Exploratory testing and bug investigation on latest Nightly build (59.0a1-21/11/2017)
    • bug 1307074 - Switching tabs will stop the HLS player
    • bug 1307078 - Interrupting an HLS video with Task manager will lead to incorrect video resume
    • bug 1308205 - While in landscape, HLS videos are played in FullScreen
    • bug 1302729 - Media control notification is not displayed when HLS video is playing
    • bug 1307076 - HLS Player breaks other video types in other tabs
    • bug 1310899 - HLS VideoView prevents interacting with page, requires back button to close
    • bug 1377891 - [Fennec][HLS] Issues with videos from twitch.tv.

Date:23-11-2017

Date:27-11-2017

Date:5-12-2017

Date:20-12-2017

  • Pre-Beta signoff: Green

Date:25-01-2018

Feature full testing on Beta 59:

  • Prestigio Grace X5 (Android 4.4.2)
  • Huawei Nexus 6P (Android 8.0)
  • HTC Desire 820 (Android 6.0.1)

Date: 30-01-2018

Feature full testing on Beta 59:

  • LG G4 (Android 5.1)
  • Samsung Galaxy S8 (Android 7.0)

Bug Work

Bug Work

  • Meta bug
    • bug 1345752 - (HLS_on_Fennec) [META][HLS] Make Gecko native support HLS on Fennec

Bugzilla query error

Array ( [type] => error [message] => http-bad-status [params] => Array ( [0] => 429 [1] => Unknown Error ) ) 1


Bug fixed

Signoff

Criteria

  • All test cases should be executed
  • All blockers, critical issues must be fixed and verified or have an agreed-upon timeline for being fixed


Results

Nightly testing

  • Done

Beta testing

  • n/a
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Fennec/Play_HLS&oldid=1188999"