QA/Fennec/Play HLS

From MozillaWiki
< QA‎ | Fennec
Jump to: navigation, search

Overview

Purpose

The purpose of this wiki is to serve as a general test plan for verifying that HLS video can be correctly played in any given condition.

Ownership

Developer contact: X

QA: :Ninu

Overall Status

[LANDED] Nightly 56.0a1

[LANDED] Nightly 59.0a1

[LANDED] Beta 59: 1/23/2018

Testing Summary

Scope of testing

The testing will be focus on

  • making the HLS video watching experience good
  • making sure that the implementation of HLS video controls don't impact other video playing type

Environments

Testing will be performed on following devices:

  • Phones
    • From Android 4.4 to 7.1
  • Tablet
    • From Android 4.4 to 7.1

Specifications

  • HLS Player Landscape
  • HLS Player Portrait

User stories

  • As a user, I want to be able to play HLS videos without glitches and streaming problems, while the other app features are not impacted by a playing video.

Acceptance Criteria

Testing details

Test Cases

Step 

1. Go to goo.gl/LLxEY8 and play the HLS video
R: The video plays correctly and the audio is correctly played

Step 

1. Go to goo.gl/LLxEY8 and play the HLS video in landscape/portrait
R: The video plays correctly and the audio is correctly played
2. Change the device's orientation 
R: The video keeps playing and no interruptions or glitches are triggered

Step 

1. Go to goo.gl/LLxEY8 and play the HLS video
R: The video plays correctly and the audio is correctly played
2. Try the video-player media controls
R: All buttons response is correctly applied in the video progress

Step 

1. Go to goo.gl/LLxEY8 and play the HLS video
R: The video plays correctly and the audio is correctly played
2. Close HLS video player and go to youtube, play a video
R: Any other video player is not impacted by the HLS video player previous use

Step 

Pre-Condition:  Make sure Auto-Play is ON
1. Go to goo.gl/LLxEY8 and play the HLS video
R: The video starts Auto-playing

Step 

Pre-Condition:  Make sure Auto-Play is OFF
1. Go to goo.gl/LLxEY8 and play the HLS video
R: The video should not start Auto-playing

Testing days

Date:05-09-2016

  • Exploratory testing with focus on the Aurora build and creating a test plan for the feature

Date:07-09-2016

  • Functional testing with focus on both Aurora and Nightly builds

Logged issues:

  • HLS video player overlaps other tabs once played - bug 1301043
  • A misleading error is received before playing an HLS video - bug 1301053
  • A HLS video cannot be played twice without reloading the whole page - bug 1301055

Date:08-09-2016

  • Creating test cases and exploratory testing on both Aurora and Nightly builds

Logged issues:

  • HLS videos automatically start playing when auto-play is OFF - bug 1301326

Date:03-10-2016

  • Functional testing with focus on all: Beta Aurora and Nightly builds

Logged issues:

  • Switching tabs will stop the HLS player - bug 1307074
  • HLS Player breaks other video types in other tabs - bug 1307076
  • Interrupting an HLS video with Task manager will lead to incorrect video resume - bug 1307078

Date:22-06-2017

  • Exploratory testing with focus on links from duplicated bugs and GitHub issues with devices Asus ZenPad 8 (Android 6.0.1) and LG G4 (Android 5.1), build Nightly 56.0a1 from 2017/06/21
  • New bugs found during testing
    • bug 1375478 - HLS: "Full-screen" button not displayed in the right position
    • bug 1375474 - Video doesn't play on observator.tv

Date:03-07-2017

  • TCs and exploratory testing with devices Asus ZenPad 8 (Android 6.0.1) on build Nightly 56.0a1 from 2017/07/03
    • Prerequisite: "media.mediasource.enabled" set to false
  • New bugs found during testing
    • bug 1377873 - [Fennec][HLS] Issues when changing device orientation
    • bug 1377891 - [Fennec][HLS] Issues with videos from twitch.tv

Date:06-07-2017

  • TCs and exploratory testing for Mid-Nightly Signoff on latest Nightly build
    • Prerequisite: "media.mediasource.enabled" set to false

Date:31-07-2017

Date:22-11-2017

  • Exploratory testing and bug investigation on latest Nightly build (59.0a1-21/11/2017)
    • bug 1307074 - Switching tabs will stop the HLS player
    • bug 1307078 - Interrupting an HLS video with Task manager will lead to incorrect video resume
    • bug 1308205 - While in landscape, HLS videos are played in FullScreen
    • bug 1302729 - Media control notification is not displayed when HLS video is playing
    • bug 1307076 - HLS Player breaks other video types in other tabs
    • bug 1310899 - HLS VideoView prevents interacting with page, requires back button to close
    • bug 1377891 - [Fennec][HLS] Issues with videos from twitch.tv.

Date:23-11-2017

Date:27-11-2017

Date:5-12-2017

Date:20-12-2017

  • Pre-Beta signoff: Green

Date:25-01-2018

Feature full testing on Beta 59:

  • Prestigio Grace X5 (Android 4.4.2)
  • Huawei Nexus 6P (Android 8.0)
  • HTC Desire 820 (Android 6.0.1)

Date: 30-01-2018

Feature full testing on Beta 59:

  • LG G4 (Android 5.1)
  • Samsung Galaxy S8 (Android 7.0)

Bug Work

Bug Work

  • Meta bug
    • bug 1345752 - (HLS_on_Fennec) [META][HLS] Make Gecko native support HLS on Fennec
Full Query
ID Priority Summary Status Assigned to
577084 -- Implement HTTP Live Streaming (HLS) for audio and video on Android RESOLVED
1341990 -- Introduce Exoplayer into build system RESOLVED Kilik Kuo [:kikuo] (inactive)
1350241 -- [Fennec][HLS] Create a demuxer-like component based on ExoPlayer. RESOLVED Kilik Kuo [:kikuo] (inactive)
1350246 -- [Fennec][HLS] Provide HLSDecoder / HLSDemuxer / HLSTrackDemuxer / HLSResouce for Gecko. RESOLVED James Cheng[:JamesCheng]
1350250 -- [Fennec][HLS] HLSResource should obtain buffer information from GeckoHlsPlayer.java RESOLVED James Cheng[:JamesCheng]
1350253 -- [Fennec][HLS] HLSDemuxer/HLSTrackDemuxer in Gecko should be able to get media & sample information from GeckoHlsPlayer.java RESOLVED Kilik Kuo [:kikuo] (inactive)
1350842 P3 Remove codes which are related with using external video player to play unsupported type media RESOLVED
1357984 P5 [META] Test case for HLS native supprts RESOLVED
1365227 -- AVCC NAL converter should generate extradata. RESOLVED Jean-Yves Avenard [:jya]
1365505 -- [Fennec][HLS] Turn on HLS native support not only for Nightly RESOLVED Kilik Kuo [:kikuo] (inactive)
1368904 P3 [Fennec][HLS] Handle unexpected OnError native callback from java GeckoHlsPlayer RESOLVED Kilik Kuo [:kikuo] (inactive)
1368907 -- [Fennec][HLS] Unify the naming from Hls into HLS for our Gecko HLS implementation RESOLVED James Cheng[:JamesCheng]
1368954 -- [Fennec][HLS] Separate source code which will affect generated JNI wrappers to avoid build bustage. RESOLVED Kilik Kuo [:kikuo] (inactive)
1368959 -- [Fennec][HLS] Set the MediaInfo duration to zero if the source is live streaming RESOLVED James Cheng[:JamesCheng]
1369598 -- [Fennec][HLS] Make HLSDemuxer initialized after underlying GekcoHlsPlayer is ready with data. RESOLVED Kilik Kuo [:kikuo] (inactive)
1371247 P5 [Fennec][HLS] Fix potential defects in exoplayer2. RESOLVED
1372501 -- [Fennec][HLS] Allow VideoDocument supporting HLS RESOLVED James Cheng[:JamesCheng]
1372841 -- Make HLSResource inherit from MediaResource instead of BaseMediaResource RESOLVED James Cheng[:JamesCheng]
1373119 -- [Fennec][HLS] Fix "explicit null dereferenced" case in GeckoHLSDemuxerWrapper.getSample RESOLVED Kilik Kuo [:kikuo] (inactive)
1373162 -- Crash in java.lang.NullPointerException: at org.mozilla.gecko.media.GeckoHlsPlayer$ComponentListener.onVideoInputFormatChanged(GeckoHlsPlayer.java) RESOLVED Kilik Kuo [:kikuo] (inactive)
1373377 -- Crash in java.lang.IllegalArgumentException: Unsupported type: 3 at org.mozilla.gecko.media.GeckoHlsPlayer.buildMediaSource(GeckoHlsPlayer.java) RESOLVED James Cheng[:JamesCheng]
1373500 -- [Fennec][HLS] Set DEBUG flag based on AppConstants.NIGHTLY_BUILD || AppConstants.DEBUG_BUILD RESOLVED James Cheng[:JamesCheng]
1374240 P1 [Fennec][HLS] Video sound stops working after a few seconds on Dailymotion RESOLVED James Cheng[:JamesCheng]
1374899 -- [Fennec][HLS] Handle InvalidResponseCodeException exception raised by Exoplayer with some url that may cause redirection. RESOLVED James Cheng[:JamesCheng]
1374953 P3 [Fennec][HLS] Take mimetype "audio/mpegurl" as HLS media type RESOLVED James Cheng[:JamesCheng]
1375474 P1 Video doesn't play on observator.tv VERIFIED James Cheng[:JamesCheng]
1375478 -- HLS: "Full-screen" button not displayed in the right position RESOLVED
1376966 P3 [Fennec][HLS] Fix defects in GeckoHlsPlayer.java RESOLVED James Cheng[:JamesCheng]
1376967 P3 [Fennec][HLS] Fix defects in GeckoHlsRendererBase.java RESOLVED Kilik Kuo [:kikuo] (inactive)
1377520 P3 [Fennec][HLS] Support encrypted HLS content playback RESOLVED
1377891 P5 [Fennec][HLS] twitch.tv does not play when media.mediasource.enabled is set to false. RESOLVED
1378852 -- [Fennec][HLS] getBufferedPosition in GeckoHlsPlayer should always return a value >= 0. RESOLVED Kilik Kuo [:kikuo] (inactive)
1379056 -- [Fennec][HLS] GeckoHlsPlayer should handle the case when obtain a negative duration from ExoPlayer. RESOLVED Kilik Kuo [:kikuo] (inactive)
1379856 P3 [Fennec][HLS] Move ExoPlayer main message handler to non-gecko main thread. RESOLVED Kilik Kuo [:kikuo] (inactive)
1379866 P1 [Fennec][HLS] App crashed a while later after stopping HLS playback. RESOLVED James Cheng[:JamesCheng]
1380201 P3 [Fennec][HLS] Make access to mIsPlayerInitDone & mResourceCallback guarded. RESOLVED Kilik Kuo [:kikuo] (inactive)
1380237 -- [Fennec][HLS] Handle audio format change by changing the stream id. RESOLVED James Cheng[:JamesCheng]
1381717 P3 [Fennec][HLS] Fix code defects found by Coverity in GeckoHlsPlayer.java RESOLVED James Cheng[:JamesCheng]
1382151 -- [Fennec][HLS] Pause ExoPlayer when player is paused to avoid OOM. RESOLVED James Cheng[:JamesCheng]
1384578 -- Crash in java.lang.NullPointerException: Null native pointer at org.mozilla.gecko.media.GeckoHLSDemuxerWrapper$Callbacks.onError(Native Method) RESOLVED James Cheng[:JamesCheng]
1387963 -- HLS tests crash on all versions Autophone RESOLVED
1418766 P1 Crash in java.lang.OutOfMemoryError: at com.google.android.exoplayer2.upstream.DefaultAllocator.allocate(DefaultAllocator.java) RESOLVED James Cheng[:JamesCheng]
1419249 P2 [Fennec][HLS] Audio only playback will not work correctly after seek. RESOLVED James Cheng[:JamesCheng]
1419669 P3 [Fennec][HLS] Refine switching Exoplayer to play/pause logic in GeckoHlsPlayer RESOLVED
1420030 -- [Fennec][HLS] Change HLS logging from BuildConfig.NIGHTLY_BUILD || BuildConfig.DEBUG_BUILD to !BuildConfig.MOZILLA_OFFICIAL RESOLVED James Cheng[:JamesCheng]
1420192 -- Youtube and twitch.com live videos don't work when auto-play is off RESOLVED Alastor Wu [:alwu]
1424168 P2 [Fennec][HLS] Exoplayer causes high memory usage on some websites. RESOLVED James Cheng[:JamesCheng]
1478843 P1 Cross-origin audio leak in HLS VERIFIED Paul Adenot (:padenot)

48 Total; 0 Open (0%); 46 Resolved (95.83%); 2 Verified (4.17%);


Bug fixed

Signoff

Criteria

  • All test cases should be executed
  • All blockers, critical issues must be fixed and verified or have an agreed-upon timeline for being fixed


Results

Nightly testing

  • Done

Beta testing

  • n/a
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Fennec/Play_HLS&oldid=1188999"