View source for QA/Firefox 4/Test Plan/WebM Support

= WebM Support  =

*'''Development Status:''' - Completed (Alpha 5) 
*'''Feature Testing:''' - On-going
*'''Team:''' Dev: cpearce, roc, QA: juanb 
*'''Tracking Bugs''': 

=== Feature Description  ===
WebM is a new open source video format. Firefox 4 will support this new video format similar to the way it supports Ogg.

<br>

= [[#Feature_Release_Readiness_Assessment_Details|Feature Release Readiness Assessment]]  =

The table below provides a top level go/no go assessment of whether the feature is release ready for the given milestone. 

{| class="fullwidth-table"
|-
! Milestone 
! Assessment
|-
| [[#Alpha 5]] 
| In-house QA has not engaged yet fully in testing WebM. If any testing has been done manually it is from community.
|-
| [[#Beta1]]
| N/A
|-
| [[#Beta2]] 
| N/A
|-
| [[#Beta3]] 
| N/A
|}

[[#Feature_Description|top]]

= [[#Feature_Documentation_Details|Feature Documentation]]  =

{| class="fullwidth-table"
|-
! Item 
! Description 
! Status
|-
| [[#Project_Wiki]] 
| Wiki Links to all feature related entries 
| 
|-
| [[#Developer_Links|#Developer_Links (blogs)]] 
| Developer links to feature related sites 
| 
|-
| [[#Other_Docs]] 
| Web links to feature related sites 
| 
|-
| [[#Developer_QA_Review]] 
| Details from developer and qa discussions regarding feature test strategies and issues. 
| 
|}

[[#Feature_Description|top]] 

= [[#Feature_Bug_Management_Details|Feature Bug Management]]  =

{| class="fullwidth-table"
|-
! Item 
! Description 
! Status
|-
| [[#Bug_Tracking]] 
| Top level bugs tracking feature 
| 
|-
| [[#Bug_Verification]] 
| Feature bugs that need verification 
| 
|-
| [[#Bug_Triage]] 
| Links triage bug tasks 
| 
|}

[[#Feature_Description|top]] 

= [[#Feature_Test_Items_Details|Feature Test Items]]  =

The table below provides a breakdown of all feature items that should be covered and how they will be tested. Not all items will be covered by internal QA team members. It is important to list what '''should''' be covered. If it is not covered, list it as not covered. 

'''Note: not all items listed below will apply for a given feature''' 

{| class="fullwidth-table"
|-
! Test Item 
! Description 
! Covered By 
! Status
|-
| Item 1 
| Item 1 Description 
| Developer Tests 
| 
|-
| Item 2 
| Item 2 Description 
| Beta tester exposure 
| 
|-
| [[#Localization]] 
| Feature localization 
| 
| 
|-
| [[#Accessibility]] 
| Feature accessibility 
| 
| 
|-
| [[#Plugins]] 
| Plugins compatibility 
| 
| 
|-
| [[#Addons]] 
| Addons compatibility 
| 
| 
|-
| [[#Topsites]] 
| Top internet sites compatibilities 
| 
| 
|}

[[#Feature_Description|top]] 

<br> 

= [[#Feature_Tests_Details|Feature Tests]]  =

== [[#Automated_Tests_Details|Automated Tests]]  ==

{| class="fullwidth-table"
|-
! Item 
! Description 
! Status
|-
| [[#Developer_Tests]] 
| Links to automated developer tests 
| Information not yet gathered.
|-
| [[#Mozmill_Tests]] 
| Links to automated mozmill feature test cases 
| Will consult with Henrik to see how applicable these can be.
|}

== [[#Manual_Tests_Details|Manual Tests]]  ==

{| class="fullwidth-table"
|-
! Item 
! Description 
! Status
|-
| [[#Smoke_Tests]] 
| link to smoke tests 
| Not started.
|-
| [[#Regression_Tests]] 
| link to BFT and/or regression tests 
| Not started.
|-
| [[#Functional_Tests]] 
| link to FFT and/or complete functional tests 
| Not started.
|}

[[#Feature_Description|top]]

= [[#Community_Test_Events_Details|Community Test Events]]  =

{| class="fullwidth-table"
|-
! Item 
! Description 
! Status
|-
| [[#Testdays]] 
| Links to test day event results for feature 
| 
|-
| [[#Bugdays]] 
| Links to bug day event results for feature 
| 
|-
| [[#Meetups]] 
| Links to Meetup events for feature 
| 
|}

[[#Feature_Description|top]] 

= [[#Feature_Documentation|Feature Documentation Details]]  =

== Project Wiki  ==
Project wiki not available. Refer to developer links with blogs on the subject.

[[#Feature_Documentation|top]] 

== Developer Links  ==
* Mozillians Blogs on WebM:
** http://weblogs.mozillazine.org/roc/archives/2010/05/its_a_relief_to.html
** http://blog.mozilla.com/blog/2010/05/19/open-web-open-video-and-webm/
** http://hacks.mozilla.org/2010/05/firefox-youtube-and-webm/
* How to try WebM videos:
** http://googlesystem.blogspot.com/2010/05/how-to-play-webm-video-on-youtube.html
* YouTube samples:
** http://www.youtube.com/watch?v=Z8EA7EbFX4k&webm=1&html5=1
**http://www.youtube.com/watch?v=rLxQiI8c1Bs&webm=1&html5=1
* A non-YouTube example:
** http://pearce.org.nz/video/bruce.webm


[[#Feature_Documentation|top]]

== Other Docs  ==
Preview builds of Firefox with WebM support are already
available at http://nightly.mozilla.org/webm.  Additionally,
the Participatory Culture Foundation has updated their Miro
Video Converter to be the first tool to produce WebM videos.

* http://www.webmproject.org/
* http://www.webmproject.org/license/
* http://bluishcoder.co.nz/2010/05/20/vp8-webm-firefox.html
* http://nightly.mozilla.org/webm/
* http://www.participatoryculture.org/
* http://www.mirovideoconverter.com/


[[#Feature_Documentation|top]]

== Developer QA Review  ==

The QA person responsible for the feature should hold a formal interview with the lead developer or feature champion. Below are questions that should be asked in the interview: 

*Do we have automated tests for the feature?
** Not yet answered.
*What do they cover?
** nya 
*What do they not cover?
** nya
*How well do they cover the feature? 
** nya
*What are the important areas we should focus on? 
** Basically, we want to know if there are WebM videos out there which don't play/seek like you'd expect them to. For example: videos that play too slow or too fast. Videos that don't play at all. Seeking (clicking on the scrubber bar to change the playback position) which doesn't work or works funny/slow. Sound or video not working, or visual artefacts (like bug 566987) during playback or seeking. Playback with the audio not in sync with the video. 
*What are the dependencies? 
** You mean what libraries do we use? We're using Google's libvpx for VP8 decoding, Xiph.org's libvorbis for Vorbis decoding, and Matthew Gregan's (a MoCo employee) libnestegg for WebM demuxing. 
*What is our comfort level with this feature in its current state? 
** The WebM playback logic is shared with the Ogg playback logic, so all the HTML5-spec compliance/UI stuff should be the same as Ogg backend. Theoretically the only new bugs you find should be in the underlying decoding logic, not the playback logic. The logic surrounding the HTML5 controls/event architecture should be backend independent now, so...
*What feedback would you like from QA?
** It would be good to know that you guys have tried playing a variety of WebM videos on a variety of platforms and found no problems. :)
** It would be good to know that playback works ok on PPC MacOSX, we've not tried WebM playback on that platform, as we don't have a PPC Mac in our office. 

[[#Feature_Documentation|top]]

= [[#Feature_Release_Readiness_Assessment|Feature Release Readiness Assessment Details]]  =

== Alpha 5 ==
WebM support was included in Alpha 5 without much in-house QA testing, and at the moment is covered by nightly usage.

== Beta1  ==
By Beta 1 we will have litmus tests and an assessment of automated tested coverage, as well as a thorough pass by in-house QA. The similarities with Ogg support from a UI perspective should allow us to build upon existing tests and approaches.

== Beta2  ==

== Beta3  ==

= [[#Feature_Bug_Management|Feature Bug Management Details]]  =

== Bug Tracking  ==

*Top level bugs tracking feature. Include any relevant bug queries that are helpful for tracking feature status.

{| class="fullwidth-table"
|-
! Query Name 
! Description
|-
| bugzilla query url link 
| query description
|}

[[#Feature_Bug_Management|top]] 

== Bug Verification  ==

*Feature bugs that need verification

[[#Feature_Bug_Management|top]] 

== Bug Triage  ==
Ideally we will triage WebM bugs daily, confirming new unconfirmed bugs, and making sure we give them the right exposure and bubble them up accordingly. A quick search for new and unconfirmed bugs can be found here:

* https://bugzilla.mozilla.org/buglist.cgi?quicksearch=webm

We should take a look at a list like this as frequently as daily.

[[#Feature_Bug_Management|top]]

= [[#Feature_Test_Items|Feature Test Items Details]]  =

== Localization  ==
Does this feature require localization testing?
* Context menu items for video, or video controls text should be localized.
* Must investigate RTL localization requirements for these elements of video.

[[#Feature_Test_Items|top]]

== Accessibility  ==
* Video controls must have accessibility controls.
* Keyboard strokes invoking the video controls must be tested for functionality.

[[#Feature_Test_Items|top]]

== Plugins  ==

* Interactions between WebM video and other types of video formats such as Ogg and Flash must be tested for compatibility. Loading tabs with different types of video formats should work well.

[[#Feature_Test_Items|top]]

== Addons  ==
* Check for top addons compatibility while loading and playing WebM videos

[[#Feature_Test_Items|top]]

== Topsites  ==
* Check for compatibility with major web sites, such as Facebook, Hulu, Yahoo, YouTube, as well as those sites which have content for WebM videos.

[[#Feature_Test_Items|top]]

= [[#Feature_Tests|Feature Tests Details]]  =

== [[#Automated_Tests|Automated Tests Details]]  ==

=== Developer Tests  ===

*Links to automated developer tests

[[#Feature_Tests|top]] 

=== Mozmill Tests  ===

If a particular feature needs manual tests which should also be covered by Mozmill tests please add the "''[mozmill-test-needed]''" whiteboard entry to the feature implementation or regression bug. 

List of Mozmill Tests: 

*Links to automated mozmill feature test cases

[[#Feature_Tests|top]] 

== [[#Manual_Tests|Manual Tests Details]]  ==

=== Smoke_Tests  ===

*links to litmus smoke tests or description

[[#Feature_Tests|top]] 

=== Regression_Tests  ===

*links to litmus BFT and/or regression tests description

[[#Feature_Tests|top]] 

=== Functional_Tests  ===

*links to litmus FFT and/or complete functional tests description

[[#Feature_Tests|top]] 

= [[#Community_Test_Events|Community Test Events Details]]  =

== Testdays  ==

* No testday has been scheduled yet.

== Bugdays  ==

* No bugday has been scheduled yet.

== Meetups  ==

* No meetup has been scheduled yet. For testing purposes testdays and bugdays seem to be of better value.