QA/Firefox 4/Test Plan/WebM Support

From MozillaWiki
< QA‎ | Firefox 4‎ | Test Plan
Jump to: navigation, search

WebM Support

  • Development Status: - Completed (Alpha 5)
  • Feature Testing: - On-going
  • Team: Dev: cpearce, roc, QA: juanb
  • Tracking Bugs:

Feature Description

WebM is a new open source video format. Firefox 4 will support this new video format similar to the way it supports Ogg.


Feature Release Readiness Assessment

The table below provides a top level go/no go assessment of whether the feature is release ready for the given milestone.

Milestone Assessment
#Alpha 5 In-house QA has not engaged yet fully in testing WebM. If any testing has been done manually it is from community.
#Beta1 N/A
#Beta2 N/A
#Beta3 N/A

top

Feature Documentation

Item Description Status
#Project_Wiki Wiki Links to all feature related entries
#Developer_Links (blogs) Developer links to feature related sites
#Other_Docs Web links to feature related sites
#Developer_QA_Review Details from developer and qa discussions regarding feature test strategies and issues.

top

Feature Bug Management

Item Description Status
#Bug_Tracking Top level bugs tracking feature
#Bug_Verification Feature bugs that need verification
#Bug_Triage Links triage bug tasks

top

Feature Test Items

The table below provides a breakdown of all feature items that should be covered and how they will be tested. Not all items will be covered by internal QA team members. It is important to list what should be covered. If it is not covered, list it as not covered.

Note: not all items listed below will apply for a given feature

Test Item Description Covered By Status
Item 1 Item 1 Description Developer Tests
Item 2 Item 2 Description Beta tester exposure
#Localization Feature localization
#Accessibility Feature accessibility
#Plugins Plugins compatibility
#Addons Addons compatibility
#Topsites Top internet sites compatibilities

top


Feature Tests

Automated Tests

Item Description Status
#Developer_Tests Links to automated developer tests Information not yet gathered.
#Mozmill_Tests Links to automated mozmill feature test cases Will consult with Henrik to see how applicable these can be.

Manual Tests

Item Description Status
#Smoke_Tests link to smoke tests Not started.
#Regression_Tests link to BFT and/or regression tests Not started.
#Functional_Tests link to FFT and/or complete functional tests Not started.

top

Community Test Events

Item Description Status
#Testdays Links to test day event results for feature
#Bugdays Links to bug day event results for feature
#Meetups Links to Meetup events for feature

top

Feature Documentation Details

Project Wiki

Project wiki not available. Refer to developer links with blogs on the subject.

top

Developer Links


top

Other Docs

Preview builds of Firefox with WebM support are already available at http://nightly.mozilla.org/webm. Additionally, the Participatory Culture Foundation has updated their Miro Video Converter to be the first tool to produce WebM videos.


top

Developer QA Review

The QA person responsible for the feature should hold a formal interview with the lead developer or feature champion. Below are questions that should be asked in the interview:

  • Do we have automated tests for the feature?
    • Not yet answered.
  • What do they cover?
    • nya
  • What do they not cover?
    • nya
  • How well do they cover the feature?
    • nya
  • What are the important areas we should focus on?
    • Basically, we want to know if there are WebM videos out there which don't play/seek like you'd expect them to. For example: videos that play too slow or too fast. Videos that don't play at all. Seeking (clicking on the scrubber bar to change the playback position) which doesn't work or works funny/slow. Sound or video not working, or visual artefacts (like bug 566987) during playback or seeking. Playback with the audio not in sync with the video.
  • What are the dependencies?
    • You mean what libraries do we use? We're using Google's libvpx for VP8 decoding, Xiph.org's libvorbis for Vorbis decoding, and Matthew Gregan's (a MoCo employee) libnestegg for WebM demuxing.
  • What is our comfort level with this feature in its current state?
    • The WebM playback logic is shared with the Ogg playback logic, so all the HTML5-spec compliance/UI stuff should be the same as Ogg backend. Theoretically the only new bugs you find should be in the underlying decoding logic, not the playback logic. The logic surrounding the HTML5 controls/event architecture should be backend independent now, so...
  • What feedback would you like from QA?
    • It would be good to know that you guys have tried playing a variety of WebM videos on a variety of platforms and found no problems. :)
    • It would be good to know that playback works ok on PPC MacOSX, we've not tried WebM playback on that platform, as we don't have a PPC Mac in our office.

top

Feature Release Readiness Assessment Details

Alpha 5

WebM support was included in Alpha 5 without much in-house QA testing, and at the moment is covered by nightly usage.

Beta1

By Beta 1 we will have litmus tests and an assessment of automated tested coverage, as well as a thorough pass by in-house QA. The similarities with Ogg support from a UI perspective should allow us to build upon existing tests and approaches.

Beta2

Beta3

Feature Bug Management Details

Bug Tracking

  • Top level bugs tracking feature. Include any relevant bug queries that are helpful for tracking feature status.
Query Name Description
bugzilla query url link query description

top

Bug Verification

  • Feature bugs that need verification

top

Bug Triage

Ideally we will triage WebM bugs daily, confirming new unconfirmed bugs, and making sure we give them the right exposure and bubble them up accordingly. A quick search for new and unconfirmed bugs can be found here:

We should take a look at a list like this as frequently as daily.

top

Feature Test Items Details

Localization

Does this feature require localization testing?

  • Context menu items for video, or video controls text should be localized.
  • Must investigate RTL localization requirements for these elements of video.

top

Accessibility

  • Video controls must have accessibility controls.
  • Keyboard strokes invoking the video controls must be tested for functionality.

top

Plugins

  • Interactions between WebM video and other types of video formats such as Ogg and Flash must be tested for compatibility. Loading tabs with different types of video formats should work well.

top

Addons

  • Check for top addons compatibility while loading and playing WebM videos

top

Topsites

  • Check for compatibility with major web sites, such as Facebook, Hulu, Yahoo, YouTube, as well as those sites which have content for WebM videos.

top

Feature Tests Details

Automated Tests Details

Developer Tests

  • Links to automated developer tests

top

Mozmill Tests

If a particular feature needs manual tests which should also be covered by Mozmill tests please add the "[mozmill-test-needed]" whiteboard entry to the feature implementation or regression bug.

List of Mozmill Tests:

  • Links to automated mozmill feature test cases

top

Manual Tests Details

Smoke_Tests

  • links to litmus smoke tests or description

top

Regression_Tests

  • links to litmus BFT and/or regression tests description

top

Functional_Tests

  • links to litmus FFT and/or complete functional tests description

top

Community Test Events Details

Testdays

  • No testday has been scheduled yet.

Bugdays

  • No bugday has been scheduled yet.

Meetups

  • No meetup has been scheduled yet. For testing purposes testdays and bugdays seem to be of better value.
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Firefox_4/Test_Plan/WebM_Support&oldid=231763"