QA/Fennec/Milestones/5.0b1

Fennec 5.0 Beta Test Plan

Overview

The goal of this Overall Test Plan is to lay out our testing strategy, determine who will own certain test areas and when they will do it and, finally, track the status. All of our releases are tested off of the "Fennec 5.0 Beta Catch-All Test Run" on the Litmus TCM. For more information about Fennec 5.0, take a look at the mobile team's Feature Planning page.

Project contacts

PM: Thomas Arend (tarend)
Dev Lead: Stuart Parmenter (stuart)
Build: John Oduinn (oduinn)
QA: Aakash Desai (aakashd)

Schedule

• Start Date: 5/17/2011
• End Date: 6/21/2011

Bugs in Release

• Bugs FIXED

Build Location and IDs

1. Go to the nightly builds folder in our ftp directory
2. Go to the "latest-mozilla-beta" folder
3. Download and install the candidate onto your Android phone/device

Features Tracked

Feature	Status	Owner
Optimized layout and interaction with some mobile web pages (improved position:fixed support)	landed SDR: N || SIR: N QA: Verified	Thomas
Add images downloaded in Firefox to the Android Gallery immediately (before: images were visible only after reboot	landed SDR: N || SIR: C - 04/27 QA: Verified	Thomas
Add a "More" button to the Android Menu when needed)	landed SDR: N || SIR: N QA: Verified	Thomas
Fix issue with color conversion for better display of images and video (Fix broken NEON Y'CbCr to RGB conversion)	landed SDR: N || SIR: P  QA: Verified	Thomas
Resolves recent breakage that would show an "undefined" error when trying to load web pages while outside of wifi or network coverave (Add remoteXUL strings for Firefox Mobile)	landed SDR: N || SIR: N QA: Verified	Thomas
Slight improvement of page load speed, especially on 3G network (e10s HTTP: Implement nsIProgressEventSink:part2)	landed SDR: N || SIR: N QA: Verified	Thomas
Add user-visible pref for Do Not Track SDR: N || SIR: N QA: Verified/Tests Created	landed	Thomas
Finish new theme styles for dialogs/prompts (prompts and confirms look like android dialogs)	landed SDR: N || SIR: C - 04-27 QA: Verified	Thomas
Smooth panning (resolves an issue where consecutive pans were a bit jumpy before)	landed SDR: N || SIR: N QA: Verified	Thomas

Test Areas

Smoketests

QA Task	Person/Group	How Often?
Smoketests	Waverley	Once

Lite verification tests (or smoketests) will be first run when the builds are ready. Tests are available on litmus under the Fennec 5.0 Test Suite. In any case of failure in any Smoketest, blockers need to be raised ASAP before proceeding to the BFTs.

Results details can be found in Waverley's results wiki.

BFTs for 5.0 Features

QA Task	Person/Group	How Often?
BFTs	Waverley	Once

BFTs are a more comprehensive set of litmus tests that cover functional testing of features that have landed on Fennec 2.0. Tests are available on litmus under the Fennec 2.0 Test Suite. Results details can be found in Waverley's results wiki.

Crasher Triage

QA Task	Person/Group	How Often?
Crasher Triage	mw22	Daily

Socorro is Mozilla's primary web interface for triaging crash reports sent in by our user base. Due to the relatively large amount of changes coming in on a nightly basis, there is a need to check for crash reports coming in for every build. The owner of this task will triage through top crashers on a daily basis, file bugs if necessary and find steps to reproduce for those crashers if possible.

Device Compatibility

QA Task	Person/Group	How Often?
Device Compatibility	AaronMT	Once

Website Compatibility

QA Task	Person/Group	How Often?
Website Compatibility	tchung	Once

Testday

QA Task	Person/Group	How Often?
Testday	kbrosnan	Once

To continue determining better quality metrics, we'll want to work with our community to find new bugs and performance/stability metrics through the use of a Testday once per cycle per channel (i.e. nightly and beta). The QA owner for this task will send announcements, create a testplan, moderate the channel and post results for the Testday.