QA/Fennec1.0/TestPlan

From MozillaWiki
< QA
Revision as of 16:26, 23 March 2009 by Kcome (talk | contribs) (→‎Platforms)
Jump to navigation Jump to search

« Mobile

Overview

Fennec 1.0 Test Strategy

The goal of this Overall Test Plan is to lay out our testing strategy, communicate it to others, and track the status.

Since Fennec is not installed or used in the same way Firefox is, the test approach needs to be modified. We will automate as much as possible. The automation we will use will be a subset (as all features are not in Fennec) of the Firefox automation with new tests developed to test specific issues of firefox. Key pieces of automation will be functionality, performance, and stability.

There will be a lot of unique tests which are device specific and Fennec specific that will be too expensive to automate. With these we will add them to litmus and run them by hand.

Items to Update

Here is a list of items that will be updated once we get more information: 

* Accessibility Litmus test cases - Beta1
* User performance Litmus test cases - Beta1
* L10n plan and list of locales - Beta1
* Security Tools and test cases - Beta1
* Flush out schedule for Beta1 process - Beta1

* Stress Testing - Post Beta1
* QA/Support Team Issues - Post Beta1
* List of Devices/Platforms - Post Beta1
* Strategy to scale testing efforts to multiple devices - Post Beta1
* Performance regression plan (maybe just user performance, but probably more)
* How to scale testing to multiple devices - Post Beta1
* How to test with wifi vs wireless carrier - Post Beta1
* Windows Mobile Performance Regression testing - need to understand what 
  we'll be testing, how regularly we'll be testing it, and what metrics to test. - Post Beta1
* Web compatibility Litmus test cases - update with web compatibility tool when available.
* Install/Update (definition of repositories) - Update with betachannel and windows mobile

Schedule

  • Alpha 1: October 17
  • Alpha 2: December 23
  • Beta 1: Feb 27 * NOTE: this is a Beta1 for Maemo, not Windows Mobile or Symbian
    • Code Complete - Feb 20
    • Handoff to Test (B1 RC1) - Feb 23
    • 48 hour test pass, Test Signs off - Feb 26
    • Upload and blog - Feb 27
  • Beta 2 - 6 weeks later
  • Beta 3 | RC1 - 6 weeks later
  • RC1 | 2 - 4-6 weeks later

Test Results

Alpha2:

Beta1:

  • Litmus results
  • Mochitest results
  • Chrome results
  • Reftest results
  • Crashtest results

Test Areas

General Test Areas

Feature Area and Spec Test Plan Location Testing Notes QA Contact Dev Contact Device Needed Links
Zoom gavin M
Add-ons manager mfinkle N
Awesome bar and search here jmaher enndeakin N
Multiple document Already some Firefox tests for this enndeakin N
Download Manager here jmaher mfinkle N
Running Firefox's mochitests here Status jmaher jmaher N
Updates N
Bookmarks N
Preferences N
Certificate Management johnath N
Password manager N
Popup Blocker N
Find neil Y
Browsing History gavin N

Mobile Specific Features

Feature Area and Spec Test Plan Location Testing Notes QA Contact Dev Contact Device Needed Links
Tel: uri blassey N
Navigation mfinkle, dougt M
Geolocation harthur dougt N
Panning gavin Y
Kinetic Scrolling Manual testing blassey Y
Weave dolske N

Platform Specific Features

Feature Area and Spec Test Plan Location Testing Notes QA Contact Dev Contact Device Needed Links
Windows Mobile here build notes jmaher wolfe Y
Performance Low disk/mem, large pages, startup time Y
Breakpad Y
Soft keyboard Y
Interop email, print, cut/paste, phone Dictionary, T9 christian Y
Maemo Keyboard open, close, access, hardkeys, user defined db48x, neil Y

Environments

The main environment that we will be focusing on for the 1.0 release will be the Nokia maemo platform (N8xx devices). In addition to the Maemo platform, we are working on Windows Mobile and Symbian support. These will be under concurrent development but not at the same feature complete and quality of the Maemo platform.

Platforms

Device OS Chip Supported
Nokia N800 Maemo ARM Yes
Nokia N800 Maemo ARM Yes
HTC Touch Pro WinMobile Pro6.1 ARM Alpha1
Nokia N95 Symbian ARM Alpha1


TODO: verify this list and update it?

Source Code and Nightly Build Locations

Fennec 1.0 is currently built from trunk (mozilla-central) as well as the front end mobile-browser repositories. To minimize branching and maintenance, we will target the 1.9.1 branch (probably with some patches). All other platforms will build from the same branch.

Nightly builds for Maemo are located here (you need both fennec*.deb and xulrunner*.deb where the timestamps match): ftp://ftp.mozilla.org/pub/firefox/tinderbox-builds/mobile-browser-linux-arm/

Currently we do not have nightly builds for Windows Mobile or Symbian.

Banking and Financial Sites

Taken directly from the Firefox test plan.

For these types of sites, we’re asking community members and MoCo QA to state which banks and financial institutions they're willing to test. Please add your name to the list.



Review Feedback

In this section, we will roll up all common feedback pulled from external sources that appears common and should be tracked by QA. Feedback sources can include:

QA / Support Team Issues

Currently, Fennec doesn't have a support channel other than IRC. Stuart is looking into SUMO and a definition for how we will provide support.

TODO: update this post Beta1 with the direction we are taking and how QA will be involved.


Automation

QA is undertaking the task of porting the unit tests from Firefox to Fennec for each platform. This is a lengthy task, but our goal is to have 100% unit tests passing by the 1.0 ship.

We have created a tracking system for the unit tests which we update on a regular basis.

Additional References

This section is left to add any additional references relevant to the Mobile project

Useful Information:

Building:

Testing:

Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Fennec1.0/TestPlan&oldid=135410"