QA/Fennec1.0/TestPlan

From MozillaWiki
< QA
Jump to: navigation, search

« Mobile

Overview

Fennec 1.0 Test Strategy

The goal of this Overall Test Plan is to lay out our testing strategy, communicate it to others, and track the status.

Since Fennec is not installed or used in the same way Firefox is, the test approach needs to be modified. We will automate as much as possible. The automation we will use will be a subset (as all features are not in Fennec) of the Firefox automation with new tests developed to test specific issues of firefox. Key pieces of automation will be functionality, performance, and stability.

There will be a lot of unique tests which are device specific and Fennec specific that will be too expensive to automate. With these we will add them to litmus and run them by hand.

Items to Update

Here is a list of items that will be updated once we get more information:

Maemo 1.0 Release (RC1):

* L10n - L10n Fennec Litmus test cases
* L10n - Spot check wiki and list of locales
* Install/Update (definition of repositories) - Update with betachannel and windows mobile
* Web compatibility Litmus test cases - update with web compatibility tool when available.
* QA/Support Team Issues
* How to scale testing to multiple devices
* How to test with wifi vs wireless carrier


Post Maemo 1.0:

* Accessibility Litmus test cases
* Security Tools and test cases
* Stress Testing
* Performance regression plan (maybe just user performance, but probably more)

Schedule

Test Areas

General Test Areas

Feature Area and Spec Test Plan Location Testing Notes QA Contact Dev Contact Device Needed Links
Zoom gavin M
Add-ons manager mfinkle N
Awesome bar and search here jmaher enndeakin N
Multiple document Already some Firefox tests for this enndeakin N
Download Manager here jmaher mfinkle N
Running Firefox's mochitests here Status jmaher jmaher N
Updates N
Bookmarks N
Preferences N
Certificate Management johnath N
Password manager N
Popup Blocker N
Find neil Y
Browsing History gavin N

Mobile Specific Features

Feature Area and Spec Test Plan Location Testing Notes QA Contact Dev Contact Device Needed Links
Tel: uri blassey N
Navigation mfinkle, dougt M
Geolocation harthur dougt N
Panning gavin Y
Kinetic Scrolling Manual testing blassey Y
Weave dolske N

Platform Specific Features

Feature Area and Spec Test Plan Location Testing Notes QA Contact Dev Contact Device Needed Links
Windows Mobile here build notes jmaher wolfe Y
Performance Low disk/mem, large pages, startup time Y
Breakpad Y
Soft keyboard Y
Interop email, print, cut/paste, phone Dictionary, T9 christian Y
Maemo Keyboard open, close, access, hardkeys, user defined db48x, neil Y

Environments

The main environment that we will be focusing on for the 1.0 release will be the Nokia maemo platform (N8xx devices). In addition to the Maemo platform, we are working on Windows Mobile and Symbian support. These will be under concurrent development but not at the same feature complete and quality of the Maemo platform.

Platforms

Device OS Manufacturer Keyboard GPS Camera Rotate Resolution
Nokia N800 Maemo Nokia No Yes Yes No 480x800
Nokia N810 Maemo Nokia Slide Out Yes Yes No 480x800
HTC Touch Pro WinMobile Pro 6.1 HTC Slide Out Yes Yes Yes 480x640
ATT Tilt WinMobile Pro 6.1 HTC Slide Out Yes Yes Yes 240x320
Omnia WinMobile Pro 6.1 Samsung No Yes Yes Yes 400x240
Omnia 2 WinMobile Pro 6.1 Samsung No Yes Yes Yes 480x800
Treo Pro WinMobile Pro 6.1 Palm Yes Yes Yes No 320x320

Source Code and Nightly Build Locations

Fennec 1.0 is currently built from trunk (mozilla-central) as well as the front end mobile-browser repositories. To minimize branching and maintenance, we will target the 1.9.2 branch (splitting from mozilla-central July 2009). All other platforms will build from the same branch.

Nightly builds for Maemo (you need both fennec*.deb and xulrunner*.deb where the timestamps match) and Windows Mobile (only the .cab file is needed) are located here: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mobile-1.9.2/

Review Feedback

In this section, we will roll up all common feedback pulled from external sources that appears common and should be tracked by QA. Feedback sources can include:

Automation

QA is undertaking the task of porting the unit tests from Firefox to Fennec for each platform. This is a lengthy task, but our goal is to have 100% unit tests passing by the 1.0 ship.

We have created a tracking system for the unit tests which we update on a regular basis.

Release Test Guidelines

  • QA Requirements:
    • E-mail to Drivers pre/post-Release Cycle and pre/post-Respin
      • Includes:
        • Build location link
        • Blocker Bugs added to the newest release build
        • Release cycle schedule
        • Dev Contact available on-hand
  • List of Disabled Features for the Release
    • Session Store
    • Private Browsing
    • Safebrowsing
    • RSS
    • Certificates Manager
    • Find In Page
Release Smoketest BFT Blocker Verifications L10n Website Updates
Early-Stage Betas YES NO YES 1 RTL, 1 non-Latin, 1 Multi-L10n Switch NO YES
Late-Stage Betas (3+) YES YES YES 1 RTL, 1 non-Latin, Full Multi-L10n Switch NO YES
Major Releases YES YES YES 2 RTL, 2 non-Latin, Full Multi-L10n Switch YES YES

Additional References

This section is left to add any additional references relevant to the Mobile project

Useful Information:

  • Original list of [requirements]
  • IRC Channels: #mobile, #wince, #symbian

Building:

Testing: