QA/Fennec/Milestones/5.0a2: Difference between revisions

From MozillaWiki
< QA‎ | Fennec‎ | Milestones
Jump to navigation Jump to search
(Created page with "= Fennec 5.0 Aurora Test Plan = == Overview == The goal of this Overall Test Plan is to lay out our testing strategy, communicate it to others, and track the status. All of ou...")
 
 
(13 intermediate revisions by the same user not shown)
Line 3: Line 3:
== Overview ==
== Overview ==


The goal of this Overall Test Plan is to lay out our testing strategy, communicate it to others, and track the status.
The goal of this Overall Test Plan is to lay out our testing strategy, determine who will own certain test areas and when they will do it and, finally, track the status. All of our releases are tested off of the "Fennec 5.0 Aurora Catch-All Test Run" on the [http://litmus.mozilla.org/ Litmus TCM]. For more information about Fennec 5.0, take a look at the mobile team's [[Firefox/Flight_Tracking#Firefox_5|Feature Planning page]].
 
All of our releases are tested off of the "Fennec 6.0 Aurora Catch-All Test Run" on the [http://litmus.mozilla.org/ Litmus TCM].
 
For more information about Fennec 5.0, take a look at the mobile team's [[Firefox/Flight_Tracking#Firefox_5|Feature Planning page]]


== Project contacts ==  
== Project contacts ==  
PM: Stuart Parmenter (stuart) <BR>
PM: Thomas Arend (tarend) <BR>
Dev Lead: Stuart Parmenter (stuart) <BR>
Build: John Oduinn  (oduinn)<BR>
Build: John Oduinn  (oduinn)<BR>
QA: Aakash Desai (aakashd)<BR>
QA: Aakash Desai (aakashd)<BR>
Line 34: Line 31:
| style="font-weight: bold; background: #DDD;" | Feature  
| style="font-weight: bold; background: #DDD;" | Feature  
| style="font-weight: bold; background: #DDD;" | Status  
| style="font-weight: bold; background: #DDD;" | Status  
| style="font-weight: bold; background: #DDD;" | ETA
| style="font-weight: bold; background: #DDD;" | Owner
| style="font-weight: bold; background: #DDD;" | Owner
|-
|-
Line 40: Line 36:
| landed  
| landed  
SDR: N &#124;&#124; SIR: N  
SDR: N &#124;&#124; SIR: N  
 
QA: Verified
| Fx5
| Thomas
| Thomas
|-
|-
Line 47: Line 42:
| landed  
| landed  
SDR: N &#124;&#124; SIR: <span style="font-weight: bold;">C</span> - 04/27  
SDR: N &#124;&#124; SIR: <span style="font-weight: bold;">C</span> - 04/27  
 
QA: Verified
| Fx5
| Thomas
| Thomas
|-
|-
Line 54: Line 48:
| landed  
| landed  
SDR: N &#124;&#124; SIR: N  
SDR: N &#124;&#124; SIR: N  
 
QA: Verified
| Fx5
| Thomas
| Thomas
|-
|-
Line 61: Line 54:
| landed  
| landed  
SDR: N &#124;&#124; '''SIR: P&nbsp;'''  
SDR: N &#124;&#124; '''SIR: P&nbsp;'''  
 
QA: Verified
| Fx5
| Thomas
| Thomas
|-
|-
Line 68: Line 60:
| landed  
| landed  
SDR: N &#124;&#124; SIR: N  
SDR: N &#124;&#124; SIR: N  
 
QA: Verified
| Fx5
| Thomas
| Thomas
|-
|-
Line 75: Line 66:
| landed  
| landed  
SDR: N &#124;&#124; SIR: N  
SDR: N &#124;&#124; SIR: N  
 
QA: Verified
| Fx5
| Thomas
| Thomas
|-
|-
| [https://bugzilla.mozilla.org/show_bug.cgi?id=648654 Add user-visible pref for Do Not Track]  
| [https://bugzilla.mozilla.org/show_bug.cgi?id=648654 Add user-visible pref for Do Not Track]  
SDR: N &#124;&#124; SIR: N  
SDR: N &#124;&#124; SIR: N  
 
QA: Verified/Tests Created
| landed  
| landed  
| Fx5
| Thomas
| Thomas
|-
|-
Line 89: Line 78:
| landed  
| landed  
SDR: N &#124;&#124; '''SIR: C''' - 04-27  
SDR: N &#124;&#124; '''SIR: C''' - 04-27  
 
QA: Verified
| Fx5
| Thomas
| Thomas
|-
|-
Line 96: Line 84:
| landed  
| landed  
SDR: N &#124;&#124; SIR: N  
SDR: N &#124;&#124; SIR: N  
 
QA: Verified
| Fx5
| Thomas
| Thomas
|}
|}


== Test Areas ==
== Test Areas ==


=== Smoketests ===
=== Smoketests ===
Lite verification tests (or smoketests) will be first run when the builds are ready. Tests are available on litmus under the [https://litmus.mozilla.org/run_tests.cgi?test_run_id=xxx Fennec 5.0 Test Suite]. In any case of failure in any Smoketest, blockers need to be raised ASAP before proceeding to the BFTs.
{|width=100%
 
|-
Please record your detailed results in the "Detailed Test Results" section below.
| style="background-color: #efefef;"|'''QA Task'''
| style="background-color: #efefef;"|'''Person/Group'''
| style="background-color: #efefef;"|'''How Often?'''
|-
| Smoketests
| Waverley
| Weekly
|}
Lite verification tests (or smoketests) will be first run when the builds are ready. Tests are available on litmus under the [https://litmus.mozilla.org/run_tests.cgi?test_run_id=xxx Fennec 5.0 Test Suite]. In any case of failure in any Smoketest, blockers need to be raised ASAP.


=== 5.0 Feature Quality Work ===
Results details can be found in [https://wiki.mozilla.org/QA/Fennec/Waverley#Results Waverley's results wiki].
'''kbrosnan'''
 
=== OS Integration ===
The Operating System Integration testsuite is a set of tests that target how Fennec works with the platform (i.e. Android, Maemo, etc.) and features of the device itself (i.e. Orientation, hard keyboard, soft keyboard, etc.). Tests are available on litmus under the [https://litmus.mozilla.org/run_tests.cgi?test_run_id=xxx Fennec 6.0 Test Suite] under the "OS Integration Tests" testgroup. 
 
Please record your detailed results in the "Detailed Test Results" section below.


=== Fixed Bugs Verifications ===
=== Fixed Bugs Verifications ===
The following link contains a list of bugs that are fixed for 6.0 and need verifications:
{|width=100%
 
|-
* Fixed [https://bugzilla.mozilla.org/buglist.cgi?list_id=258704&field0-0-0=cf_blocking_fennec&resolution=FIXED&query_format=advanced&bug_status=RESOLVED&bug_status=VERIFIED&type0-0-0=substring&value0-0-0=%2B&target_milestone=Firefox%205&target_milestone=mozilla5 Firefox Mobile 5.0a2 bugs] (even verified bugs) should be checked to make sure the fix landed in the Mobile-5.0 Aurora tree.
| style="background-color: #efefef;"|'''QA Task'''
 
| style="background-color: #efefef;"|'''Person/Group'''
=== Automation ===
| style="background-color: #efefef;"|'''How Often?'''
QA is undertaking the task of porting the unit tests from Firefox to Fennec for each platform. This is a lengthy task, but our goal is to have 100% unit tests passing by the 6.0 ship.
|-
| Bug Verification
| nhirata/Andreea
| As-needed
|}


We have created a [https://wiki.mozilla.org/Mobile/Fennec_Automation tracking system] for the unit tests which we update on a regular basis.
The following link contains a list of bugs that are fixed for 6.0 and need verifications: Fixed [https://bugzilla.mozilla.org/buglist.cgi?list_id=258704&field0-0-0=cf_blocking_fennec&resolution=FIXED&query_format=advanced&bug_status=RESOLVED&bug_status=VERIFIED&type0-0-0=substring&value0-0-0=%2B&target_milestone=Firefox%205&target_milestone=mozilla5 Firefox Mobile 5.0a2 bugs] (even verified bugs) should be checked to make sure the fix landed in the Mobile-5.0 Aurora tree.

Latest revision as of 16:32, 17 May 2011

Fennec 5.0 Aurora Test Plan

Overview

The goal of this Overall Test Plan is to lay out our testing strategy, determine who will own certain test areas and when they will do it and, finally, track the status. All of our releases are tested off of the "Fennec 5.0 Aurora Catch-All Test Run" on the Litmus TCM. For more information about Fennec 5.0, take a look at the mobile team's Feature Planning page.

Project contacts

PM: Thomas Arend (tarend)
Dev Lead: Stuart Parmenter (stuart)
Build: John Oduinn (oduinn)
QA: Aakash Desai (aakashd)

Schedule

  • Start Date: 4/11/2011
  • End Date: 5/17/2011

Bugs in Release

Build Location and IDs

  1. Go to the nightly builds folder in our ftp directory
  2. Go to the "latest-mozilla-aurora" folder
  3. Download and install the candidate onto your Android phone/device

Features Tracked

Feature Status Owner
Optimized layout and interaction with some mobile web pages (improved position:fixed support) landed

SDR: N || SIR: N QA: Verified

Thomas
Add images downloaded in Firefox to the Android Gallery immediately (before: images were visible only after reboot landed

SDR: N || SIR: C - 04/27 QA: Verified

Thomas
Add a "More" button to the Android Menu when needed) landed

SDR: N || SIR: N QA: Verified

Thomas
Fix issue with color conversion for better display of images and video (Fix broken NEON Y'CbCr to RGB conversion) landed

SDR: N || SIR: P  QA: Verified

Thomas
Resolves recent breakage that would show an "undefined" error when trying to load web pages while outside of wifi or network coverave (Add remoteXUL strings for Firefox Mobile) landed

SDR: N || SIR: N QA: Verified

Thomas
Slight improvement of page load speed, especially on 3G network (e10s HTTP: Implement nsIProgressEventSink:part2) landed

SDR: N || SIR: N QA: Verified

Thomas
Add user-visible pref for Do Not Track

SDR: N || SIR: N QA: Verified/Tests Created

landed Thomas
Finish new theme styles for dialogs/prompts (prompts and confirms look like android dialogs) landed

SDR: N || SIR: C - 04-27 QA: Verified

Thomas
Smooth panning (resolves an issue where consecutive pans were a bit jumpy before) landed

SDR: N || SIR: N QA: Verified

Thomas

Test Areas

Smoketests

QA Task Person/Group How Often?
Smoketests Waverley Weekly

Lite verification tests (or smoketests) will be first run when the builds are ready. Tests are available on litmus under the Fennec 5.0 Test Suite. In any case of failure in any Smoketest, blockers need to be raised ASAP.

Results details can be found in Waverley's results wiki.

Fixed Bugs Verifications

QA Task Person/Group How Often?
Bug Verification nhirata/Andreea As-needed

The following link contains a list of bugs that are fixed for 6.0 and need verifications: Fixed Firefox Mobile 5.0a2 bugs (even verified bugs) should be checked to make sure the fix landed in the Mobile-5.0 Aurora tree.

Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Fennec/Milestones/5.0a2&oldid=309504"