QA/Firefox3/Strategy: Difference between revisions

Jump to navigation Jump to search

QA/Firefox3/Strategy (view source)

Revision as of 19:13, 22 August 2007

2,655 bytes removed ,  22 August 2007
m
→‎Profile Suite Test Suite
Line 5: Line 5:
== Overview ==
== Overview ==


This is a tracking document that outlines the test strategy that Mozilla QA will follow regarding a thorough test pass of Firefox 3.  It will follow the general plan on steps and process that QA will take to accomplish a "sign off" to the release strategy.  Mozilla QA strives to uphold quality software, and a successful pass corresponds to no blocking issues, Green test pass on test cases in both execution and automation, and a thorough bug verification window of all Pri 1 and top Pri 2 tier bugs.
This document is a tracking strategy that outlines the test strategy that Mozilla QA will follow regarding a thorough test pass of Firefox.  


== Areas Covered ==
== Areas Covered ==
Line 63: Line 63:


;Planning
;Planning
We will be gathering the most relevant and widely used web applications today.  Web compatibility tests will be broken down by new feature tests for Firefox 3 specifically, as well as regression across the top extensions, plugins, and themes posted on AMO.  Tests will be expected to run during the later stages of alpha all the way up to RC2.
We will be gathering the most relevant and widely used web applications today.  Test breakdown of areas will  


;Test Suite
;Test Suite
Line 73: Line 73:


;Results
;Results
Test results are tracked in execution of FFTs, BFTs, and smoketests in litmus.
Test cases will be tracked in FFTs, BFTs, and smoketests in litmus.


=== User Performance Test Suite ===
=== User Performance Test Suite ===


;Summary
The User Performance Test Suite is not to be confused with Browser performance tests that are run with automation.   
The User Performance Test Suite is not to be confused with Browser performance tests that are run with automation.  Instead, we will concentrate on day to day usage of firefox and any performance issues that we see.  These would includes areas like downloading, idling, and other firefox daily usage cases.
 
;Planning
Different scenarios will be conducted and posted on litmus or a public wiki.  Since this test suite involves a different mix of computing environments, we will ask for community assistance to help.  The timing can exist around the Beta schedule, for about 1 or 2 cycles.   


;Test Suite
* performance against other browsers?  Do we have benchmark comparisons?
* performance against other browsers?  Do we have benchmark comparisons?
* Hard drives, throttling CPU’s
* Hard drives, throttling CPU’s
* Internet connection (wireless, dial up, LAN)
* Internet connection (wireless, dial up, LAN)
* Tab switching tests (martijn’s reftests)
* Tab switching tests (martijn’s reftests)
;Results
Test cases will exist in litmus and the public wiki.  Any issues found will be tracked in bugzilla.


=== Profile Suite Test Suite ===
=== Profile Suite Test Suite ===
Line 97: Line 89:


;Planning
;Planning
QA is developing virtualization test suites that contains images of various profiles across XP, vista, and Linux virtual machines.  This will help to assure a clean environment for testing, as well as cross functional platform operation. 
The [http://wiki.mozilla.org/MozillaQualityAssurance:Environments Test Environments TestPlan] will be divided up among the team and ran during the Beta stages of the overall schedule.


;Test Suite
;Test Suite
* [https://intranet.mozilla.org/QA:Virtualization VM environment]
* VM environment (see al’s link to different VM scenarios
* Assorted Profile settings (see marcia’s link to all the different profile combos)  
* Assorted Profile settings (see marcia’s link to all the different profile combos)  
* Platforms specific (mac, vista, linux, xp)
* Platforms specific (mac, vista, linux, xp)


;Results
;Results
Issues found will be tracked in bugzilla, and test results will be recorded in Litmus.  Virtual Machines are tracked on the file server and available for download with VMWare licensing.
Issues found will be tracked in bugzilla, and test results will be recorded in Litmus.


=== Security Test Suite ===
=== Security Test Suite ===
Line 119: Line 109:


=== Accessibility Test Suite ===
=== Accessibility Test Suite ===
;Summary
Accessibility Tests will be developed and ran against the new features coming within Firefox 3.  These cases will ensure we are following proper A11y guidelines. 
;Planning
Testing of Accessibility features will be within the Alpha to Beta stage of the schedule. 
;Test Suite
* Accessiblity areas against new features
* Accessiblity areas against new features
* Regression of areas
* Regression of areas  
 
;Results
Test cases will exist in litmus and the public wiki.  Any issues found will be tracked in bugzilla.


=== Bug Verifications ===
=== Bug Verifications ===
Line 158: Line 137:




== Weekly Smoketests ==
Weekly Smoketests
 
- Tiger team (smoketesting on different platform) –Tracy to watch the results
Each week, the Tiger team will be running a series of regression tests against the nightly builds on trunk.  The purpose of these tests are to plant cross-platform overlapping tests, to ensure stability and reliability of the builds post nightly checkins.  See the detailed description and schedule [http://wiki.mozilla.org/MozillaQualityAssurance:Home_Page:Firefox_3.0_TestPlan:Fx3_QA_Daily_Smoketest#Tiger_Team_Testing_Squad Here].
- Focused testing (across platforms)
 
Tests will cover:
* Smoketests across all supported Firefox platforms
* Focused feature areas across supported Firefox platforms
* Feature review of nightly checkins found on tinderbox
Tchung
Confirmed users
6,361

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/66146"

Navigation menu