QA: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
mNo edit summary
No edit summary
Line 1: Line 1:
== Who We Are ==


[http://weblogs.mozillazine.org/qa/ QA Blog]
* Alice (alice)
* Bob (bc) [http://bclary.com/blog/ Blog]
* Chris (coop).  [http://coop.deadsquid.com/ Blog] [http://people.mozilla.com/~timr/QATeam/Pictures/Pic-coop.jpg Picture]
* Dave (davel) [http://www.daveliebreich.com/blog/ Blog]
* Jay (jay). [http://people.mozilla.com/~timr/QATeam/Pictures/Pic-jay.jpg Picture]
* Juan (juanb)
* Marcia (marcia) [http://weblogs.mozillazine.org/marcia/ Blog] [http://people.mozilla.com/~timr/QATeam/Pictures/Pic-marcia.jpg Picture]
* Martijn (mw22) [http://flickr.com/photos/nitot/73723011/in/set-1603279/ Picture]
* Rob (robcee) [http://antennasoft.net/robcee/ Blog]
* Tim (timr). [http://people.mozilla.com/~timr/QATeam/Pictures/Pic-timr.jpg Picture]
* Tracy (tracy)
== Mozilla QA Community ==
A huge part of what we do is work with the Mozilla community to ensure we ship quality software.  To learn more about how to help, please check out the [[Mozilla QA Community]] wiki page.
== Projects ==
=== Firefox/Thunderbird 1.5.0.5 ===
* [[Firefox#Firefox_1.5.0.5_.28Deer_Park.29_Plan|Main Firefox 1.5.0.5 page]]
* [[Firefox:1.5.0.5:Community|1.5.0.5 Community Page/Test Plan]]
* [[Firefox:1.5.0.6:Community|1.5.0.6 Community Page/Test Plan]]
=== Bon Echo (Firefox 2) ===
* [[Firefox#Firefox_2.0_.28Bon_Echo.29_Plan|Main Bon Echo page]]
* [[Firefox:2.0_QA_Activities|Testing Activities]]
=== Firefox 3 ===
* [[Firefox#Firefox_3.0_Plan|Main Firefox 3 site]]
* Test Plan - TBD
* [[MozillaQualityAssurance:Home_Page:Planning Meeting Notes|Planning Meeting Notes]]
==What We Use==
We typically use nightly optimized (non-debug) builds for daily usage. However, we also use the release builds (of course!), as well as older builds when trying to narrow down regression windows.
* For nightly builds, check out any of the [http://www.mozilla.org/mirrors.html mirrors], then drill down to the '''<product_name>/nightly/''' directory. While you can go to '''<product_name>/latest-*''' directories, the problem there is that you don't necessarily known '''''when''''' those builds were made. It's best to access the '''specific build-date directory''' (e.g., 2005-03-17-08-trunk), to know what you're grabbing.
* For older builds not listed in the mirror pages, check out the archives at http://archive.mozilla.org/pub/
* For release builds, simply go to any of the mirrors and drill down to '''<product_name>/releases/''' and select the appropriate directories for version, platform and locale.
===Bugzilla===
We depend on [https://bugzilla.mozilla.org Bugzilla] for filing and tracking bugs and features. We frequently use the [https://bugzilla.mozilla.org/query.cgi query tools], both the "Advanced Search" and "Find a Specific Bug" queries. With the bug count reaching 300,000, there are a couple ways to see what's been frequently reported and duplicated:
* [http://bugzilla.mozilla.org/duplicates.html Most frequently reported bugs query]
* [http://bugzilla.mozilla.org/duplicates.cgi?sortby=bug_no Most recently duplicated bug query]
People wanting to help triage bugs can refer to the [[MozillaQualityAssurance:Triage| Mozilla Triage Guide]]
===Litmus ===
We are currently using Litmus [http://litmus.mozilla.org/ Litmus] as a replacement for Testrunner (our previous test management tool). Litmus will build upon the testcase management aspects of Testrunner, adding the searchability, reporting, and trending tools that are critical to our QA work. More information about Litmus development is available in the [[Litmus|Litmus wiki]].
===Talkback===
Talkback info has moved to a new location: [[Mozilla_QA_Community:Talkback|All Things Talkback]].
===Development tools===
We also use several development tools for tracking changes, especially useful for narrowing down regression windows!
* Tinderbox (http://tinderbox.mozilla.org/showbuilds.cgi) to visually display our continuous build system. Great to see who checked in what, the state of the build, as well as quick links to automated performance tests.
* LXR (http://lxr.mozilla.org) to examine the source code.
* Bonsai (http://bonsai.mozilla.org) to narrow down when changes were made.
===Automation===
In an effort to the improve the testing potential of our team, we are constantly investigating ways to leverage automated testing tools to offload some of the regression testing burden from our core testers.
* [http://www.redstonesoftware.com/ Eggplant] GUI automation tool
* A [[QA Automation Candidate Software|listing of the various candidate software we have considered]] and the status of each.
== Test Suites and Test Tools ==
* Firefox User-based tests
** Sign into the [http://litmus.mozilla.org Litmus test tool], specify ''Firefox'' in the System Information and see the top level test groups with the following names:
*** Smoke Test
*** Basic Functional Test (BFT)
*** Full Functional Test (FFT)
*** L10n
* Thunderbird User-based tests
** Sign into the [http://litmus.mozilla.org Litmus test tool], specify ''Thunderbird'' in the System Information and see the top level test groups with the following names:
*** Smoke Test
*** Basic Functional Test (BFT)
*** Full Functional Test (FFT)
*** L10n
* JS Test Suite
* [[MozillaQualityAssurance:MetaDiff|MetaDiff - L10n checker]]
* [[MozillaQualityAssurance:Update Checker|Update Checker]]
* [[MozillaQualityAssurance:Download Checker | Download Checker]]
* [[MozillaQualityAssurance:l10n Search Verifier|l10n Search Verifier]]
== Sites to scan for Community and User Feedback ==
* [http://www.mozillazine.org/ MozillaZine]
* [http://slashdot.org/ Slashdot] - Source for technology related news with a heavy slant towards Linux and Open Source issues.
* [http://www.digg.com/ Digg] - rumors, issues, etc.
* [http://www.computergripes.com/firefox.html Firefox Gripes by Michael Horowitz] - identified by Marcia
* Bug site run my PPK - Maretijn identified these.
** [http://www.quirksmode.org/bugreports/index.html Site entirely dedicated to finding, mending and publishing CSS and JavaScript browser bugs]
** [http://www.quirksmode.org/blog/mt-search.cgi?IncludeBlogs=2&search=mozilla&CategoryID=2  bug identified with Mozilla] The closest category to Firefox.
* News Groups:
** mozilla.dev.quality
** mozilla.dev.general
== Websites for verification ==
=== Java Acceptance ===
[http://www.basemetals.com/ http://www.basemetals.com/]
== Test Cases that need to be modified in Litmus (Tracking Page)==
*Firefox 2.0 FFTs and BFTs
**Find toolbar - "/" functionality has changed (but maybe only for beta??)
**Removal of match case functionality
***https://bugzilla.mozilla.org/show_bug.cgi?id=340747
***https://bugzilla.mozilla.org/show_bug.cgi?id=340849
== Test Cases Added to Litmus (Tracking Page)==
Please use this page to list new test cases that you have added to Litmus.
*Firefox 1.5. BFTs
*Firefox 1.5 FFTs
**[http://litmus.mozilla.org/show_test.cgi?id=2268 2268] Intel Mac Sound Test/marcia/061506
**[http://litmus.mozilla.org/show_test.cgi?id=2269 2269] Launch In Safe Mode/marcia/061506
**[http://litmus.mozilla.org/show_test.cgi?id=2299] AJAX testing/marcia
**[http://litmus.mozilla.org/show_test.cgi?id=2254 2254] Switching tabs using the keyboard (Page Up/Down)/ispiked
**[http://litmus.mozilla.org/show_test.cgi?id=2252 2252] Tab focusing behavior/ispiked
**[http://litmus.mozilla.org/show_test.cgi?id=2249 2249] Switching tabs with the keyboard (using Ctrl)/ispiked
**[http://litmus.mozilla.org/show_test.cgi?id=2250 2250] Switching tabs using the keyboard (using Tab button)/ispiked
**[http://litmus.mozilla.org/show_test.cgi?id=2251 2251] Moving tabs with the keyboard/ispiked
**[http://litmus.mozilla.org/show_test.cgi?id=2253 2253] Closing tabs with the keyboard/ispiked
*Firefox 2.0 BFTs
*Firefox 2.0 FFTs
**[http://litmus.mozilla.org/show_test.cgi?id=2267 2267] Intel Mac Sound Test/marcia/061506
**[http://litmus.mozilla.org/show_test.cgi?id=2270 2270] Launch in Safe Mode/marcia/061506
**[http://litmus.mozilla.org/show_test.cgi?id=2271 2271]  Warn if opening too many tabs preference (enabled)/ispiked/061505
**[http://litmus.mozilla.org/show_test.cgi?id=2272 2272] Warn if opening too many tabs preference (enabled)/ispiked/061506
**[http://litmus.mozilla.org/show_test.cgi?id=2254 2254] Switching tabs using the keyboard (Page Up/Down)/ispiked
**[http://litmus.mozilla.org/show_test.cgi?id=2247 2247] Tab focusing behavior/ispiked
**[http://litmus.mozilla.org/show_test.cgi?id=2244 2244] Switching tabs with the keyboard (using Ctrl)/ispiked
**[http://litmus.mozilla.org/show_test.cgi?id=2245 2245] Switching tabs using the keyboard (using Tab button)/ispiked
**[http://litmus.mozilla.org/show_test.cgi?id=2246 2246] Moving tabs with the keyboard/ispiked
**[http://litmus.mozilla.org/show_test.cgi?id=2248 2248] Closing tabs with the keyboard/ispiked
**[http://litmus.mozilla.org/show_test.cgi?id=2257 2257] Quick Find link and word/ispiked
**[http://litmus.mozilla.org/show_test.cgi?id=2256 2256] Open and close the Quick Find bar/ispiked
**[http://litmus.mozilla.org/show_test.cgi?id=2258 2258]  Quick Find forward and backwards/ispiked
**[http://litmus.mozilla.org/show_test.cgi?id=2260 2260] Add a basic microsummary/ispiked
**[http://litmus.mozilla.org/show_test.cgi?id=2261 2261] Add a microsummary generator/ispiked
**[http://litmus.mozilla.org/show_test.cgi?id=2262 2262] Install a microsummary generator/ispiked
**[http://litmus.mozilla.org/show_test.cgi?id=2263 2263] Verify basic microsummary updating/ispiked
**[http://litmus.mozilla.org/show_test.cgi?id=2264 2264] Verify microsummary generator updating/ispiked
**[http://litmus.mozilla.org/show_test.cgi?id=2265 2265] Verify microsummary generator updating when it uses another page/ispiked
**[http://litmus.mozilla.org/show_test.cgi?id=2274 2274] Verify extension and theme context menu/ispiked
**[http://litmus.mozilla.org/show_test.cgi?id=2300] AJAX testing/marcia
*Thunderbird 1.5. BFTs
*Thunderbird 1.5 FFTs
**[http://litmus.mozilla.org/show_test.cgi?id=2273 2273] Test new mail sound/ispiked
*Thunderbird 2.0 BFTs
*Thunderbird 2.0 FFTs
**[http://litmus.mozilla.org/show_test.cgi?id=2259 2259]  Test new mail sound/ispiked
==Release Data - Firefox==
A collection of data from the various releases of Firefox. (currently rather empty, just set up and will fill in holes an time permits
-tracy)
===User Agent strings===
Windows:
* Fx 1.5rc1 - Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8) Gecko/20051025 Firefox/1.5
* Fx 1.5beta2 - Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8b5) Gecko/20051006 Firefox/1.4.1
* Fx 1.5beta1 - Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8b4) Gecko/20050908 Firefox/1.4
* DP alpha2 - Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8b3) Gecko/20050712 Firefox/1.0+
* DP alpha1 - Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8b2) Gecko/20050531 Firefox/1.0+
Macintosh:
* Fx 1.5.0.1 - Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1
* Fx 1.5 - Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8) Gecko/20051111 Firefox/1.5
* Fx 1.5rc1 - Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8) Gecko/20051025 Firefox/1.5
* Fx 1.5beta2 - Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8b5) Gecko/20051006 Firefox/1.4.1
* Fx 1.5beta1 - Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8b4) Gecko/20050908 Firefox/1.4
* DP alpha2 - Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8b3) Gecko/20050712 Firefox/1.0+
* DP alpha1 - Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8b2) Gecko/20050531 Firefox/1.0+
Linux:
* Fx 1.5 rc1 - Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8) Gecko/20051025 Firefox/1.5
* Fx 1.5 Beta2 - Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b5) Gecko/20051006 Firefox/1.4.1
* Fx 1.5 Beta1 - Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b4) Gecko/20050908 Firefox/1.4
* DP alpha2 - Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b3) Gecko/20050712 Firefox/1.0+
* DP alpha1 - Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b2) Gecko/20050531 Firefox/1.0+
===Talkback Build IDs===
==== Talkback Firefox Build IDS for 1.5.0.x en-US releases ====
{| border="1" cellpadding="2"
! style="background:#efefef" | '''en-US Release'''
! style="background:#efefef" | '''Windows'''
! style="background:#efefef" | '''Mac PPC'''
! style="background:#efefef" | '''Mac UB'''
! style="background:#efefef" | '''Linux'''
|-
| Firefox 1.5
| 2005111116
| 2005111116
| No UB Release
| 2005111116
|-
| Firefox 1.5.0.1
| 2006011112
| 2006011112
| No UB Release
| 2006012415
|-
| Firefox 1.5.0.2
| 2006030804
| 2006030803
| 2006032818
| 2006030812
|-
| Firefox 1.5.0.3
| 2006042618
| 2006042618
| 2006042618
| 2006042618
|-
| Firefox 1.5.0.4
| 2006050817
| No PPC release
| 2006050817
| 2006050817
|-
| Firefox 1.5.0.5
| 2006071912
| No PPC release
| 2006072310
| 2006071912
|-
| Firefox 1.5.0.6
| 2006072814
| No PPC release
| 2006072814
| 2006072814
|-
| Firefox 1.5.0.7
| 2006090918
| No PPC release
| 2006090921
| 2006090918
|-
| Firefox 1.5.0.8
| 2006102516
| No PPC release
| 2006102516
| 2006102516
|-
| Firefox 1.5.0.9
|
| No PPC release
|
|
|}
==== Talkback Firefox Build IDS for 2.0 en-US releases ====
{| border="1" cellpadding="2"
! style="background:#efefef" | '''en-US Release'''
! style="background:#efefef" | '''Windows'''
! style="background:#efefef" | '''Mac PPC'''
! style="background:#efefef" | '''Mac UB'''
! style="background:#efefef" | '''Linux'''
|-
| Firefox 2.0a1
| 2006032100
| 2006032113
| 2006032717
| 2006032100
|-
| Firefox 2.0a2
| 2006051217
| No PPC
| 2006051203
| 2006051217
|-
| Firefox 2.0a3
| 2006052616
| No PPC
| 2006052616
| 2006052616
|-
| Firefox 2.0b1
| 2006071020
| No PPC
| 2006071020
| 2006071020
|-
| Firefox 2.0b2
| 2006082101
| No PPC
| 2006082115
| 2006082101
|-
| Firefox 2.0 RC1
| 2006091818
| No PPC
| 2006091817
| 2006091817
|-
| Firefox 2.0 RC2
| 2006100319
| No PPC
| 2006100316
| 2006100316
|-
| Firefox 2.0 RC3
| 2006101023
| No PPC
| 2006101022
| 2006101022
|-
| Firefox 2.0 FINAL (same build as RC3)
| 2006101023
| No PPC
| 2006101022
| 2006101022
|}
==Release Data - Thunderbird==
A collection of data from the various releases of Thunderbird. (currently rather empty, just set up and will fill in holes an time permits
-tracy)
===User Agent strings===
===Talkback Build IDs===
==== Talkback Thunderbird Build IDS for en-US releases ====
{| border="1" cellpadding="2"
! style="background:#efefef" | '''en-US Release'''
! style="background:#efefef" | '''Windows'''
! style="background:#efefef" | '''Mac PPC'''
! style="background:#efefef" | '''Mac UB'''
! style="background:#efefef" | '''Linux'''
|-
| Thunderbird 1.5
| 2005120115
| 2005120115
| No UB Release
| 2005120113
|-
| Thunderbird 1.5.0.2
| 2006030803
| 2006030804
| No UB Release
| 2006042011
|-
| Thunderbird 1.5.0.4
| 2006051612
| 2006053016
| 2006051612
| 2006051612
|-
|Thunderbird 1.5.0.5
| 2006071912
| No PPC release
| 2006071914
| 2006071913
|-
|Thunderbird 1.5.0.7
| 2006090918
| No PPC release
| 2006090918
| 2006090918
|-
|Thunderbird 1.5.0.8
| 2006102517
| No PPC release
| 2006102517
| 2006102517
|-
|Thunderbird 1.5.0.?
|
| No PPC release
|
|
|}
==== Talkback Thunderbird Build IDS for 2.0 en-US releases ====
{| border="1" cellpadding="2"
! style="background:#efefef" | '''en-US Release'''
! style="background:#efefef" | '''Windows'''
! style="background:#efefef" | '''Mac PPC'''
! style="background:#efefef" | '''Mac UB'''
! style="background:#efefef" | '''Linux'''
|-
| Thunderbird 2.0a1
| 2006072414
| TBC
| No UB Release
| 2006072415
|}
==Tasks for Volunteers==
A [[SoftwareTesting:VolunteerTasks|list of tasks for volunteers]] interested in helping test mozilla software.
==Test Case Wish List==
Firefox
*Safe Mode
**We have general test cases for launching safe mode, but need test cases for each radio button item [owner needed]
*Firefox Context Menu [marcia]
*AJAX sites
**http://www.dynamicajax.com/fr/AJAX_Example_Sites-271_287.html
*Java Sites for the Mac (to test applets)
*Exit Survey (Windows only new feature)
*Test cases to check out handling of Plugins [windows only][mkairys working on them 8-24-06]
**'''Not the plugin finder service'''
**How we handle these on new installs and existing installs
**Top 7 such as Quicktime, Adobe, Flash, etc.
*Live Video streaming test cases
**mlb.com, for example
**Test all of the various protocols that are out there
**Printing Test Cases [owner needed]
*Windows Vista Specific Test Cases
**Parental Controls
**Plug in Handling
**Making Firefox the default application
Thunderbird
*Exit Survey (Windows only new feature)
*Encryption and Digital Signing Test cases

Revision as of 21:12, 15 November 2006

Retrieved from "https://wiki.mozilla.org/index.php?title=QA&oldid=41971"