QA: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
 
(422 intermediate revisions by 59 users not shown)
Line 1: Line 1:
==What We Use==
[[File:Qa-badge.jpg|250px|left]]
We typically use nightly optimized (non-debug) builds for daily usage. However, we also use the release builds (of course!), as well as older builds when trying to narrow down regression windows.
* For nightly builds, check out any of the [http://www.mozilla.org/mirrors.html mirrors], then drill down to the '''<product_name>/nightly/''' directory. While you can go to '''<product_name>/latest-*''' directories, the problem there is that you don't necessarily known '''''when''''' those builds were made. It's best to access the '''specific build-date directory''' (e.g., 2005-03-17-08-trunk), to know what you're grabbing.
* For older builds not listed in the mirror pages, check out the archives at http://archive.mozilla.org/pub/
* For release builds, simply go to any of the mirrors and drill down to '''<product_name>/releases/''' and select the appropriate directories for version, platform and locale.


===Bugzilla===
<br><br>
We depend on [https://bugzilla.mozilla.org Bugzilla] for filing and tracking bugs and features. We frequently use the [https://bugzilla.mozilla.org/query.cgi query tools], both the "Advanced Search" and "Find a Specific Bug" queries. With the bug count reaching 300,000, there are a couple ways to see what's been frequently reported and duplicated:
* [http://bugzilla.mozilla.org/duplicates.html Most frequently reported bugs query]
* [http://bugzilla.mozilla.org/duplicates.cgi?sortby=bug_no Most recently duplicated bug query]


===Testrunner / Litmus ===
Welcome to the Mozilla QA wiki home page.  
We currently use [[Testrunner]] at http://testrunner.mozilla.org for test development and execution of various types of test runs like smoketests and basic functional tests (BFT's). To view the following test plans you need a Testrunner login.
* [http://testrunner.mozilla.org/tr_edittestcases.cgi?plan_id=25&product_id=21 Firefox smoketests]
* [http://testrunner.mozilla.org/tr_edittestcases.cgi?plan_id=26&product_id=23 Thunderbird smoketests]
* [http://testrunner.mozilla.org/tr_edittestcases.cgi?plan_id=27&product_id=21 Firefox BFT's]
* [http://testrunner.mozilla.org/tr_edittestcases.cgi?plan_id=32&product_id=23 Thunderbird BFT's]


We are also currently in the process of developing [http://litmus.mozilla.org/ Litmus] as a replacement for Testrunner. Litmus will build upon the testcase management aspects of Testrunner, adding the searchability, reporting, and trending tools that are critical to our QA work. More information about Litmus development is available in the [[Litmus|Litmus wiki]].
We are a guild of individuals united by a common passion to make the quality of the products Mozilla creates the best it can possibly be. There are a plethora of ways to get involved and be a part of the Mozilla Quality Project to help ensure, empower, extend, and enjoy the open web. We're excited you're here to join us.
<br clear=all>


===Talkback===
= Quality Assurance Principles =
When an application crashes, we use Talkback to examine the crash information. A publically available Talkback server can be accessed at http://talkback-public.mozilla.org
These are the foundational elements that we evaluate to judge quality for each of our projects. These are all interdependent and interrelated and the list is probably not exhaustive, but it will give you an idea of what we are passionate about.
* Dependability - Every product should be stable and dependable
* Delight - Doing "what it should do" is not enough. The product must delight the user with its functionality.
* Security & Privacy - Mozilla believes very strongly that our products must put our users in control of their data and must take every step we can to ensure your security while on the web.
* Performance - The products must meet or exceed the performance requirements we and our users demand.
* Web Platform - Our mission is to extend, empower, and keep the web platform open for all. We do that by releasing great products that people love, which push the boundaries and capabilities of the open web ever farther.


There are a number of tools available there:
= Get In Touch! =
* Reports (http://talkback-public.mozilla.org/reports) - Browse topcrash data for all Mozilla products. Topcrash lists, stack traces, user comments and urls are available to help users reproduce crash bugs.
We'd love to hear from you. We hang out on Mozilla IRC (irc.mozilla.org) in the #qa channel. We use the [https://groups.google.com/forum/#!forum/mozilla.dev.quality dev-quality] mailing list for communication and discussion. Our blog is at [https://quality.mozilla.org quality.mozilla.org (also known as QMO)].
* FastFind - Lookup individual crash incidents.
* QuickSearch - Run customized queries to get a better understanding of specific crashes.


How To:
While you're at it, please register as a Mozillian and [https://mozillians.org/en-US/group/qa/ join us]!
* Find your Talkback incident id
* Pick a component for crash bugs (http://www.mozilla.org/quality/help/pick-a-component-for-crashers.html)
* Talkback Talkback Analysis, or [[Talkback_Topcrash_Analysis|how to log a topcrash bug]]
* Use FastFind results
* Use QuickSearch results
* see also [http://kb.mozillazine.org/Talkback Talkback] article at kb.mozillazine.org.


ToDo List:
= Teams =
* Clean up and update Talkback reporting system and website
We are organized into product focused areas. You can find our teams and team leads' IRC nicknames below.
* Create Talkback/release database to track builds
{| class="fullwidth-table" border="1"
* Bug fixes and enhancements for Talkback FastFind/QuickSearch tools.
|-
| style="font-weight: bold; background: #DDD; width: 15%" | QMO Team Page
| style="font-weight: bold; background: #DDD; width: 15%" | Team Lead
| style="font-weight: bold; background: #DDD; width: 20%" | Team Wiki
| style="font-weight: bold; background: #DDD; width: 30%" | Primary Focus Areas
| style="font-weight: bold; background: #DDD; width: 20%" | Primary Team IRC Channels
|-
| [https://quality.mozilla.org/teams/firefoxqe/ Firefox QE]
| None at this time. Team members are embedded in Engineering Teams
| [[QA/Firefox|QA/Firefox]]
| Firefox browser (Desktop and Android)
| #qa
|-
| [https://quality.mozilla.org/teams/test-engineering/ Firefox Test Engineering]
| sphilp
| [https://wiki.mozilla.org/TestEngineering https://wiki.mozilla.org/TestEngineering]
| Firefox Test Engineering
| #qa, #fx-test
|-
| [https://quality.mozilla.org/teams/web-qa/ Web and QA Services]
| Stephend
| [[QA/Execution/Web_Testing|QA/Execution/Web_Testing]]
| Mozilla websites, Marketplace, QA specific sites
| #qa, #mozwebqa
|-
| [https://quality.mozilla.org/teams/services/ Cloud Services QA]
| sphilp
| [[QA/Services|QA/Services]]
| Mozilla Cloud Services (sync, simplepush, OAuth, Firefox Accounts etc)
| #qa, #services-qa
|-
| [https://github.com/mozilla/iris Mozilla Iris]
| Matt Wobensmith (Core)<br />Tracy Walker (Test)<br />Anthony Hughes (Deploy)
| [[QA/Product_Integrity/Mozilla_Iris]]
| Test automation
| #qa-automation
|}


===Development tools===
[[File:Mozilla_QA_2014_team.jpg|border|600px|2014 QA Workweek in Mountain View]]
We also use several development tools for tracking changes, especially useful for narrowing down regression windows!
* Tinderbox (http://tinderbox.mozilla.org/showbuilds.cgi) to visually display our continuous build system. Great to see who checked in what, the state of the build, as well as quick links to automated performance tests.
* LXR (http://lxr.mozilla.org) to examine the source code.
* Bonsai (http://bonsai.mozilla.org) to narrow down when changes were made.


===Automation===
<small> The old version of this [https://wiki.mozilla.org/QA_Archived_Home_Page page is archived here]</small>
In an effort to the improve the testing potential of our team, we are constantly investigating ways to leverage automated testing tools to offload some of the regression testing burden from our core testers.  
 
* A [[QA Automation Candidate Software|listing of the various candidate software we have considered]] and the status of each.

Latest revision as of 21:18, 10 January 2019

Qa-badge.jpg



Welcome to the Mozilla QA wiki home page.

We are a guild of individuals united by a common passion to make the quality of the products Mozilla creates the best it can possibly be. There are a plethora of ways to get involved and be a part of the Mozilla Quality Project to help ensure, empower, extend, and enjoy the open web. We're excited you're here to join us.

Quality Assurance Principles

These are the foundational elements that we evaluate to judge quality for each of our projects. These are all interdependent and interrelated and the list is probably not exhaustive, but it will give you an idea of what we are passionate about.

  • Dependability - Every product should be stable and dependable
  • Delight - Doing "what it should do" is not enough. The product must delight the user with its functionality.
  • Security & Privacy - Mozilla believes very strongly that our products must put our users in control of their data and must take every step we can to ensure your security while on the web.
  • Performance - The products must meet or exceed the performance requirements we and our users demand.
  • Web Platform - Our mission is to extend, empower, and keep the web platform open for all. We do that by releasing great products that people love, which push the boundaries and capabilities of the open web ever farther.

Get In Touch!

We'd love to hear from you. We hang out on Mozilla IRC (irc.mozilla.org) in the #qa channel. We use the dev-quality mailing list for communication and discussion. Our blog is at quality.mozilla.org (also known as QMO).

While you're at it, please register as a Mozillian and join us!

Teams

We are organized into product focused areas. You can find our teams and team leads' IRC nicknames below.

QMO Team Page Team Lead Team Wiki Primary Focus Areas Primary Team IRC Channels
Firefox QE None at this time. Team members are embedded in Engineering Teams QA/Firefox Firefox browser (Desktop and Android) #qa
Firefox Test Engineering sphilp https://wiki.mozilla.org/TestEngineering Firefox Test Engineering #qa, #fx-test
Web and QA Services Stephend QA/Execution/Web_Testing Mozilla websites, Marketplace, QA specific sites #qa, #mozwebqa
Cloud Services QA sphilp QA/Services Mozilla Cloud Services (sync, simplepush, OAuth, Firefox Accounts etc) #qa, #services-qa
Mozilla Iris Matt Wobensmith (Core)
Tracy Walker (Test)
Anthony Hughes (Deploy) 		QA/Product_Integrity/Mozilla_Iris Test automation #qa-automation

2014 QA Workweek in Mountain View

The old version of this page is archived here

Retrieved from "https://wiki.mozilla.org/index.php?title=QA&oldid=1206065"