QA archive/Team/What We Do

From MozillaWiki
Jump to: navigation, search

« QA/Team

This is an overview of what we do from a July 2007 presentation.

QA Team structure and members

  • Tools dev - robcee, alice, coop
  • Automated test development - Martijn (mw22), bc, ctalbert
  • Test Execution - marcia, tracy, juanb, tomcat, tchung, abillings, stephend
  • Other: Community-jay, Analysis-ss, Accessibility-tkeenen

Concepts

  • Developers handle unit tests
  • Create automated functional/integration/system tests where possible
  • Use human eyes and ears where necessary
  • Right tool for the job - depends on stage of the project and nature of the thing being testing

Test Execution

  • Products covered: FFx, TBird, customer extensions or branding (kodak, ebay, world cup)
  • Releases covered: Major releases, security releases, partner builds, AMO/WebDev,
  • Activities:
    • bug verification
    • en-US smoke tests, BFT, FFT
    • L10n
    • focused testing/exploratory testing
    • Updates
    • Top sites
    • Extension testing

Analysis

  • bug triage, troubleshooting, investigation
  • Hendrix
  • Reporter
  • crashers - Talkback/Breakpad
  • UNCO - unconfirmed bugs triage- graph


Test Development

  • Manual tests in litmus
  • Automated tests
    • from bugs
    • for new features
    • eggplant tests
    • Test Tools (see below)

Test Tools

  • Tools we have developed for QA use or general use.
  • Litmus
  • RefTest
  • Mochitest
  • XPCShell
  • Buildbot execution Infrastructure
  • Talos - performance framework
  • Minotaur - preferences checking tool for L10n and Partner builds
  • L10n Metadiff
  • Update Checker
  • Spider

Community

  • QMO
  • Test days
  • Bug Days
  • Mailing lists
  • Work with Universities - ex: Seneca College and Club Moz