Personal tools

QA

From MozillaWiki

Jump to: navigation, search

Contents

Mozilla QA

The mozilla.com QA organization is divided into teams focused on Mozilla product areas or technologies.

Mozilla QA Teams

Team Home Page Team Description Go to person Team Email
Desktop Firefox QA Performs testing and release assessment of the desktop Firefox browser Juan Becerra

irc: juanb

Firefox Automation Performs creation of automated tests for Firefox, and maintains the testing infrastructure Henrik Skupin

irc: whimboo

dev-automation@lists.mozilla.org
Browser Technologies QA Performs testing and release assessment of the other Firefox related technologies and services. Tony Chung

irc: tchung

Web QA Performs testing and release assessment of the Mozilla web sites. Stephen Donner

irc: stephend

mozwebqa@mozilla.org
QA Community Supports building the QA Community. Everyone in QA is part of this team :) Marcia Knous

irc: marcia

Services QA Develops, builds, and tests all things client-server Edwin Wong

irc: edwong

Note: Any general questions regarding Mozilla QA can be sent to Bob Moss

Team Home Page Template

Who is Mozilla QA?

Who is Mozilla QA?

Team Meetings

Each of the Mozilla QA Teams conducts a team meeting every other week to discuss release and team specific issues. In addition there is a QA staff meeting conducted every week as well. The table below contains links to the team calendars and links to the current and prior meeting agendas.

Team Meeting Details Meeting Agendas
Mozilla QA Every Wed, 1:30p, Warp Core, Conf 9245 (See Agenda for full dial in info) QA Staff

Last Meeting
Next Meeting

Desktop Firefox Every Tuesday at 2:00pm Pacific, Zombocom Vidyo: QA-Desktop Desktop Firefox QA

Staff Meeting Notes

Firefox Automation Team: Monday, 8:45am Pacific

Ask an Expert: Thursday, 12:00pm UTC

Firefox Automation

Ask an Expert

Browser Technologies Every Other Thurs, 11:00a, Warp Core, Conf 265 Browser Technologies QA

Last Meeting
Next Meeting

Web QA Every Thursday, 9am PDT/PST Web QA

Meetings page

Cloud Services QA Every Tuesday, 3pm PDT/PST Cloud Services QA
QA Community Weekly Meeting have been deprecated and are now integrated into the QA Weekly Staff Meeting QA Community
Taiwan QA team Every Friday, 3pm GMT+8


Mozilla QA Projects

The projects listed below are included from the team project pages. If you need to edit the project, click through to the team page.


Desktop Firefox QA

Project Test Plan Description Scope Timeframe
Firefox 31 QA activities revolving around mozilla-central in the time frame for Firefox 28 merge. Features in Fx31 Firefox 31
Firefox 30 QA activities revolving around mozilla-central in the time frame for Firefox 30 merge. Features in Fx30 Firefox 30
Firefox 29 QA activities revolving around mozilla-central in the time frame for Firefox 29 merge. Features in Fx29 Firefox 29
Firefox 28 QA activities revolving around mozilla-central in the time frame for Firefox 28 merge. Features in Fx28 Firefox 28
Firefox 27 QA activities in the time frame for Firefox 27 merge. Features in Fx27 Firefox 27
Firefox 26 QA activities in the time frame for Firefox 26 merge. Features in Fx26 Firefox 26
Firefox 25 QA activities in the time frame for Firefox 25 merge. Features in Fx25 Firefox 25
Automation Development of automated tests for Desktop Firefox features using Mozmill
Feature Ownership Testing, Planning, and Bug Triage of new and stable Firefox features


Firefox Automation

Some of our projects can be accomplished in a short time while others will take longer, even several months. The prioritized projects reflect the most important projects in the current quarter and include the appropriate tasks which have to be fulfilled during that time. Other active projects have lower priority.

If you are interested in already finished projects, check our projects archive. We also have a some projects that we would like to do in the future!

Prioritized Projects

Each quarter the Mozilla QA team has some goals to finish, and some of those are dedicated to our team. That means we have some prioritized projects to handle throughout the current quarter, which have higher priority then other listed projects on that page.

For 2013 Q4 our prioritized projects are:

  • [DONE] (crossteam) Get SoftVision to lead the automation efforts for QA with our oversight
  • [DONE] (crossteam) Switch Mozmill CI from Mozmill 1.5.x to Mozmill 2.0 (see issue 329)
  • [MISSED] (crossteam) Get Firefox Metro tests executed in production (see issue 215)
  • [DONE] (crossteam) AWSY Firefox OS Memory Tests (bug 899194)
  • [MISSED] Extend Mozmill CI to cover most of our localized beta and release candidate builds (see issue 270)
  • [DONE] Create web application to generate the beta and release Jenkins job configuations (see bug 915164)
  • [DONE] Deploy automated Eideticker system to run nightly for Firefox OS (see bug 904837)

Beside those we also have goals which are stretched and we might work on if there is time left:

  • Setup tryserver Mozmill CI system assist team members in testing their patches
  • Investigate requirements for community driving tool by working together with jdm, lizzard, and pieros

All Projects

Mochitests

Project Technologies Description
WebRTC Javascript, HTML Creation of a testsuite to qualify stability of WebRTC

Mozmill

Project Technologies Description
Automation Python, Jenkins Scripts and Tools to enhance the automation for Mozmill related testing
Dashboard Javascript, HTML, CSS, AJAX, CouchDB, ElasticSearch Dashboard to visualize Mozmill test results from the automated test-runs and the Mozmill Crowd extension
Shared Modules Javascript Building and enhancing the API and shared modules for our Mozmill tests for Firefox

Other Projects

Project Technologies Description
Add-ons CSS, Javascript, XML To help any team in the Mozilla project we work on several extensions.
Documentation Wiki Establish the team presence across websites (Wiki, MDN) and improve the documentation to increase collaboration with the test automation community
Infrastructure ESX, VMware Building up and maintaining the infrastructure for automation tasks
Open Web Applications Javascript, Python Implemention of necessary APIs and support the addition of tests to execute automated tests for web apps in Firefox


Browser Technologies

This section should contain a list to the active current team project page. The section will be included as part of the top level QA organization page.

Project Description
Fennec 10.0 Test Plan Mobile Firefox QA site summarizing Test Strategy, Feature set, assignments, Milestone Testplans, and schedules
Firefox Sync Test Plan Firefox Sync site (both extension and Integrated feature) summarizing Test strategy, testcases, Milestone testplans, and schedules
Test Pilot Test Plan Test Pilot QA site summarizing Test strategy, testcases, Milestone testplans, and schedules
Jetpack Test Plan Jetpack QA site summarizing Test strategy, testcases, Milestone testplans, and schedules
Firefox Home 2 Test Plan] Firefox Home 2 (aka Iphone Sync App) site summarizing Test strategy, testcases, Milestone testplans, and schedules
Web APIs Test Plans] Web APIs home page summarizing test pages to support upcoming APIs for mobile


Web QA

Web QA Overview

Through a variety of tools and approaches, Web QA aims to ensure high quality web applications, of both external and internal applications and websites.

Contact Us

I have something I need tested -- now what?

  1. We'd really appreciate it if you could sync up with the Web Production team; letting them know ahead of time on something you're working on helps us all get in-sync with the project, and shortens turnaround time
  2. Please take a look through our Support Levels document, and let's begin the discussion of what support your project needs, by when, and how we can help work towards making that a reality
  3. File a bug in Bugzilla, under the product Testing, and component "WebQA", asking us to test your site/application.  assigning to "mozwebqa@mozilla.org", and I'll be sure to assign it to the right person, or make sure the team tests
  4. In the bug, please let us know of any pertinent schedules, deadlines, requirements, limitations, etc.  Also, pointing us to design documentation or specs, such as mockups/HTML cuts/wireframes, etc. would be immensely helpful
  5. Please also list any third-party developers/firms we'll have to work with, to get the potential issues we might have found, addressed.  (And please make sure they have a Bugzilla account, too.)
  6. How to get a project in automation

New Hires (useful chiefly for paid Mozilla folks)

Some useful information about getting started is available here.

Team Details

We welcome community members to join us and take on tasks and responsibilities; please see our quality.mozilla.org page for more information/to engage.

Get Involved

We'd love to have your help in testing our many projects!

Community

Community contributor documentation

Next Team Work Week

March 17th - 21st

QA PTO Calendar

Team Members and Assignments

Name Email Address Availability Project Assignments / Interests
Raymond Etornam Agbeame [withheld to avoid spam] Employee Mozilla.com, Plugin Check, MDN
Rebecca Billings rbillings at moz Employee SUMO lead, MozTrap, QMO
Matt Brandt mbrandt@mozilla.com Employee Firefox OS, Socorro, Affiliates, Mozillians, Firefox Health Report, +automation
Zac Campbell zcampbell@mozilla.com Employee Web QA automation, Gaia automation
Stephen Donner stephend@mozilla.com Employee Manager
Andrei Hutusoru andrei.hutusoru@softvision.ro Contractor Automation
Viorela Ioia viorela.ioia@softvision.ro Contractor Automation
Krupa Raj kraj@mozilla.com Employee AMO/Mozilla Marketplace lead
Bob Silverberg bsilverberg@mozilla.com Employee Automation
Florin Bogdan Strugariu florin.strugariu@softvisioninc.eu Contractor Automation

Automation

See our Automation page for details of Web QA's automation efforts.

Road to Gaia

Gaia Testing Plan

  1. Run MarionetteJS Tests on Device, in Jenkins - bug 979077
    1. Which requires profile pushing/on-device target in the runner - bug 978884
  2. Port the existing Clock tests in Python over to JavaScript - bug 971135
    1. Done, but needs both of the above bugs finished, to be useful

The following bugs are all Marionette or Gaia issues prohibiting more reliable testing/feature-testing:

ID Summary Priority Status
972557 To address Bug 971747 - Add an exception that will report when b2g cannot be stopped -- NEW
997791 Investigate and fix tests fail to enable wifi during setup -- NEW
813561 [B2G] Buttons under lockscreen are returning as visible -- NEW
813583 [B2G] Homescreen iframe not interpreted as visible -- NEW
833761 [B2G] Marionette is switching frames even when not asked to -- NEW
846456 [B2G] Marionette Cards view elements not returning is_displayed() correctly -- NEW
859264 Click's scroll into view is too aggressive on the Select wrapper -- NEW
859748 [B2G] ElementNotVisible exception against element that is_displayed=True -- NEW
840931 [B2G] test_call_contact fails when a sleep is introduced into it -- NEW
853622 need sendNativeTouchEvent -- NEW
853878 [B2G] system-banner element returning false is_displayed() status -- NEW
860854 JavascriptException: AbortError on a run of test_kill_multiple in CI -- NEW
862167 Can neither tap nor click on the Calendar Delete event button -- NEW
863117 After running a Gaia UI Test and locking, unlocking homescreen, UI shifts dramatically and unusably -- NEW
865232 Get is_displayed() in Marionette working as well as is reasonably possible, relative to life, happiness and human mortality -- NEW
869688 Click works but tap does not on msg textarea in Messages app -- NEW
876411 Click works but tap does not on tabs on browser app -- NEW
877163 still need to scrollIntoView when using el.tap() in email app -- NEW
883279 Cannot perform move action in time picker system dialog -- NEW
887274 Mechanism for handling security dialogs with Marionette -- NEW
924912 Provide a way to determine that B2G is ready to use -- NEW
947317 MarionetteException after a tap() closes a frame - MarionetteException: {u'message': u'"toString" is read-only', u'error': u"error occurred while processing 'findElement"} -- NEW
971747 Phone freezes while starting and stopping the b2g process -- NEW
932804 Tapping on select all button doesn't work on Gmail or Outlook frame, before importing contacts P2 NEW
946130 Flick and other actions not working on System app P2 NEW
965256 Make desktopb2g socket.timeout error messages more useful P2 NEW
862156 Marionette thinks that the play button in the music app is not displayed P3 NEW
987760 [v1.4] Investigate test_settings_wifi.py failure -- REOPENED

Open; Resolved; Total (0% complete)


Meetings

Every Thursday @ 9am PDT/PST

Meeting Notes

Team Goals

Current Projects + Webdev Processes

Test Plan Template and Project Checklist

<onlyinclude>

Project (page/bug) Description Kanban Lead/contact Staging server(s) Jenkins URL
Affiliates Affiliates . Team https://affiliates-dev.allizom.org CI
Mozillians Mozillians - the Mozilla Community Phonebook 1.0 Mozillians kanban Matt https://mozillians-dev.allizom.org CI
Snippets Snippets Snippets kanban Raymond (Pacific) / Zac (Europe) http://snippets.stage.mozilla.com CI
AMO Add-ons website
Krupa https://addons-dev.allizom.org/ (trunk/master) CI
AMO Automation Add-ons Automated Tests
Team / Softvision http://support.allizom.org/ (trunk); http://support-release.allizom.org/ (branch) CI
Raymond / Bob http://www-dev.allizom.org CI
Plugin Check Plugin Check Plugin Check Raymond http://www-dev.allizom.org/en-US/plugincheck/
Socorro Crash Stats (see https://intranet.mozilla.org/Socorro_Servers) for more info about the staging env) 2.4.2 Matt Brandt / Stephen (backup) http://crash-stats-dev.allizom.org (trunk/dev-staging)
https://crash-stats.allizom.org (release-staging)
CI
Mozilla Developer Network (MDN) Developer Documentation, News, Tips and Tricks TestPlan Raymond https://developer-dev.allizom.org/en-US/ CI
QMO 4.0 Test Plan Team, Rbillings CI
wiki.mozilla.org wiki.mozilla.org . Team, Glenn CI
Bouncer Bouncer Team, Matt http://download.allizom.org CI
Firefox OS Firefox OS Team CI
Marketplace Marketplace Team CI
Moztrap Moztrap Team CI

** Don't forget to make yourself familiar with the latest Firefox release schedules, as we might be doing work involved with them **

Web Engagement Projects


Mozwebqabot - Our IRC Bot

See our Mozwebqabot info page for details of Web QA's friendly IRC bot.

Resources

Support Levels

Please read through our support levels when proposing a new project for us.

Performance and Load-Testing Tools

Useful Tools

Security

Infrastructure / Server Info

Tips and Pointers

Community Outreach

Interview Process + Questions

Reference Links

Presentation Slide Archives

QA Community

This section should contain a list to the active current team project page. The section will be included as part of the top level QA organization page.

Project Description
QMO Mozilla QA's homepage!
TCM An update of our manual test case management system
Crashkill Crashkill Home Page.
HTML5 form enhancements HTML5 form accessibility tracking.
Event Coalescing Coalesce accessibility events.
Bug Days Bug day instructions and templates
Test Days Test Day instructions, scheduling and templates
Community Plans Community Plans - Planning for QA Community Events
Video Projects Video Projects that our team is working on


Mozilla Community Champions

Mission

  • The overarching goal for 2014 is to have at least 80 community members engaged and active each week by Q4 2014.

Members

  • Marcia Knous, FX OS [irc: marcia]
  • Liz Henry, Desktop [irc: lizzard]
  • Aaron Train, Firefox for Android [irc: AaronMT]
  • Rebecca Billings, Web QA [irc: rbillings]
  • Karl Thiessen, Services QA [irc: kthiessen]
  • Parul Mathur [irc: pragmatic]

Meeting Notes

Note: Meet in Marcia's vidyo room. Mt. View Angel Conference Room

2014 Goals

Reference/Brainstorming Documents

Mozilla QA Weekly Scrum Pads

For team communications, each QA team maintains an etherpad document for each week. Within this document team members keep a running list of weekly tasks they are working on as well as relevant project updates and issues. Each team member should check into the etherpad and enter some updates on the right side of the etherpad document.

Helpful Links

These are tools and/or webpages the QA community uses to interact with the development of Mozilla's products, but are not necessarily tools we use to assure their quality.