B2G/QA/Triage

Overview

This document provides a summary of how we manage requests for QA support through bugzilla, more specifically for the B2G QA team.

Summary of Keywords and Flags

Keywords

Note: Only one keyword should be used optimally to represent what is needed by QA.

steps-wanted

When reproducible steps to reproduce are needed for a certain bug.

testcase-wanted

When a reduced test case is needed for a certain bug.

regressionwindow-wanted

When a regression range is necessary to identify a first good build and first bad build for a particular bug.

verifyme

When a verification of a bug is needed to ensure that the patch on the bug actually fixed the problem.

qawanted

When QA support is needed on the bug, but none of the above keywords apply. When this keyword is used, the person flagging qawanted needs to indicate what QA support is needed.

Flags

in-moztrap

When a bug or feature could benefit from having formal test case coverage. See below for a more detailed discussion on the workflow of using this flag.

QA Contact

The use of QA contact field should be set optimally when the person is actively investigating something in relation to the bug, usually in connection to one of the keywords above. The QA contact field should aim to only be set by the person who is planning on working on the bug for some piece of QA.

There are some exception cases to this rule however stated below.

The first case is if an explicit owner is known up front for a component who actively manages the bugs. In this case, the default QA contact will be used to address QA needs for the bugs.

The second case is if a high need is raised in triage to find an owner for a bug. In this case, triagers should set the QA contact to the QA representative in triage to find a QA owner to address QA needs for the bug. If no QA representative is present, then one of the two sheriffs should be set as the QA owner by area:

• Gaia General: Naoki Hirata
• Platform General: Geo Mealer
• Apps Gaia and Platform: Jason Smith

QA Requests

For addressing bugs that need QA involvement with the keywords stated above except verifyme, the table below summarizes the urgency of response we follow for QA requests.

Urgency	Bugzilla Flags	Query
Very High	blocking-b2g = ?, tef+, shira+, leo+	Bug Query
High	blocking-b2g = -	Bug Query
Medium	tracking-b2g = ? or +	Bug Query
Low	Anything else	Bug Query

Note: For very high urgency bugs, we plan to provide an initial response on the bug within two business days.

Test Case Coverage Wanted

Summary

For requesting test case coverage, we shall make use of the in-moztrap flag. This flag aims to capture when someone thinks test case coverage would be a good for a feature or bug filed in Bugzilla. From a feature perspective for a release, this aims to help capture a secondary checklist to make sure that our QA team has captured test cases for all features for a particular release. However, for feature release test case development, usage of this flag is not supposed to be used for being the sole tracking area - a separate spreadsheet will be maintained for tracking the details of the work being done. For miscellaneous cases needing test case coverage, this process also helps find missing test case coverage and sometimes, can identify untracked features that product did not know about.

To learn more information about the workflow for test case coverage, see this diagram that describes the workflow with the in-moztrap flag.

Request Importance

The following table below summarizes the priority for test case coverage requests that come from the in-moztrap flag.

Importance	Bugzilla Flags	in-moztrap? Query	Untriaged Query
High	blocking-b2g = tef+ and shira+	Bug Query	Bug Query
Medium	blocking-b2g = leo+	Bug Query	Bug Query
Low	tracking-b2g18+, approval-b2g18, and approval-v1 bugs	Bug Query	Bug Query

Verifications

For addressing bugs that need verification, our QA team will follow a manual process of analyzing resolved fixed bugs, determining which bugs need verification, and verifying those bugs on the appropriate branches. If someone internal or external to the team knows that a bug is worthwhile verifying, then they will flag the bug as verifyme. The following table below summarizes what bugs should be verified and in what order.

Importance	Bugzilla Flags	Branches to Verify	Query
High	blocking-b2g = tef+ and shira+	v1.01 and v1-train	Bug Query
Medium	blocking-b2g = leo+	v1-train	Bug Query
Low	tracking-b2g18+, approval-b2g18, and approval-v1 bugs	v1-train	Bug Query

Recommended QA Contact by Area

For reference, here is a table indicating recommended QA contacts by major functional areas for B2G.

Area	QA Contact
Gallery	John Hammink
Camera	John Hammink
Calendar	Jason Smith
Bluetooth	Walter Chen
Clock	Askeing Yen
Keyboard	William Hsu
Settings - App Permissions	Jason Smith
Lockscreen	Al Tsai
SMS	Paul Yang
Browser	Naoki Hirata
Contacts	Isabel Rios
Cost Control	Carlos Martinez Toral
E-Mail	Tracy Walker
FMRadio	Askeing Yen
Music	Marcia Knous
App Install	Jason Smith
App Permissions	Jason Smith
App Updates	Jason Smith
Video	Marcia Knous
Preinstalled Apps	Jason Smith
Payments - Gaia & Gecko	Jason Smith
Identity - Gaia & Gecko	John Morrison
Captive Portal	Askeing Yen
Multimedia Streaming	Al Tsai
Roaming Charges	Walter Chen
Dialer	Rafael Marquez
Localization	Delphine Lebédel
Miscellaneous	John Hammink