THIS PAGE IS A WORKING DRAFT
	The page may be difficult to navigate, and some information on its subject might be incomplete and/or evolving rapidly. If you have any questions or ideas, please add them as a new topic on the discussion page.

Overview

Key Principles

• For all fixed bugs, the state of test coverage should be clear and unambiguous
• Developers have the best insight into which bugs carry the most risk and require attention from QA
• QA is the best group to effectively test fixes, and has the best insight into which bugs require additional testing and verification

Proposal

The thrust of the proposal is simple: we should replace the current assortment of keywords, flags, and whiteboard annotations with a single multi-state flag, and make setting this flag part of the required workflow when resolving a bug in a product component as FIXED. The value of this flag will indicate the next step in testing a bug, or denote what the final state of testing is for that bug. This proposed flag would be known as "testing-status" for the purpose of this proposal.

Visual Workflow

Detailed description of workflow

On resolving a bug (Developer)

For a developer resolving a bug, one of three options should be chosen:

not-testable

The fix is not effectively testable through manual or automated means. No further action will be required.

in-automated-testsuite

The fix has sufficient coverage through automated testing. No further action will be required.

testing-needed

If neither case above applies, QA should assess this bug for further action. (This replaces the verifyme keyword.) Developers are asked to provide as much relevant insight and information as possible for QA when setting this value.

When testing-needed is set (QA)

For all bugs where the flag is set to testing-needed, QA will review that bug and set one of the following four options:

framework-needed

The fix should be tested through automated testing, but current testing frameworks are insufficient.

The automation tools team (A-Team) is responsible for assessing these bugs and providing framework support on a priority basis. Once a framework exists, the flag should be set to automated-test-needed

automated-test-needed

The fix can be tested through existing frameworks.

Once a test is created and checked in, the flag should be set to in-automated-testsuite

manual-test-needed

QA has determined that this bug cannot be tested through automated means, but should be periodically tested as part of manual test runs.

Once a test has been documented and added to moztrap, the flag should be set to in-manual-testsuite

manually-verified

QA has determined one-off testing is sufficient, and no followup action is required. At QA's discretion, time spent on one-off testing may be as little as zero. No further action will be required.