Firefox OS/Performance/Automation/Metric Template

From MozillaWiki
Jump to: navigation, search

Metric Name

REMOVEME: This is a fluid format. Anything that doesn't fit the metric at hand should be removed or changed. Try to include some version of all this information, if it would otherwise be appropriate.

Summary

REPLACEME: with a general summary of the metric

Usage

Test Cases

REMOVEME: Add names and high-level descriptions of available cases

Case 1 
Description
Case 2 
Description

Precision

REPLACEME: with information about the generally expected precision of the measurements. This determines how much of a change in results should be considered significant.

How to Run On-Demand

REPLACEME: with information on how a developer can run the cases on-demand from command line or otherwise

Published Results

REMOVEME: Add links to the dashboards or areas where the results of tests are published

Branch 1

Device 1 Device 2
Case 1 Datazilla N/A
Case 2 Eideticker N/A

Branch 2

Device 1 Device 2
Case 1 Datazilla N/A
Case 2 Eideticker N/A

References

  • Reference 1
  • Reference 2

Development

Design

REMOVEME: This section is documentation of the steps a case goes through, at enough detail to review or duplicate them. Delete any unneeded setup/teardown sections. These can also be links into MozTrap, if the tests are documented there.

Common Setup

  1. Step 1
  2. Step 2

Case 1

  • Setup
    1. Step 1
    2. Step 2
  • Test
    1. Step 1
    2. Step 2
  • Teardown
    1. Step 1
    2. Step 2

Case 2

  • Setup
    1. Step 1
    2. Step 2
  • Test
    1. Step 1
    2. Step 2
  • Teardown
    1. Step 1
    2. Step 2

Results

  • Repetition, results, etc.

Common Teardown

  1. Step 1
  2. Step 2

Project Roadmap

  • Lead: TBD
  • Tracking Bug: TBD

REMOVEME: This indexes bugs and progress for each step of the test case creation from design to publication. Modify the tables to factor out common designs, remove the instrumentation columns, or whatever is appropriate. The "Availability" column should either be "ETA <release>", "release" for delivered items, or "TBD"

Milestone 1: Design Milestone 2: Test Milestone 3: Publication
Availability Test Design Validity Review Instrumentation Workload On-Demand Test Results Review Published Results Documentation
Case 1 TBD Bug Bug Bug Bug Bug Bug Bug Bug
Case 2 TBD Bug Bug Bug Bug Bug Bug Bug Bug

template