Firefox OS/Performance/Automation/Metric Template
Contents
Metric Name
REMOVEME: This is a fluid format. Anything that doesn't fit the metric at hand should be removed or changed. Try to include some version of all this information, if it would otherwise be appropriate.
Summary
REPLACEME: with a general summary of the metric
Usage
Test Cases
REMOVEME: Add names and high-level descriptions of available cases
- Case 1
- Description
- Case 2
- Description
Precision
REPLACEME: with information about the generally expected precision of the measurements. This determines how much of a change in results should be considered significant.
How to Run On-Demand
REPLACEME: with information on how a developer can run the cases on-demand from command line or otherwise
Published Results
REMOVEME: Add links to the dashboards or areas where the results of tests are published
Branch 1
Device 1 | Device 2 | |
---|---|---|
Case 1 | Datazilla | N/A |
Case 2 | Eideticker | N/A |
Branch 2
Device 1 | Device 2 | |
---|---|---|
Case 1 | Datazilla | N/A |
Case 2 | Eideticker | N/A |
References
- Reference 1
- Reference 2
Development
Design
REMOVEME: This section is documentation of the steps a case goes through, at enough detail to review or duplicate them. Delete any unneeded setup/teardown sections. These can also be links into MozTrap, if the tests are documented there.
Common Setup
- Step 1
- Step 2
Case 1
- Setup
- Step 1
- Step 2
- Test
- Step 1
- Step 2
- Teardown
- Step 1
- Step 2
Case 2
- Setup
- Step 1
- Step 2
- Test
- Step 1
- Step 2
- Teardown
- Step 1
- Step 2
Results
- Repetition, results, etc.
Common Teardown
- Step 1
- Step 2
Project Roadmap
- Lead: TBD
- Tracking Bug: TBD
REMOVEME: This indexes bugs and progress for each step of the test case creation from design to publication. Modify the tables to factor out common designs, remove the instrumentation columns, or whatever is appropriate. The "Availability" column should either be "ETA <release>", "release" for delivered items, or "TBD"
Milestone 1: Design | Milestone 2: Test | Milestone 3: Publication | |||||||
---|---|---|---|---|---|---|---|---|---|
Availability | Test Design | Validity Review | Instrumentation | Workload | On-Demand Test | Results Review | Published Results | Documentation | |
Case 1 | TBD | Bug | Bug | Bug | Bug | Bug | Bug | Bug | Bug |
Case 2 | TBD | Bug | Bug | Bug | Bug | Bug | Bug | Bug | Bug |