Firefox OS/Performance/Automation/Metric Template: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
m (Lakrits moved page FirefoxOS/Performance/Automation/Metric Template to Firefox OS/Performance/Automation/Metric Template: The official spelling of "Firefox OS" leaves a space between the two parts of the name. It's easier to find a page if the...)
 
(14 intermediate revisions by one other user not shown)
Line 1: Line 1:
= Metric Name =
= Metric Name =
''REMOVEME: This is a fluid format. Anything that doesn't fit the metric at hand should be removed or changed. Try to include some version of all this information, if it would otherwise be appropriate.''
== Summary ==
== Summary ==


Line 6: Line 8:
== Usage ==
== Usage ==


=== Cases ===
=== Test Cases ===


''REMOVEME: Add names and high-level descriptions of available cases''
''REMOVEME: Add names and high-level descriptions of available cases''
Line 15: Line 17:
=== Precision ===
=== Precision ===


''REPLACEME: with information about the generally expected precision of the measurements''
''REPLACEME: with information about the generally expected precision of the measurements. This determines how much of a change in results should be considered significant.''
 
=== How to Run On-Demand ===
 
''REPLACEME: with information on how a developer can run the cases on-demand from command line or otherwise''


=== Published Results ===
=== Published Results ===
Line 24: Line 30:


{| class="wikitable"
{| class="wikitable"
! width="120px" |
!
! Device 1
! Device 1
! Device 2
! Device 2
|-
|-
| '''Case 1''' || [http://www.example.com Yes] || N/A
| '''Case 1''' || [http://www.example.com Datazilla] || N/A
|-
|-
| '''Case 2''' || ||
| '''Case 2''' || [http://www.example.com Eideticker] || N/A
|-
|-
|}
|}
Line 37: Line 43:


{| class="wikitable"
{| class="wikitable"
! width="120px" |
!
! Device 1
! Device 1
! Device 2
! Device 2
|-
|-
| '''Case 1''' || [http://www.example.com Yes] || N/A
| '''Case 1''' || [http://www.example.com Datazilla] || N/A
|-
|-
| '''Case 2''' || ||
| '''Case 2''' || [http://www.example.com Eideticker] || N/A
|-
|-
|}
|}
=== How to Run On-Demand ===
''REPLACEME: with information on how to run the cases on-demand from command line or otherwise''


=== References ===
=== References ===
Line 60: Line 62:
=== Design ===
=== Design ===


''REMOVEME: This section is documentation of the steps a case goes through, at enough detail to review or duplicate them. Delete any unneeded setup/teardown sections.
''REMOVEME: This section is documentation of the steps a case goes through, at enough detail to review or duplicate them. Delete any unneeded setup/teardown sections. These can also be links into [https://moztrap.mozilla.org/results/runs/ MozTrap], if the tests are documented there.


==== Common Setup ====
==== Common Setup ====
Line 91: Line 93:
*# Step 1
*# Step 1
*# Step 2
*# Step 2
=== Results ===
* Repetition, results, etc.


==== Common Teardown ====
==== Common Teardown ====
Line 98: Line 104:
=== Project Roadmap ===
=== Project Roadmap ===


* Lead: TBD
* '''Lead:''' TBD
* '''Tracking Bug:''' TBD


''REMOVEME: This indexes bugs and progress for each step of the test case creation from design to publication. Modify the tables to factor out common designs, remove the instrumentation columns, or whatever is appropriate. The "Availability" column should either be "ETA <release>", "release" for delivered items, or "TBD"''
''REMOVEME: This indexes bugs and progress for each step of the test case creation from design to publication. Modify the tables to factor out common designs, remove the instrumentation columns, or whatever is appropriate. The "Availability" column should either be "ETA <release>", "release" for delivered items, or "TBD"''


{| class="wikitable"
{| class="wikitable"
!
! width="120px" |
!
!
! colspan="2" | Milestone 1: Design
! colspan="2" | Milestone 1: Design
Line 109: Line 116:
! colspan="2" | Milestone 3: Publication
! colspan="2" | Milestone 3: Publication
|-
|-
! Name
!  
! Availability
! Availability
! Test Design
! Test Design

Latest revision as of 13:59, 1 February 2015

Metric Name

REMOVEME: This is a fluid format. Anything that doesn't fit the metric at hand should be removed or changed. Try to include some version of all this information, if it would otherwise be appropriate.

Summary

REPLACEME: with a general summary of the metric

Usage

Test Cases

REMOVEME: Add names and high-level descriptions of available cases

Case 1
Description
Case 2
Description

Precision

REPLACEME: with information about the generally expected precision of the measurements. This determines how much of a change in results should be considered significant.

How to Run On-Demand

REPLACEME: with information on how a developer can run the cases on-demand from command line or otherwise

Published Results

REMOVEME: Add links to the dashboards or areas where the results of tests are published

Branch 1

Device 1 Device 2
Case 1 Datazilla N/A
Case 2 Eideticker N/A

Branch 2

Device 1 Device 2
Case 1 Datazilla N/A
Case 2 Eideticker N/A

References

  • Reference 1
  • Reference 2

Development

Design

REMOVEME: This section is documentation of the steps a case goes through, at enough detail to review or duplicate them. Delete any unneeded setup/teardown sections. These can also be links into MozTrap, if the tests are documented there.

Common Setup

  1. Step 1
  2. Step 2

Case 1

  • Setup
    1. Step 1
    2. Step 2
  • Test
    1. Step 1
    2. Step 2
  • Teardown
    1. Step 1
    2. Step 2

Case 2

  • Setup
    1. Step 1
    2. Step 2
  • Test
    1. Step 1
    2. Step 2
  • Teardown
    1. Step 1
    2. Step 2

Results

  • Repetition, results, etc.

Common Teardown

  1. Step 1
  2. Step 2

Project Roadmap

  • Lead: TBD
  • Tracking Bug: TBD

REMOVEME: This indexes bugs and progress for each step of the test case creation from design to publication. Modify the tables to factor out common designs, remove the instrumentation columns, or whatever is appropriate. The "Availability" column should either be "ETA <release>", "release" for delivered items, or "TBD"

Milestone 1: Design Milestone 2: Test Milestone 3: Publication
Availability Test Design Validity Review Instrumentation Workload On-Demand Test Results Review Published Results Documentation
Case 1 TBD Bug Bug Bug Bug Bug Bug Bug Bug
Case 2 TBD Bug Bug Bug Bug Bug Bug Bug Bug

template

Retrieved from "https://wiki.mozilla.org/index.php?title=Firefox_OS/Performance/Automation/Metric_Template&oldid=1052499"