QA/Automation/Projects/Endurance Tests: Difference between revisions

QA/Automation/Projects/Endurance Tests (view source)

Revision as of 15:32, 19 April 2011

56 bytes added , 19 April 2011

no edit summary

Davehunt

Confirmed users

2,197

edits

@@ Line 13: / Line 13: @@
 |-
 | valign="top" | '''Dates:'''
-| Kick-off: December 2010<br />Phase I: Q1/2011 (goal)
+| Kick-off: December 2010<br />Phase I: Q1/2011<br />Phase II: Q2/2011 (goal)
 |-
 | valign="top" | '''Status:'''
-| Phase I in progress
+| Phase II in progress
 |-
 | valign="top" | '''Discussion:'''
@@ Line 64: / Line 64: @@
 = Roadmap =
-== Phase I - Planned Q1/2011 ==
+== Phase I - Completed ==
 # Command line wrapper for Mozmill that:
@@ Line 74: / Line 74: @@
 # A 'snippet' that uses the new command line wrapper and outputs results that would potentially expose degradation of performance
 # New reports for endurance tests in Mozmill dashboard
+== Phase II (Draft) ==
+# Determine achievable additional performance metrics
+# Determine achievable client hardware details
+# Restart between each test by default (override from command line)
+# All tests must use local content
+# Improve performance of charts with considerable data
+# Create a view for results listed by memory consumption
+# Cache metrics to disk instead of storing in memory during the testrun
 == Phase ? ==
@@ Line 87: / Line 97: @@
 * Abort tests that exceed predetermined resource thresholds
 * Determining saturation points (baselines) in the AUT and monitor for major discrepancies
-== Outstanding Items ==
-# Endurance Automation Wrapper - This is the harness that will launch the AUT, run the test snippet until either the loop count is reached or the maximum time has elapsed, and report results.
-# Initial Test Snippet - The intention is to write a simple test snippet that opens and closes the add-ons manager.
-# Dashboard report for Endurance Tests.
 = Additional Requirements =
 * Use debug nightly builds of Firefox to be able to get a full stack of information