Confirmed users
2,197
edits
QA/Automation/Projects/Endurance Tests: Difference between revisions
Jump to navigation
Jump to search
QA/Automation/Projects/Endurance Tests (view source)
Revision as of 00:34, 15 December 2010
, 15 December 2010no edit summary
No edit summary |
No edit summary |
||
| Line 31: | Line 31: | ||
|} | |} | ||
= | = Summary = | ||
We're exploring different ways and or abilities to utilize [https://developer.mozilla.org/en/Mozmill Mozmill] and its testing capabilities to determine if Firefox can sustain the continuous expected load seen through simulated 'real' web usage. These tests would create a simple real world model that employs a high but normal load pattern of actions for the browser in order to determine any potential: | We're exploring different ways and or abilities to utilize [https://developer.mozilla.org/en/Mozmill Mozmill] and its testing capabilities to determine if Firefox can sustain the continuous expected load seen through simulated 'real' web usage. These tests would create a simple real world model that employs a high but normal load pattern of actions for the browser in order to determine any potential: | ||
| Line 37: | Line 38: | ||
* Crashes | * Crashes | ||
= | = Definitions = | ||
Test Snippet: | |||
A test snippet will be one or more actions that can be performed within the AUT written in JavaScript according to the Mozmill specifications. | |||
== | = Roadmap = | ||
== Phase I - Planned Q4 2010 == | |||
# Command line wrapper for Mozmill that: | # Command line wrapper for Mozmill that: | ||
| Line 54: | Line 55: | ||
# A 'snippet' that uses the new command line wrapper and outputs results that would potentially expose degradation of performance | # A 'snippet' that uses the new command line wrapper and outputs results that would potentially expose degradation of performance | ||
== Phase ? == | |||
* Gracefully handle the AUT crashing, and ensure all reports are intact | |||
* Output results to a location on the file system | * Output results to a location on the file system | ||
* Read parameters in from disk | * Read parameters in from disk | ||
| Line 63: | Line 65: | ||
* Present results in an easy to digest way | * Present results in an easy to digest way | ||
* Automate the running of endurance tests | * Automate the running of endurance tests | ||
* Abort tests that exceed predetermined resource thresholds | |||
* Determining saturation points (baselines) in the AUT and monitor for major discrepancies | |||
== Outstanding Items == | == Outstanding Items == | ||
| Line 68: | Line 72: | ||
# Endurance Automation Wrapper - This is the harness that will launch the AUT, run the test snippet until either the loop count is reached or the maximum time has elapsed, and report results. | # Endurance Automation Wrapper - This is the harness that will launch the AUT, run the test snippet until either the loop count is reached or the maximum time has elapsed, and report results. | ||
# Initial Test Snippet - The intention is to write a simple test snippet that opens and closes the add-ons manager. | # Initial Test Snippet - The intention is to write a simple test snippet that opens and closes the add-ons manager. | ||
= Additional Requirements = | |||
* Use debug nightly builds of Firefox to be able to get a full stack of information | |||