Confirmed users
1,927
edits
Auto-tools/Projects/SpeedTests: Difference between revisions
Auto-tools/Projects/SpeedTests (view source)
Revision as of 17:43, 30 October 2013
, 30 October 2013no edit summary
No edit summary |
|||
| (2 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
== Status == | |||
'''Disabled'''. This was an interesting project, but the results were not widely used, and periodic breakages due to its mostly-but-not-entirely-automated nature resulted in time spent doing maintenance that had no clear benefit. If any interest in this project rises again, we can resurrect the system and hopefully make improvements to ease maintenance. | |||
However, SpeedTests lives on in a new project, a fork designed for measuring HTML game performance: https://github.com/Mozilla-Games/speedtests | |||
== Results == | == Results == | ||
Results were hosted at http://speedtests.mozilla.org/. | |||
== Description == | == Description == | ||
SpeedTests is a ''semi''automated system that runs a suite of JS-based tests on a series of browsers. Tests are generally free form but are expected to (a) report results and (b) have a definite termination point, for which purpose a small set of JavaScript functions are provided. Tests include suitable modified versions of the IE Test Drive tests, Kraken, V8, and test262. | SpeedTests is a ''semi''automated system that runs a suite of JS-based tests on a series of browsers. Tests are generally free form but are expected to (a) report results and (b) have a definite termination point, for which purpose a small set of JavaScript functions are provided. Tests include suitable modified versions of the IE Test Drive tests, [[Kraken]], V8, and test262. | ||
== Architecture == | == Architecture == | ||
| Line 101: | Line 107: | ||
This is the trickiest part. The framework needs relatively clean profiles with nothing cached, with popups enabled for the test server (since tests are opened in a separate window to control the size), and with access to localhost permitted (only an issue with Opera, I believe). There are also other random things that need to be dismissed, like add-on compatibility dialogs. | This is the trickiest part. The framework needs relatively clean profiles with nothing cached, with popups enabled for the test server (since tests are opened in a separate window to control the size), and with access to localhost permitted (only an issue with Opera, I believe). There are also other random things that need to be dismissed, like add-on compatibility dialogs. | ||
When setting up a new client, I first start a test run with --testmode. This will verify most of the settings without having to sit through all the tests. If I see a browser not being able to load a test, I stop the controller, set all the correct permissions, clear the cache, and run <tt>python speedtests.py archive <browser name></tt>, where <tt><browser name></tt> is one of <tt>firefox</tt>, <tt>nightly</tt>, <tt>safari</tt>, <tt>"internet explorer"</tt>, or <tt>chrome</tt>. To retest this browser, I start the tests again in testmode, providing the browser name. However, this won't quite work in Linux, since killing the speedtests client will kill the browser subprocess as well... So this method will have to be refined. | |||
If something funny is still happening, you can run with --noresults, which will execute the full tests but not report results back to the server, so you don't have to worry about affecting the results with VNC open (VNC is *supposed* to not interfere with framerates, but I have my doubts...). | If something funny is still happening, you can run with --noresults, which will execute the full tests but not report results back to the server, so you don't have to worry about affecting the results with VNC open (VNC is *supposed* to not interfere with framerates, but I have my doubts...). | ||