Confirmed users
270
edits
Benchmarks: Difference between revisions
→Graphics
Jprosevear (talk | contribs) |
|||
| (19 intermediate revisions by 7 users not shown) | |||
| Line 8: | Line 8: | ||
* '''[http://html5test.com HTML5Test]''' | * '''[http://html5test.com HTML5Test]''' | ||
** Targeted for "HTML5 Ready" apps/usage | ** Targeted for "HTML5 Ready" apps/usage | ||
** Tests existence of certain APIs but not | ** Tests existence of certain APIs but not really compliance; easy to game and being gamed. | ||
** We disagree that the FileSystem API should be part of this suite, no vendor other than Google implements and they seem to have given up on this spec too | ** We disagree that the FileSystem API should be part of this suite, no vendor other than Google implements and they seem to have given up on this spec too. | ||
* '''[http://rng.io Ringmark]''' | * '''[http://rng.io Ringmark]''' | ||
** Targeted for mobile HTML5 app capabilities | ** Targeted for mobile HTML5 app capabilities | ||
| Line 18: | Line 18: | ||
* '''[http://acid3.acidtests.org/ Acid3]''' | * '''[http://acid3.acidtests.org/ Acid3]''' | ||
** All major browsers pass this | ** All major browsers pass this | ||
** Should not be used as a performance test | |||
* [http://test262.ecmascript.org/ Test262] | * [http://test262.ecmascript.org/ Test262] | ||
** Javascript compliance | ** Javascript compliance | ||
| Line 34: | Line 35: | ||
** test suite source at [https://github.com/eoftedal/csp-testing https://github.com/eoftedal/csp-testing] | ** test suite source at [https://github.com/eoftedal/csp-testing https://github.com/eoftedal/csp-testing] | ||
** Webkit and Gecko have implementations of CSP, IE 10 only implements the CSP sandbox directive (optional in CSP 1.0, required in CSP 1.1) | ** Webkit and Gecko have implementations of CSP, IE 10 only implements the CSP sandbox directive (optional in CSP 1.0, required in CSP 1.1) | ||
** https://bugzilla.mozilla.org/show_bug.cgi?id=746978 | |||
** CSP 1.0 Browser Compliance Testing - http://csptesting.herokuapp.com/?disable_old_headers=true and http://csptesting.herokuapp.com/ | |||
== Performance == | == Performance == | ||
| Line 44: | Line 47: | ||
** Only tests built-in browser | ** Only tests built-in browser | ||
** No source code or detailed description of tests available | ** No source code or detailed description of tests available | ||
* [http://www.robohornet.org/# RoboHornet] | |||
** Brand new, seems to be in an alpha state | |||
=== Graphics === | === Graphics === | ||
| Line 54: | Line 59: | ||
** Text Column Test | ** Text Column Test | ||
* '''[http://www.craftymind.com/guimark3/ GUIMark3]''' | * '''[http://www.craftymind.com/guimark3/ GUIMark3]''' | ||
** A basic test of Canvas and video performance, largely focusing on comparing similar workloads between Flash and HTML5. | |||
** The result of each test is intended to be a framerate, with frames rendered via setInterval. | |||
** On a fast machine, results are meaningless -- likely to hit the throttled rate given the simple workload. | |||
** Bitmap Test | ** Bitmap Test | ||
*** 2D canvas | *** 2D canvas | ||
** Vector Test | ** Vector Test | ||
*** 2D canvas | *** 2D canvas | ||
*** Dominated by radial gradient performance | *** Dominated by radial gradient performance when not running with Direct2D | ||
** Compute Test | ** Compute Test | ||
** Video Test | ** Video Test | ||
| Line 64: | Line 72: | ||
** Only tests drawImage() | ** Only tests drawImage() | ||
** http://muizelaar.blogspot.ca/2011/02/drawing-sprites-canvas-2d-vs-webgl.html | ** http://muizelaar.blogspot.ca/2011/02/drawing-sprites-canvas-2d-vs-webgl.html | ||
* [http://ie.microsoft.com/testdrive/Performance/PenguinMark/ PenguinMark] | |||
=== Javascript === | === Javascript === | ||
* '''[http://v8.googlecode.com/svn/data/benchmarks/ | * '''[http://v8.googlecode.com/svn/data/benchmarks/v7/run.html V8]''' | ||
* [https://developers.google.com/octane/ Octane] | * [https://developers.google.com/octane/ Octane] | ||
** Essentially V8 version 8 | ** Essentially V8 version 8 | ||
** Alon is working on an emscripten generated alternative | *** Repackages various parts of V8 | ||
*** https://hacks.mozilla.org/2012/08/no-single-benchmark-for-the-web/ | |||
** Uses WebBox2D that is slower than emscripten generated version | |||
*** Alon is working on an emscripten generated alternative | |||
* [http://v8.googlecode.com/svn/branches/bleeding_edge/benchmarks/spinning-balls/index.html Spinning Balls] | |||
** Used to test effectiveness of [http://blog.chromium.org/2011/11/game-changer-for-interactive.html garbage collection] | |||
* '''[http://krakenbenchmark.mozilla.org/ Kraken ]''' | * '''[http://krakenbenchmark.mozilla.org/ Kraken ]''' | ||
** Mozilla developed | ** Mozilla developed | ||
| Line 75: | Line 90: | ||
* '''[http://www.webkit.org/perf/sunspider/sunspider.html Sunspider]''' | * '''[http://www.webkit.org/perf/sunspider/sunspider.html Sunspider]''' | ||
** Tests run very fast which makes measurement error significant and means that the tasks are not scaled to a significant amount of work. | ** Tests run very fast which makes measurement error significant and means that the tasks are not scaled to a significant amount of work. | ||
*** Has 26 tests that run in 200-300ms, depending on the machine, so about 10ms per test. The differences between current browsers are now on the order of running a test in 9ms instead of 10ms, so practical significance is limited. Another problem with SunSpider is that with 10ms per tests, making a more advanced JIT doesn't improve your score, because the compile time ends up outweighing the improved run time. This doesn't matter so much for comparing browsers, but it means SunSpider doesn't really drive JS engines to get faster on big apps. If anything, it tells browsers to create new startup modes so they can run small programs a shade faster. (dmandelin) | |||
=== DOM === | === DOM === | ||
* '''[http://dromaeo.com/ Dromaeo]''' | * '''[http://dromaeo.com/ Dromaeo]''' | ||
** Mozilla developed | ** Mozilla developed. | ||
** Maintenance status unclear; no reasonable contact info for how to get the deployed benchmark updated. | |||
** Source lives at https://github.com/jeresig/dromaeo | |||
** Microbenchmarks that are not representative of any real workloads and in fact are often completely bogus. | |||
** Easily gamed (and has been actively gamed in the past by UAs). | |||
** Incentivizes making already fast stuff faster more than making slow stuff faster. | |||
** Often doesn't measure what it claims to be measuring. | |||
** Best (because pretty much only, not because it's so good) DOM benchmark around. | |||
=== Page Load === | === Page Load === | ||
* [http://webpagetest.org WebPageTest] | |||
** seems very good | |||
** cross-browser | |||
** simulates different networks | |||
* iBench | * iBench | ||
** uses onLoad() which is not credible for performance | ** uses onLoad() which is not credible for performance | ||
| Line 88: | Line 115: | ||
* [http://wrla.ch/eideticker/dashboard Eideticker] | * [http://wrla.ch/eideticker/dashboard Eideticker] | ||
** Mozilla developed | ** Mozilla developed | ||
* [https://ie.microsoft.com/testdrive/Performance/Chalkboard/ Chalkboard] | |||
** Does CSS transforms on a single SVG image | |||
*** Tests rasterization of SVG images | |||
** Not that interesting for real pan and zoom | |||
== Other == | == Other == | ||