There's also a [https://bugzilla.mozilla.org/show_bug.cgi?id=640791 tracking bug] for bugs relating to profiling, tools and infrastructure. (These bugs overlap several of the above lists.)
== Current Infrastructure ==
The main automated infrastructure we currently have is the Tp4 and Tp5 Talos tests. They load a bunch of popular (Alexa top 100) pages sequentially, take memory measurements (eg. private bytes on Windows) periodically, and somehow average those measurements into a single number. If that number increases significantly, certain people receive a warning email.
Problems with this:
* Sometimes increases in the measurements fall through the cracks. I'm not sure what to do about this. The variation in measurements between runs doesn't help here.
* They open only one tab at a time. This doesn't reflect how people browse, and doesn't stress the browser. In comparison, Sayre's [https://bugzilla.mozilla.org/show_bug.cgi?id=631705 Membuster] benchmark opens a heap of new tabs.
* The "average of a sample taken every 20 seconds" is a curious measure.
* The metrics are odd and inconsistent across platforms. Using the "explicit" and "resident" memory reporters (see about:memory for details) would be better. Some statistics about page faults would also be good.
Even with these flaws, they're a good start, but there's room to improve.
== Additional Infrastructure ==
* Automated testing using other existing tools (Valgrind, trace-malloc) should be set up. (Valgrind is covered by {{bug|631811}}.)
* jst has some ideas about using static analysis in place to automatically detect some kinds of leaks ({{bug|423032}}).
* Telemetry will help a lot. There would be 2 kinds of telemetry: built-in idle-daily stats and an addon. Builtin idle-daily would report a sanitized version of about:memory, number of open tabs, among other non-privacy-invasive stats. We would also distribute a telemetry addon allowing users to opt into more invasive telemetry. This addon would be able to run benchmarks, and allow the users to explicitly trigger submission of detailed privacy/perf-invasive information (such as open websites). ({{bug|663423}})
* Mozmill endurance tests can track memory usage over time, which is much more informative than snapshot measurements or averaged measurements. It's unclear how much overhead they introduce, however. Leveraging them somehow could avoid a lot of duplicated work.
== areweslimyet.com ==
