Latest revision as of 17:58, 4 January 2012

Current Manual Testing

Top 500 addons

Linux:

sorted average Talos Ts

no plans to allow addons to 'call home' - we will still be working in the talos testing environment where we are proxied to localhost, so there will be no live web interaction
no current plans to interact with the addon (no clicks, no visiting specific pages)
- this sort of perf test would have to be designed/built per-addon to get the most bang for the buck

integrate into buildbot and have run on production machines
- how frequently?
- which tests?
  - for now, we'll limit to clean ts starts with just the addon installed
- where would the list of addons be maintained?
- where do we download the addons from?
results reported as .csv files
- where should these be sent?
- do we want the data on the graph server?
- how do we compare the results?

How to test for performance differences yourselves: Measuring Startup.
Authors of add-ons with the greatest performance impact are being contacted about this, almost always with suggestions on where to improve.
How to Improve Extension Startup Performance.

@@ Line 11: / Line 11: @@
 '''Linux:'''
-[http://github.com/jonallengriffin/blob/master/results/results_sorted_linux0.csv sorted average Talos Ts]
+[http://github.com/jonallengriffin/dirtyharry/blob/master/results/results_sorted_linux0.csv sorted average Talos Ts]
-[http://github.com/jonallengriffin/blob/master/results/raw_results_linux0.csv Talos Ts]
+[http://github.com/jonallengriffin/dirtyharry/blob/master/results/raw_results_linux0.csv Talos Ts]
 = Automated Testing =