Auto-tools/Projects/AddonStartupPerf: Difference between revisions

← Older edit

Auto-tools/Projects/AddonStartupPerf (view source)

Revision as of 17:58, 4 January 2012

1,226 bytes added , 4 January 2012

→‎results

Jgriffin

Confirmed users

1,905

edits

@@ Line 1: / Line 1: @@
-==Goal==
+= Current Manual Testing =
-Get addon startup impact (and in the future, general performance and unittest failures) automated and reporting on a regular basis.
-=Pieces=
+== code ==
-* addon fetcher - look at feed of popular addons and download latest versions to put in db
-* profile generator - create clean profile with given set of addons/preferences
+https://github.com/jonallengriffin/dirtyharry
-* runner - firefox runner than can receive results asynchronously
+== results ==
+Top 500 addons
+'''Linux:'''
+[http://github.com/jonallengriffin/dirtyharry/blob/master/results/results_sorted_linux0.csv sorted average Talos Ts]
+[http://github.com/jonallengriffin/dirtyharry/blob/master/results/raw_results_linux0.csv Talos Ts]
+= Automated Testing =
+== Limitations  ==
+*no plans to allow addons to 'call home' - we will still be working in the talos testing environment where we are proxied to localhost, so there will be no live web interaction
+*no current plans to interact with the addon (no clicks, no visiting specific pages)
+**this sort of perf test would have to be designed/built per-addon to get the most bang for the buck
+== Plans  ==
+*integrate into buildbot and have run on production machines
+**how frequently?
+**which tests?
+***for now, we'll limit to clean ts starts with just the addon installed
+**where would the list of addons be maintained?
+**where do we download the addons from?
+*results reported as .csv files
+**where should these be sent?
+**do we want the data on the graph server?
+**how do we compare the results?
+= Information for Add-on Authors =
+*How to test for performance differences yourselves: [[Firefox/Projects/StartupPerformance/MeasuringStartup|Measuring Startup]].
+*Authors of add-ons with the greatest performance impact are being contacted about this, almost always with suggestions on where to improve.
+* [http://blog.mozilla.com/addons/2010/06/14/improve-extension-startup-performance/ How to Improve Extension Startup Performance].