User:Clouserw/AMO/loadtest

From MozillaWiki
Jump to: navigation, search

Command is something like : python replay_logs.py -t 252 --test=twisted -H addons.allizom.org -i "10.2.83.11" --port=81 -f ~/amologs/details.log -p "/z"

Zamboni is listed second

/ (frontpage)

Good: 5900
Max/Avg/Min/StdDev: 21402ms/9586ms/5ms/6066ms
Bad: 0
Total Time: 408.942674875
Req/s: 13.7060775125

Good: 4118
Max/Avg/Min/StdDev: 11198ms/4166ms/570ms/1776ms
Bad: 0
Total Time: 132.783806086
Req/s: 29.461423914

/addon/$id (details)

Good: 4999
Max/Avg/Min/StdDev: 14940ms/9217ms/1768ms/2558ms
Bad: 0
Total Time: 345.420506001
Req/s: 13.7484599713
Note: -t 252; NAMO

Good: 4998
Max/Avg/Min/StdDev: 12075ms/7939ms/1989ms/1648ms
Bad: 0
Total Time: 300.64410615
Req/s: 15.7927592887
Note: -t 252; PAMO

/addon/$id/about (addon-about.log)

Good: 207
Max/Avg/Min/StdDev: 9983ms/5512ms/1081ms/2513ms
Bad: 0
Total Time: 11.3381149769
Req/s: 17.2868241678

# The second time I ran this everything was cached and got 74r/s.  anyway, this page is low traffic
Good: 210
Max/Avg/Min/StdDev: 8394ms/5653ms/2027ms/2113ms
Bad: 0
Total Time: 9.33746385574
Req/s: 21.3119968199

/addon/$id/developers (addon-developers.log)

Good: 2167
Max/Avg/Min/StdDev: 23664ms/14544ms/2242ms/4091ms
Bad: 0
Total Time: 175.972939014
Req/s: 11.6949799868

Good: 2853
Max/Avg/Min/StdDev: 23472ms/12543ms/4011ms/3913ms
Bad: 43
Total Time: 159.714007139
Req/s: 16.9678292377

/addon/$id/developers/roadblock (developers-roadblock.log)

Good: 2279
Max/Avg/Min/StdDev: 15237ms/8535ms/749ms/3036ms
Bad: 0
Total Time: 147.318010092
Req/s: 14.6960985873

Good: 2273
Max/Avg/Min/StdDev: 7157ms/4756ms/690ms/1124ms
Bad: 17
Total Time: 80.1929559708
Req/s: 26.9225641313

/addon/$id/eula/$id (addon-eula)

Good: 2989
Max/Avg/Min/StdDev: 12957ms/7207ms/755ms/2546ms
Bad: 0
Total Time: 154.800768852
Req/s: 18.339702193

Good: 1597
Max/Avg/Min/StdDev: 7795ms/4285ms/272ms/1493ms
Bad: 26
Total Time: 38.6124460697
Req/s: 39.2878502766

/addon/$id/privacy (addon-privacy)

Good: 1779
Max/Avg/Min/StdDev: 12157ms/6556ms/649ms/2424ms
Bad: 0
Total Time: 84.1034400463
Req/s: 20.0943029092

Good: 1595
Max/Avg/Min/StdDev: 8646ms/5007ms/251ms/1556ms
Bad: 27
Total Time: 37.56189394
Req/s: 40.3334294703

/addon/$id/versions (addons-versions)

Good: 4075
Max/Avg/Min/StdDev: 16361ms/8808ms/557ms/3133ms
Bad: 0
Total Time: 280.404783964
Req/s: 13.8050426433

Good: 4525
Max/Avg/Min/StdDev: 7862ms/5878ms/816ms/1024ms
Bad: 52
Total Time: 223.31447196
Req/s: 19.2464015533

/api/$version/list/ (api_list)

Good: 13399
Max/Avg/Min/StdDev: 26851ms/14744ms/864ms/5256ms
Bad: 0
Total Time: 934.10345006
Req/s: 13.6269703309
Fri Apr 16 17:28:28 PDT 2010

Good: 55801
Max/Avg/Min/StdDev: 1126ms/573ms/68ms/142ms
Bad: 148
Total Time: 279.188308954
Req/s: 189.871847423
Fri Apr 16 19:45:21 PDT 2010

/blocklist/

# Bad looks like malformed requests to me
Good: 156673
Max/Avg/Min/StdDev: 169ms/36ms/6ms/43ms
Bad: 46
Total Time: 434.149641037
Req/s: 342.828798947

Good: 193290
Max/Avg/Min/StdDev: 1655ms/310ms/22ms/330ms
Bad: 60
Total Time: 440.036703825
Req/s: 417.294735652

Updated 2011-06-09
Good: 102902
Max/Avg/Min/StdDev: 239ms/54ms/6ms/56ms
Bad: 32
Total Time: 279.481642962
Req/s: 349.776103232

Good: 93713
Max/Avg/Min/StdDev: 2189ms/289ms/44ms/337ms
Bad: 31
Total Time: 197.444929123
Req/s: 450.89534786

/collections/

# Remora is unknown, but zamboni is fast

ab -c 150 -n 600 http://preview.addons.mozilla.org:81/en-US/firefox/collections/
Requests per second:    38.63 [#/sec] (mean)

/collections/$name/$collection

A weak spot test because log_replay doesn't follow 301s. 

ab -c 150 -n 600 http://preview.addons.mozilla.org:81/en-US/firefox/collection/enkei
Requests per second:    16.98 [#/sec] (mean)

ab -c 150 -n 600 http://preview.addons.mozilla.org:81/z/en-US/firefox/collection/enkei/enkei/
Requests per second:    49.98 [#/sec] (mean)

/extensions

ab -c 150 -n 600 -H 'Host: preview.addons.mozilla.org' "http://10.2.83.11:81/en-US/firefox/browse/type:1/cat:all?sort=popular" 
Requests per second:    8.12 [#/sec] (mean)

ab -c 150 -n 600 -H 'Host: preview.addons.mozilla.org' "http://10.2.83.11:81/z/en-US/firefox/extensions/"
Requests per second:    44.09 [#/sec] (mean)

/extensions/$slug

ab -c 150 -n 600 -H 'Host: preview.addons.mozilla.org' "http://10.2.83.11:81/en-US/firefox/browse/type:1/cat:22"
Requests per second:    8.51 [#/sec] (mean)

ab -c 150 -n 600 -H 'Host: preview.addons.mozilla.org' "http://10.2.83.11:81/en-US/firefox/browse/type:1/cat:73"
Requests per second:    5.68 [#/sec] (mean)

ab -c 150 -n 600 -H 'Host: preview.addons.mozilla.org' "http://10.2.83.11:81/en-US/firefox/browse/type:2/cat:21"
Requests per second:    15.16 [#/sec] (mean)

ab -c 150 -n 600 -H 'Host: preview.addons.mozilla.org' "http://10.2.83.11:81/en-US/thunderbird/browse/type:2/cat:60?sort=name"
Requests per second:    16.21 [#/sec] (mean)



ab -c 150 -n 600 -H 'Host: preview.addons.mozilla.org' "http://10.2.83.11:81/z/en-US/firefox/extensions/bookmarks"
Requests per second:    31.11 [#/sec] (mean)

ab -c 150 -n 600 -H 'Host: preview.addons.mozilla.org' "http://10.2.83.11:81/z/en-US/firefox/extensions/other"
Requests per second:    34.20 [#/sec] (mean)

ab -c 150 -n 600 -H 'Host: preview.addons.mozilla.org' "http://10.2.83.11:81/z/en-US/firefox/themes/miscellaneous"
Requests per second:    47.15 [#/sec] (mean)

ab -c 150 -n 600 -H 'Host: preview.addons.mozilla.org' "http://10.2.83.11:81/z/en-US/thunderbird/themes/miscellaneous?sort=downloads"
Requests per second:    68.66 [#/sec] (mean)

/extensions/$slug/featured

# N/A on Remora

$ ab -c 150 -n 600 -H 'Host: preview.addons.mozilla.org' http://10.2.83.11:81/z/en-US/firefox/extensions/bookmarks/featured
Requests per second:    44.18 [#/sec] (mean)

$ ab -c 150 -n 600 -H 'Host: preview.addons.mozilla.org' http://10.2.83.11:81/z/en-US/firefox/extensions/privacy-security/featured
Requests per second:    58.80 [#/sec] (mean)
 
$ ab -c 150 -n 600 -H 'Host: preview.addons.mozilla.org' http://10.2.83.11:81/z/en-US/firefox/extensions/other/featured
Requests per second:    92.32 [#/sec] (mean)

/featured

$ ab -c 150 -n 600 "http://preview.addons.mozilla.org:81/en-US/firefox/recommended"
Requests per second:    10.61 [#/sec] (mean)

ab -c 150 -n 600 "http://preview.addons.mozilla.org:81/z/en-US/firefox/featured"
Requests per second:    46.94 [#/sec] (mean)

/language-tools

ab -c 150 -n 600 -H 'Host: preview.addons.mozilla.org' "http://10.2.83.11:81/en-US/firefox/browse/type:3" 
Requests per second:    5.05 [#/sec] (mean)

ab -c 150 -n 600 -H 'Host: preview.addons.mozilla.org' "http://10.2.83.11:81/z/en-US/firefox/language-tools/" 
Requests per second:    84.57 [#/sec] (mean)

/pages/about

ab -c 252 -n 2500 -H 'Host: preview.addons.mozilla.org' http://10.2.83.11:81/en-US/firefox/about
Requests per second:    32.37 [#/sec] (mean)

ab -c 252 -n 2500 -H 'Host: preview.addons.mozilla.org' http://10.2.83.11:81/z/en-US/firefox/about
Requests per second:    107.17 [#/sec] (mean)

/pages/faq

ab -c 150 -n 600 -H 'Host: preview.addons.mozilla.org' "http://10.2.83.11:81/en-US/firefox/pages/faq"
Requests per second:    27.59 [#/sec] (mean)

ab -c 150 -n 600 -H 'Host: preview.addons.mozilla.org' "http://10.2.83.11:81/z/en-US/firefox/faq"
Requests per second:    100.40 [#/sec] (mean)

/personas/$category (personas_category_landing)

Good: 2007
Max/Avg/Min/StdDev: 30557ms/16921ms/1948ms/5763ms
Bad: 1
Total Time: 263.24000001
Req/s: 7.24054095096

# This started out very slow, but got a lot faster over time.  The bad here is bug 567779
Good: 6041
Max/Avg/Min/StdDev: 5744ms/3116ms/1321ms/889ms
Bad: 854
Total Time: 228.346203089
Req/s: 25.1285106666


/personas/*

python scripts/siege.py http://10.2.83.11:81/z personas{,-sort} > personas-siege.txt
siege -c50 -r150 -i -f personas-siege.txt

Transactions:                   7050 hits
Availability:                  94.00 %
Elapsed time:                 459.62 secs
Data transferred:             398.07 MB
Response time:                  2.48 secs
Transaction rate:              15.34 trans/sec
Throughput:                     0.87 MB/sec
Concurrency:                   38.06
Successful transactions:        7479
Failed transactions:             450
Longest transaction:           29.80
Shortest transaction:           0.00

Transactions:                   7444 hits
Availability:                  99.25 %
Elapsed time:                 367.20 secs
Data transferred:             501.85 MB
Response time:                  1.57 secs
Transaction rate:              20.27 trans/sec
Throughput:                     1.37 MB/sec
Concurrency:                   31.76
Successful transactions:        7444
Failed transactions:              56
Longest transaction:           29.97
Shortest transaction:           0.07

/search

Good: 633
Max/Avg/Min/StdDev: 30705ms/15887ms/780ms/7139ms
Bad: 0
Total Time: 90.8286411762
Req/s: 6.61685556689

Good: 3267
Max/Avg/Min/StdDev: 19493ms/13767ms/5966ms/2439ms
Bad: 4
Total Time: 375.414680004
Req/s: 8.26552653712

New run:
Req/s: 11.3

/search-tools

ab -c 150 -n 600  "http://preview.addons.mozilla.org:81/en-US/firefox/browse/type:4"
Requests per second:    17.92 [#/sec] (mean)

ab -c 150 -n 600  "http://preview.addons.mozilla.org:81/z/en-US/firefox/search-tools/"
Requests per second:    66.02 [#/sec] (mean)

(after bug 605665)
ab -c 150 -n 600  "http://addons.allizom.org:81/en-US/firefox/search-tools/"
Requests per second:    31.77 [#/sec] (mean)

/themes

ab -c 150 -n 600 -H 'Host: preview.addons.mozilla.org' "http://10.2.83.11:81/en-US/firefox/browse/type:2/cat:all?sort=popular" 
Requests per second:    17.40 [#/sec] (mean)

ab -c 150 -n 600 -H 'Host: preview.addons.mozilla.org' "http://10.2.83.11:81/z/en-US/firefox/themes/"
Requests per second:    48.33 [#/sec] (mean)

/update (update.*.log) bug 624095

replay_logs.py -t252 --test=twisted -H addons.allizom.org -i "10.2.83.11" --port=81 -f ~/amologs/update.z.log -p "/z/services"

Good: 70543
Max/Avg/Min/StdDev: 825ms/197ms/48ms/186ms
Bad: 0
Total Time: 121.608709812
Req/s: 551.070725966

Good: 17151
Max/Avg/Min/StdDev: 4336ms/1936ms/557ms/786ms
Bad: 0
Total Time: 144.322224855
Req/s: 112.893215278

# Using full SQL in Django with max-requests at 1000:
Good: 45209
Max/Avg/Min/StdDev: 2039ms/808ms/294ms/357ms
Bad: 0
Total Time: 204.310770988
Req/s: 210.209181788

/user/$id (user_detail)

Good: 2533
Max/Avg/Min/StdDev: 16151ms/8824ms/690ms/3204ms
Bad: 0
Total Time: 172.725419044
Req/s: 13.9296231748

# Bad here is because remora doesn't report "User not found" as a 404
Good: 5270
Max/Avg/Min/StdDev: 9576ms/6159ms/471ms/1253ms
Bad: 247
Total Time: 241.375447989
Req/s: 20.7394747134
Retrieved from "https://wiki.mozilla.org/index.php?title=User:Clouserw/AMO/loadtest&oldid=317491"