Confirmed users
487
edits
Services/TrackingProtection/Shavar Server - Testing: Difference between revisions
Jump to navigation
Jump to search
Services/TrackingProtection/Shavar Server - Testing (view source)
Revision as of 21:03, 17 December 2015
, 17 December 2015Updated for loadtesting strategy notes
Rpappalardo (talk | contribs) (Add loadtest notes) |
Rpappalardo (talk | contribs) (Updated for loadtesting strategy notes) |
||
| Line 248: | Line 248: | ||
'''SUMMARY''' | '''SUMMARY''' | ||
Going forward we will standardize our loadtesting of Shavar server using a fork of the [https://github.com/ckolos/beeswithmachineguns/beeswithmachineguns] repository which is optimized for our staging environment. | |||
[[#top]] | |||
'''INSTALLATION''' | |||
* | |||
* Clone the test tool repo: https://github.com/ckolos/beeswithmachineguns/beeswithmachineguns | |||
* Follow instructions on README for installation | |||
[[#top]] | |||
'''TEST STRATEGY''' | |||
In general, we break loadtesting strategy into 2 blocks: | |||
# Standalone instance test | |||
## Find max RPS (requests / second) that 1 EC2 instance can support at desired CPU capacity (and without throwing errors) | |||
## Result might be recorded as: "standalone instance capacity: X RPS @ Y% CPU" | |||
# ELB test | |||
## Setup auto-scaling parameters in AWS | |||
## Warm up ELB (see notes below) | |||
## Kickoff loadtest and allow it to run until scaled to desired number of instances | |||
## Result might be recorded as: "ELB capacity: X RPS across Y number of instances" (CPU percentage is implicit in auto-scaling parameters) | |||
## Also make a note of number of concurrent connections needed, 'bee'-size used etc. | |||
## For our purposes, we will use the largest (HVM instance) bee-size possible to ensure that network is optimized from the bee end (and won't risk bottle-necking there) | |||
NOTES: | |||
* EC2 instances must be "warmed-up" prior to a test run | |||
* This can be accomplished by throwing a nominal load against test stack for at least 5 minutes just prior to initiating a loadtest run. | |||
[[#top]] | |||
''' | '''EXAMPLE TEST RUN''' | ||
The following test will | The following test will: | ||
* instantiate -s [# of bees] | * instantiate -s [# of bees] | ||
* of -t [AWS instance size] | * of -t [AWS instance size] | ||
| Line 272: | Line 290: | ||
<code> | <code> | ||
$ cd beeswithmachineguns | |||
$ ./bees up -s 20 -k loads -t t2.medium | |||
</code> | </code> | ||
| Line 294: | Line 313: | ||
https://bugzilla.mozilla.org/show_bug.cgi?id=1204668#c6 | https://bugzilla.mozilla.org/show_bug.cgi?id=1204668#c6 | ||
[[#top]] | |||
'''TESTING WITH SEIGE''' | |||
In Cloud Services, we we'll only use beeswithmachineguns for consistency and optimization around our test environemnt. However, loadtesting of Shavar server can also be done using: [https://github.com/JoeDog/siege seige]. See the example below. | |||
The following test will: | |||
* create -c [# of concurrent users] | |||
* execute -r [# of repetitions (hits)] | |||
<code> | |||
siege -b -i -v -c 1000 -r 1000 'https://<shavar-stage-url-here>/downloads?client=LOADTEST&appver=36.0a1&pver=2.2 POST < blist.txt' | |||
</code> | |||
[[#top]] | [[#top]] | ||