Loop/Test/Loadtesting: Difference between revisions
Jump to navigation
Jump to search
Rpappalardo (talk | contribs) (monitoring section) |
Rpappalardo (talk | contribs) m (formatting) |
||
| (2 intermediate revisions by the same user not shown) | |||
| Line 77: | Line 77: | ||
Of the three tests available, <code>make megabench</code> will probably be the one you use most. | Of the three tests available, <code>make megabench</code> will probably be the one you use most. | ||
<code>make test</code> is largely for a quick smoke test and <code>make bench</code> doesn't do anything you can't do with the other two. | <code>make test</code> is largely for a quick smoke test and <code>make bench</code> doesn't do anything you can't do with the other two. | ||
For information about test monitoring: | |||
[[Loop/Test/Monitoring]] | |||
| Line 203: | Line 207: | ||
- register : 4 | - register : 4 | ||
</pre> | </pre> | ||
| Line 448: | Line 225: | ||
= Reference = | = Reference = | ||
== Monitoring == | |||
* [[Loop/Test/Monitoring]] | |||
| Line 456: | Line 236: | ||
== Loads Tool and Cluster == | == Loads Tool and Cluster == | ||
* http://loads.services.mozilla.com | |||
* https://loads.readthedocs.org/ | |||
* https://github.com/mozilla-services/loads/issues | |||
* https://github.com/mozilla-services/loads-web/issues | |||
* https://wiki.mozilla.org/QA/Services/LoadsV1ClientTestHost | * https://wiki.mozilla.org/QA/Services/LoadsV1ClientTestHost | ||
* https://wiki.mozilla.org/QA/Services/LoadsToolsAndTesting1 | * https://wiki.mozilla.org/QA/Services/LoadsToolsAndTesting1 | ||
Latest revision as of 16:15, 22 May 2015
Overview
Summary
Loadtesting Loop is currently performed with loadsv1 tool. Loadsv2 is under development (stay tuned for more info...)
Loadtesting should be performed against
- loop-server
- msisdn-gateway
It's not necessary to loadtest against loop-client
Guidelines for loadtesting with loadsv1 tool are provided below.
Loadsv1
Installing Loop-Server and the Loads tool (on Localhost or AWS)
- refer to github README
Installation
loop-server
$ git clone https://github.com/mozilla-services/loop-server $ cd loop-server/loadtests </pred> $ make install
msisdn-gateway
$ git clone https://github.com/mozilla-services/loop-server $ cd msisdn-gateway/loadtests </pred> $ make install
Pre-requisite
- edit loadtest config file
- enter desired values for current test
$ vim /loadtests/config/megabench.ini
Run Test There are 3 commands available for testing with loadsv1:
$ make megabench$ make test$ make bench(use megabench instead)
MSISDN - loadtesting
There are 2 staging environments for msisdn-gateway
- msisdn.stage.mozaws.net - for end-2-end testing (NOT for loadtesting!)
- msisdn-loadtest.stage.mozaws.net - for loadtesting
- This environment also contains its own mock server: http://omxen.dev.mozaws.net
see:
configuration file on the Stage server: /data/msisdn-gateway/config/production.json
Test Execution
Summary
Of the three tests available, make megabench will probably be the one you use most.
make test is largely for a quick smoke test and make bench doesn't do anything you can't do with the other two.
For information about test monitoring:
make megabench
In Terminal $ make megabench
./venv/bin/loads-runner --config=./config/megabench.ini --user-id=bonzogonzo --server-url=https://loop.stage.mozaws.net:443 loadtest.TestLoop.test_all [2015-05-21 16:00:21,868][11285] Detached. run --attach to reattach
'Open the following URL in browser:'
https://loads.mozilla.services.com
- Login with credentials provided by Loop dev/QA team
- You will see real-time output from your test run in your browser
make test
Description
Running $ make test allows you to kick off a loadtest locally against a remote endpoint, but then receive test output locally.
Optional: if you want to specify a test endpoint other than the default (see: Makefile), you can do it like so:
$ make test SERVER_URL=https://loop.stage.mozaws.net - or - $ make test SERVER_URL=https://msisdn.stage.mozaws.net
WARNING: Do not launch loadtests against any PRODUCTION server, ever!
Example Output
$ make test SERVER_URL=https://loop.stage.mozaws.net SIMPLE_PUSH_URL=https://call.stage.mozaws.net/ ./venv/bin/loads-runner --config=./config/test.ini --server-url=https://loop.stage.mozaws.net loadtest.TestLoop.test_http_calls [ ] 0%/Users/rpappalardo/git/services-test/build/loop-server/loadtests/venv/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning. InsecurePlatformWarning [=================================================================================================================================================================================================================================] 100% Duration: 4.99 seconds Hits: 17 Started: 2015-05-21 22:26:10.930852 Approximate Average RPS: 3 Average request time: 0.28s Opened web sockets: 0 Bytes received via web sockets : 0 Success: 1 Errors: 0 Failures: 0 Slowest URL: https://loop.stage.mozaws.net/ Average Request Time: 1.121348 Stats by URLs: - https://loop.stage.mozaws.net/ Average request time: 1.121348 Hits success rate: 1.0 - https://loop.stage.mozaws.net/calls/Dko0_Uvdks8 Average request time: 0.689918 Hits success rate: 1.0 - https://loop.stage.mozaws.net/rooms Average request time: 0.429905 Hits success rate: 1.0 - https://loop.stage.mozaws.net/rooms/C4lqDXc6W3M Average request time: 0.189937375 Hits success rate: 1.0 - https://loop.stage.mozaws.net/registration Average request time: 0.16524675 Hits success rate: 1.0 - https://loop.stage.mozaws.net/call-url Average request time: 0.163699 Hits success rate: 1.0 - https://loop.stage.mozaws.net/calls?version=200 Average request time: 0.163517 Hits success rate: 1.0 Custom metrics: - delete-room : 1 - register : 4 - generate-call-url : 1 - refresh-room-presence : 1 - initiate-call : 1 - leave-room : 3 - room-with-context : 1 - join-room : 3 - create-room : 1 - list-pending-calls : 1 - num-participants-3 : 1 SIMPLE_PUSH_URL=https://call.stage.mozaws.net/ ./venv/bin/loads-runner --config=./config/test.ini --server-url=https://loop.stage.mozaws.net loadtest.TestLoop.test_websockets [ ] 0%/Users/rpappalardo/git/services-test/build/loop-server/loadtests/venv/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning. InsecurePlatformWarning [ ] 0%e [=================================================================================================================================================================================================================================] 100% Duration: 81.80 seconds Hits: 17 Started: 2015-05-21 22:26:16.133047 Approximate Average RPS: 0 Average request time: 0.24s Opened web sockets: 7 Bytes received via web sockets : 1350 Success: 1 Errors: 0 Failures: 0 Slowest URL: https://loop.stage.mozaws.net/ Average Request Time: 0.713177 Stats by URLs: - https://loop.stage.mozaws.net/ Average request time: 0.713177 Hits success rate: 1.0 - https://loop.stage.mozaws.net/calls/KR414mzrf-s Average request time: 0.359527 Hits success rate: 1.0 - https://loop.stage.mozaws.net/calls/bYhXDs86FTQ Average request time: 0.358428 Hits success rate: 1.0 - https://loop.stage.mozaws.net/calls/vqp9-BYyYBQ Average request time: 0.357578 Hits success rate: 1.0 - https://loop.stage.mozaws.net/calls/Nbi4t9_pLAc Average request time: 0.356943 Hits success rate: 1.0 - https://loop.stage.mozaws.net/registration Average request time: 0.1638505 Hits success rate: 1.0 - https://loop.stage.mozaws.net/call-url Average request time: 0.16342975 Hits success rate: 1.0 - https://loop.stage.mozaws.net/calls?version=200 Average request time: 0.16315475 Hits success rate: 1.0 Custom metrics: - websocket-basic-callee-messages : 5 - websocket-supervisory-caller-messages : 2 - list-pending-calls : 4 - generate-call-url : 4 - initiate-call : 4 - websocket-ringing-callee-messages : 3 - websocket-basic-caller-messages : 5 - websocket-ringing-caller-messages : 3 - register : 4
Loadtest - Tweaking
loadtests can be configured in the code...
- Test server URL
- Error percentages
Reference
Monitoring
Ops / Infra
- https://github.com/mozilla-services/puppet-config/issues
- https://github.com/mozilla-services/svcops/issues