Loop/Test/Loadtesting
Jump to navigation
Jump to search
Overview
Summary
Loadtesting Loop is currently performed with loadsv1 tool. Loadsv2 is under development (stay tuned for more info...)
Loadtesting should be performed against
- loop-server
- msisdn-gateway
It's not necessary to loadtest against loop-client
Guidelines for loadtesting with loadsv1 tool are provided below.
Loadsv1
Installing Loop-Server and the Loads tool (on Localhost or AWS)
- refer to github README
Installation
loop-server
$ git clone https://github.com/mozilla-services/loop-server $ cd loop-server/loadtests </pred> $ make install
msisdn-gateway
$ git clone https://github.com/mozilla-services/loop-server $ cd msisdn-gateway/loadtests </pred> $ make install
Pre-requisite
- edit loadtest config file
- enter desired values for current test
$ vim /loadtests/config/megabench.ini
Run Test There are 3 commands available for testing with loadsv1:
$ make megabench$ make test$ make bench(use megabench instead)
MSISDN - loadtesting
There are 2 staging environments for msisdn-gateway
- msisdn.stage.mozaws.net - for end-2-end testing (NOT for loadtesting!)
- msisdn-loadtest.stage.mozaws.net - for loadtesting
- This environment also contains its own mock server: http://omxen.dev.mozaws.net
see:
configuration file on the Stage server: /data/msisdn-gateway/config/production.json
Test Execution
Summary
Of the three tests available, make megabench will probably be the one you use most.
make test is largely for a quick smoke test and make bench doesn't do anything you can't do with the other two.
For information about test monitoring:
make megabench
In Terminal $ make megabench
./venv/bin/loads-runner --config=./config/megabench.ini --user-id=bonzogonzo --server-url=https://loop.stage.mozaws.net:443 loadtest.TestLoop.test_all [2015-05-21 16:00:21,868][11285] Detached. run --attach to reattach
'Open the following URL in browser:'
https://loads.mozilla.services.com
- Login with credentials provided by Loop dev/QA team
- You will see real-time output from your test run in your browser
make test
Description
Running $ make test allows you to kick off a loadtest locally against a remote endpoint, but then receive test output locally.
Optional: if you want to specify a test endpoint other than the default (see: Makefile), you can do it like so:
$ make test SERVER_URL=https://loop.stage.mozaws.net - or - $ make test SERVER_URL=https://msisdn.stage.mozaws.net
WARNING: Do not launch loadtests against any PRODUCTION server, ever!
Example Output
$ make test SERVER_URL=https://loop.stage.mozaws.net SIMPLE_PUSH_URL=https://call.stage.mozaws.net/ ./venv/bin/loads-runner --config=./config/test.ini --server-url=https://loop.stage.mozaws.net loadtest.TestLoop.test_http_calls [ ] 0%/Users/rpappalardo/git/services-test/build/loop-server/loadtests/venv/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning. InsecurePlatformWarning [=================================================================================================================================================================================================================================] 100% Duration: 4.99 seconds Hits: 17 Started: 2015-05-21 22:26:10.930852 Approximate Average RPS: 3 Average request time: 0.28s Opened web sockets: 0 Bytes received via web sockets : 0 Success: 1 Errors: 0 Failures: 0 Slowest URL: https://loop.stage.mozaws.net/ Average Request Time: 1.121348 Stats by URLs: - https://loop.stage.mozaws.net/ Average request time: 1.121348 Hits success rate: 1.0 - https://loop.stage.mozaws.net/calls/Dko0_Uvdks8 Average request time: 0.689918 Hits success rate: 1.0 - https://loop.stage.mozaws.net/rooms Average request time: 0.429905 Hits success rate: 1.0 - https://loop.stage.mozaws.net/rooms/C4lqDXc6W3M Average request time: 0.189937375 Hits success rate: 1.0 - https://loop.stage.mozaws.net/registration Average request time: 0.16524675 Hits success rate: 1.0 - https://loop.stage.mozaws.net/call-url Average request time: 0.163699 Hits success rate: 1.0 - https://loop.stage.mozaws.net/calls?version=200 Average request time: 0.163517 Hits success rate: 1.0 Custom metrics: - delete-room : 1 - register : 4 - generate-call-url : 1 - refresh-room-presence : 1 - initiate-call : 1 - leave-room : 3 - room-with-context : 1 - join-room : 3 - create-room : 1 - list-pending-calls : 1 - num-participants-3 : 1 SIMPLE_PUSH_URL=https://call.stage.mozaws.net/ ./venv/bin/loads-runner --config=./config/test.ini --server-url=https://loop.stage.mozaws.net loadtest.TestLoop.test_websockets [ ] 0%/Users/rpappalardo/git/services-test/build/loop-server/loadtests/venv/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning. InsecurePlatformWarning [ ] 0%e [=================================================================================================================================================================================================================================] 100% Duration: 81.80 seconds Hits: 17 Started: 2015-05-21 22:26:16.133047 Approximate Average RPS: 0 Average request time: 0.24s Opened web sockets: 7 Bytes received via web sockets : 1350 Success: 1 Errors: 0 Failures: 0 Slowest URL: https://loop.stage.mozaws.net/ Average Request Time: 0.713177 Stats by URLs: - https://loop.stage.mozaws.net/ Average request time: 0.713177 Hits success rate: 1.0 - https://loop.stage.mozaws.net/calls/KR414mzrf-s Average request time: 0.359527 Hits success rate: 1.0 - https://loop.stage.mozaws.net/calls/bYhXDs86FTQ Average request time: 0.358428 Hits success rate: 1.0 - https://loop.stage.mozaws.net/calls/vqp9-BYyYBQ Average request time: 0.357578 Hits success rate: 1.0 - https://loop.stage.mozaws.net/calls/Nbi4t9_pLAc Average request time: 0.356943 Hits success rate: 1.0 - https://loop.stage.mozaws.net/registration Average request time: 0.1638505 Hits success rate: 1.0 - https://loop.stage.mozaws.net/call-url Average request time: 0.16342975 Hits success rate: 1.0 - https://loop.stage.mozaws.net/calls?version=200 Average request time: 0.16315475 Hits success rate: 1.0 Custom metrics: - websocket-basic-callee-messages : 5 - websocket-supervisory-caller-messages : 2 - list-pending-calls : 4 - generate-call-url : 4 - initiate-call : 4 - websocket-ringing-callee-messages : 3 - websocket-basic-caller-messages : 5 - websocket-ringing-caller-messages : 3 - register : 4
Loadtest - Tweaking
loadtests can be configured in the code...
- Test server URL
- Error percentages
Reference
Monitoring
Ops / Infra
- https://github.com/mozilla-services/puppet-config/issues
- https://github.com/mozilla-services/svcops/issues