TestEngineering/Services/LoopTestEnvironments

Loop Server Production Environments

  • Loop-Server:
  • MSISDN Gateway:
  • Monitoring
    • NOTE: TBD
  • QA Access: not allowed
  • Pointing Fx Nightly to Production
    • NOTE: Fx Nightly is already set to point to Production by default
    • Loop-Server - TBD
    • MSISDN Gateway - TBD

Loop Server Stage Environment

Loop Server Stage Details

  • Versions
    • TBD
  • AWS
    • CF stack: loop-server-stage (or similar name)
    • ELB: loop-server-st-ELB-<some value>
    • One instance: look for "loop-server app server"
  • OS
    • Running SciLinux 6.4
  • Files
    • /data/loop-server
    • /data/hekad
    • /etc/circus.ini
    • /etc/circus.d
    • /etc/circus.d/loopserver.ini
    • /etc/circus.d/hekad-loop.ini
    • /etc/puppet/*
    • /var/log/*
  • Processes
    • node loop/index.js
    • stackdriver-collectd
    • circusd
    • heka.d
    • nginx
  • Logs
    • /var/log/circus.log
    • /var/log/loop_err.log
    • /var/log/loop_out.log
    • /var/log/hekad/loop.stdout.log
    • /var/log/hekad/loop.stderr.log
    • /var/log/nginx/access.log
    • /var/log/nginx/error.log
  • Heka
    • heka.d
    • /var/log/hekad/loop.stdout.log
    • /var/log/hekad/loop.stderr.log
  • Setting Stage configuration:
    • Toggling the config setting to use either a mock server or a third-party server
    • File: /data/loop-server/config/settings.json
  • Quick Verifications
curl https://loop.stage.mozaws.net
should return something similar to this
{"name":"mozilla-loop-server","description":"The Mozilla Loop (WebRTC App) server","version":"0.6.0-DEV","homepage":"https://github.com/mozilla-services/loop-server/","endpoint":"http://loop.stage.mozaws.net"}
curl -I https://loop.stage.mozaws.net
should return something similar to this
    HTTP/1.1 200 OK
    Content-length: 209
    Content-Type: application/json; charset=utf-8
    Date: Wed, 11 Jun 2014 23:01:11 GMT
    ETag: W/"d1-3663081557"
    Server: nginx/1.6.0
    Timestamp: 1402527671972
    Connection: keep-alive
In the browser: https://loop.stage.mozaws.net/
should return something similar to this
{"name":"mozilla-loop-server","description":"The Mozilla Loop (WebRTC App) server","version":"0.6.0-DEV","homepage":"https://github.com/mozilla-services/loop-server/","endpoint":"http://loop.stage.mozaws.net"}

Loop Client Stage Details

  • Versions
    • TBD
  • AWS
    • CF Stack: loop-client-stage2
    • ELB: loop-client-st-ELB-<some value>
    • One instance: look for "loop-client app server"
  • OS
    • Scientific Linux release 6.4
  • Files
    • /opt/aws
    • /opt/ec2
    • /opt/rh
    • /opt/stackdriver
    • /data/loop-client
  • Processes
    • TBD
  • Logs
    • /var/log/nginx
    • /media/ephemeral0 (empty)
  • Heka
    • TBD
  • Quick Verifications
In the browser: https://call.stage.mozaws.net
Should see this
    Loop
    Welcome to the Loop web client.
curl https://call.stage.mozaws.net
Returns a bunch of HTML
curl -I https://call.stage.mozaws.net
should return something similar to this
    HTTP/1.1 200 OK
    Accept-Ranges: bytes
    Content-length: 1935
    Content-Type: text/html
    Date: Wed, 11 Jun 2014 22:24:19 GMT
    ETag: "539220d7-78f"
    Last-Modified: Fri, 06 Jun 2014 20:13:11 GMT
    Server: nginx/1.6.0
    Connection: keep-alive
Check that Loop-Client Stage is talking to Loop-Server Stage
In the browser: https://call.stage.mozaws.net/config.js
should return something similar to this
var loop = loop || {};
loop.config = {serverUrl: 'https://https://loop.stage.mozaws.net'};

Mock Server Stage Details

    • NOTE: This server is used for load testing whenever we do not want to hit the third-party servers. A configuration change must be made in the Loop Server in order to make this work.
  • Versions
    • TBD
  • AWS
    • Loop-Server
    • MSISDN Gateway
    • Shared: TBD
  • OS
  • Files
  • Processes
  • Logs
  • Heka
  • Quick Verifications
In the browser: https://loop-delayed-response.stage.mozaws.net
Returns:
ok

MSISDN Gateway Server Stage Details

  • Versions
  • AWS
    • Loop-Server
    • MSISDN Gateway
    • Shared: TBD
  • OS
  • Files
  • Processes
  • Logs
  • Heka
  • Quick Verifications

QA Access via a Bastion Host

  • SSH with AWS keys to the Stage bastion host in US East 1. From there SSH directly into any instance using your LDAP name. For the mock server, log in as "ec2-user".

Stage Monitoring

Loads Services Cluster Environment

  • Versions
Loads Cluster/Broker/Agents:
$ cd /home/ubuntu/loads/bin
$ ./loads-runner --version
  • AWS in US West
    • loads-master (broker and agent processes)
    • loads-slave-1 (agent processes)
    • loads-slave-2 (agent processes)
    • NOTE: there is no stack or ELB for this cluster
  • Files
    • /home/ubuntu
      • loads
      • loads-aws
      • loads-web
  • Processes
    • Search for processes owned by ubuntu, loads, nginx, circus
  • Logs
    • /var/log/redis
    • /var/log/nginx
  • QA access
    • You need special access to be able to SSH into these devices
    • You need to make some changes to your .ssh/config file
  • Monitoring
  • Cluster status
    • Check from any loadtest folder or loads install:
../bin/loads-runner --ping-broker --ssh=ubuntu@loads.services.mozilla.com
../bin/loads-runner --check-cluster --ssh=ubuntu@loads.services.mozilla.com
(or similar)

References

OPs Mana and GitHub Pages