TestEngineering/Services/FxATestEnvironments

From MozillaWiki
Jump to navigation Jump to search

FxA, TokenServer, and Sync Production Environments

  • QA Access: not allowed

FxA Stage Environment

  • AWS
    • New FxA-shared services:
      • svcops-fxa-stage-heka
      • svcops-fxa-stage-amqp
      • svcops-fxa-stage-elasticsearch
      • svcops-fxa-stage-asg-sns-topic
    • Other shared Stage Services:
      • svcops-stage-heka (combined heka+kibana behind an ELB)
      • svcops-stage-amqp (shared RabbitMQ cluster)
      • svcops-stage-elasticsearch ( Elasticsearch cluster)
      • svcops-stage-asg-sns-topic
      • svcops-stage-bastion-servers (Bastion hosts for Stage)
    • Load Balancers:
      • ContentServerELB
      • AuthServerELB
      • BastionELB
      • BastionExternalFQDN
    • DB Instances (RDS):
      • RDSInstance = fxa-rds-stage
    • All instances: Search in AWS Console using the Env column to find "stage"
      • shared-elasticsearch
      • shared-elasticsearch
      • shared-elasticsearch
      • shared-elasticsearch
      • shared-rabbitmq
      • shared-rabbitmq
      • shared-bastion
      • shared-heka
      • fxa-elasticsearch
      • fxa-rabbitmq
      • fxa-heka
      • fxa-logbox
      • fxa-auth (2 instances, m3.large)
      • fxa-content_server (1 small instance)
  • Files
    • /data
    • /etc/puppet
  • Processes
    • fxa-auth-server:
      • Look for processes owned by app, nginx, circus
    • fxa-content-server:
      • Look for processes owned by node, heka, nginx, circus
  • Logs
    • /var/log/nginx
    • /var/log/circus.log
    • fxa-auth-server
      • /media/ephemeral0/fxa-auth-server/auth_err.log.*
      • /media/ephemeral0/fxa-auth-server/auth_out.log
      • /media/ephemeral0/heka/hekad_err.log
      • /media/ephemeral0/heka/hekad_out.log
      • /media/ephemeral0/nginx/logs/access.log
      • /media/ephemeral0/nginx/logs/error.log
    • fxa-content-server
      • /media/ephemeral0/fxa-content-server/content_err.log
      • /media/ephemeral0/fxa-content-server/content_out.log
      • /media/ephemeral0/heka/hekad_err.log
      • /media/ephemeral0/heka/hekad_out.log
      • /media/ephemeral0/nginx/access.log
      • /media/ephemeral0/nginx/logs/error.log
  • Log Aggregation
    • Look at /data/hekad/hekad.toml
    • Check here: [aggregator-output]
    • hekad --> rabbitmq --> elasticsearch --> kibana (since Kibana is the UI on top of ES)
fxa-auth-server.json:
    defaults for "Environment", "AuthServerInstanceType", and "AuthServerDesiredCapacity"
fxa-content-server.json:
    defaults for "Environment", "InstanceType", and "NumServers"
fxa-db-rds.json:
    defaults for "Environment" and "DBInstanceType"
Compare this to what is actually live in AWS (via the Console or awsboxen)
  • QA Access via a Bastion Host
    • SSH with AWS keys to the Stage bastion host in US East 1. From there SSH directly into any instance.

TokenServer Stage Environment

  • Versions:
TokenServer:
rpm -qa | grep token
Example: tokenserver-svcops <latest version>

Verifier:
rpm -qa | grep verifier
Example: fxa-browserid-verifier-svcops <latest version>

Sync 1.5 Stage Environment

  • Versions
Server-Syncstorage (sync node):
rpm -qa | grep syncstorage
Example: server-syncstorage <latest version>
  • Note: There is no longer a Sync 1.1 Stage environment.

FxA Load Test Environment

Loads Services Cluster Environment

  • Versions
Loads Cluster/Broker/Agents:
$ cd /home/ubuntu/loads/bin
$ ./loads-runner --version
  • AWS in US West
    • loads-master (broker and agent processes)
    • loads-slave-1 (agent processes)
    • loads-slave-2 (agent processes)
    • NOTE: there is no stack or ELB for this cluster
  • Files
    • /home/ubuntu
      • loads
      • loads-aws
      • loads-web
  • Processes
    • Search for processes owned by ubuntu, loads, nginx, circus
  • Logs
    • /var/log/redis
    • /var/log/nginx
  • QA access
    • You need special access to be able to SSH into these devices
    • You need to make some changes to your .ssh/config file
  • Monitoring
  • Cluster status
    • Check from any loadtest folder or loads install:
../bin/loads-runner --ping-broker --ssh=ubuntu@loads.services.mozilla.com
../bin/loads-runner --check-cluster --ssh=ubuntu@loads.services.mozilla.com
(or similar)

TokenServer and Sync 1.5 Dev Environments

  • Versions
TokenServer:
rpm -qa | grep token
Example: tokenserver-svcops <latest version>

Verifier:
rpm -qa | grep verifier
Example: fxa-browserid-verifier-svcops <latest version>

Server-Syncstorage (sync node):
rpm -qa | grep syncstorage
Example: server-syncstorage <latest version>
  • AWS
    • This is part of the dev-lcip-org CloudFormation stack
      • TokenServer: Search for the fxa-tokenserver instance
      • SyncServer2: Search for the fxa-syncstorage instance
      • SyncServer1: Search for the fxa-syncstorage instance
      • LogServer: Search for the fxa-logbox instance
  • Files
    • TBD
  • Processes
    • TBD
  • Logs
    • TBD
  • QA Access
    • SSH with AWS keys to the various instances
  • Firefox configs:

FxA Development Environments - partial

  • QA Access
    • SSH with AWS keys to the various instances


  • Other Dev Deployments:
    • AWS: api-accounts-legacy.dev.lcip.org
    • AWS: accounts-legacy.dev.lcip.org
    • AWS: api-accounts-onepw.dev.lcip.org

OPs Mana and GitHub Pages

Retrieved from "https://wiki.mozilla.org/index.php?title=TestEngineering/Services/FxATestEnvironments&oldid=988661"