Canmove, confirm
6,438
edits
Changes
→ELB Logs
The ELB logs for the public-facing application are replicated to the '''balrog-us-west-2-elb-logs''' S3 bucket, located in us-west-2. Logs are rotated very quickly, and we end up with tens of thousands of separate files each day. Because of this, and the fact that S3 has a lot of overhead per-file, it can be tricky to do analysis on them. You're unlikely to be able to download the logs locally in any reasonable amount of time (ie, less than a day), but mounting them on an EC2 instance in us-west-2 should provide you with reasonably quick access. Here's an example:
* Launch EC2 instance (you probably a compute-optimized one, and at least 100GB of storage).
* Generate an access token for your CloudOps AWS account. If you don't have a CloudOps AWS account, talk to Ben Hearsum or Bensong Wong. Put the token in a plaintext file somewher somewhere on the instance.
** If you've chosen local storage, you'll probably need to format and mount volume.
* Install s3fs by following the instructions on https://github.com/s3fs-fuse/s3fs-fuse.
