48
edits
Auto-tools/Projects/EC2Builder: Difference between revisions
no edit summary
No edit summary |
No edit summary |
||
| Line 30: | Line 30: | ||
== About EC2 == | == About EC2 == | ||
EC2 is Amazon's cloud computing service. Most commonly used for site hosting, there is a separate service called Elastic Load Balancer that provides hooks for spawning new servers under load <br> | EC2 is Amazon's cloud computing service. Most commonly used for site hosting, there is a separate service called Elastic Load Balancer that provides hooks for spawning new servers under load <br> | ||
E.g | E.g | ||
CPU > x% would trigger new virtual server | CPU > x% would trigger new virtual server | ||
CPU < x% would turn off virtual certain servers | CPU < x% would turn off virtual certain servers | ||
Since we're using it to do builds, we thought about maybe using elastic load balancer (builds usually put the system at CPU close to 100%) but it seemed more effective to spin up 1 server per build and to queue excess jobs (limit active servers to some number, like 5, for cost reasons). Amazon provides a Queuing service as well. <br> | Since we're using it to do builds, we thought about maybe using elastic load balancer (builds usually put the system at CPU close to 100%) but it seemed more effective to spin up 1 server per build and to queue excess jobs (limit active servers to some number, like 5, for cost reasons). Amazon provides a Queuing service as well. <br> | ||
== EBS-backed Instances vs S3-backed Instances == | == EBS-backed Instances vs S3-backed Instances == | ||
| Line 44: | Line 45: | ||
Relatively recently, EBS (Elastic Block Storage) has also become an option. Instead of mounting a drive stored in S3, you can store the AMI in elastic block storage. This allows faster boot up, as well as an option to "stop" an image and preserve its state prior to stopping. There is also no size limit (or well, it's such a high limit that we don't really care) to the drive. However it is more expensive because instead of just paying hourly running cost + storage cost, we pay additional storage cost. We do save a bit of money on the AMI storage: we're only charged for the memory that our additional modifications to the AMI have made, whereas an S3 instance is charged for storing the whole AMI. | Relatively recently, EBS (Elastic Block Storage) has also become an option. Instead of mounting a drive stored in S3, you can store the AMI in elastic block storage. This allows faster boot up, as well as an option to "stop" an image and preserve its state prior to stopping. There is also no size limit (or well, it's such a high limit that we don't really care) to the drive. However it is more expensive because instead of just paying hourly running cost + storage cost, we pay additional storage cost. We do save a bit of money on the AMI storage: we're only charged for the memory that our additional modifications to the AMI have made, whereas an S3 instance is charged for storing the whole AMI. | ||
It wasn't super clear how much of a difference this made from the price perspective, but from a performance perspective EBS is clearly superior and AMI creation and everything related to that is significantly easier given that Amazon provides GUI tools and snapshot tools for AMI creation. As a result the AMIs that are prepared right now are EBS-backed and not S3-backed. | It wasn't super clear how much of a difference this made from the price perspective, but from a performance perspective EBS is clearly superior and AMI creation and everything related to that is significantly easier given that Amazon provides GUI tools and snapshot tools for AMI creation. As a result the AMIs that are prepared right now are EBS-backed and not S3-backed. | ||
== Preliminary Design == | |||
One server remains always-on (probably an in-house box of some sort). It contains the keypair necessary to access Amazon. A python script listens to mozilla pulse messages and simultaneously listens to requests on a socket port. A client program will send a changeset # to the socket server, and the server will in turn spin up an instance (using synchronization primitives to keep track of how many resources are in use), or it will queue the job if too many instances are already running. When a job completes, the server downloads the built binary from the EC2 instance, shuts down the instance (freeing the resource unless another queued job exists in which case it gets the job and executes it), and serves the binary via HTTP. It provides the download URL via a pulse message. The client program knows its build is complete when the pulse message comes through. Binaries older than 3 days are deleted via a cron job. <br><img url="http://i.imgur.com/xXLp3.gif" alt="" /> | |||
== | == Side Project == | ||
Create a wrapper that pushes to mozillapulse | |||
== | == Other ideas == | ||
Create parameter that makes build servers clone try instead of moz-central and build try requests! | |||
Create parameter that makes build servers clone try instead of moz-central and build try requests! | |||
== Proposed Usage == | == Proposed Usage == | ||