ReleaseEngineering/How To/Set Up a New AWS Slave Class
work in progress
We currently run most of Linux based tests on AWS. AWS has many instance types. We use spot instances because they are cheaper than on-demand. The default instance type we run most of our Linux based tests is m1.medium. It is low cost but is not powerful enough for some of our tests, like crashtests and reftests on Android emulators, or media or gaia-ui tests on B2G. So how do you add a new slave class for so that different tests can run within different instance types within the for the same platform (i.e. Android 2.3 opt)?
This example will focus on the adding tst-emulator64-spot instance slave type because this is the one I recently added. The repo you'll need to work with are puppet , cloud tools and buildbot-configs.
First you need to create an AMI for the new slave type.
- To do: describe how to create an AMI from scratch, I just reused an exisiting AMI
The configs for each AMI are stored in cloud tools. For the tst-emulator64-spot AMI I created, I reused the existing tst-linux64 AMI id and copied it into the new configs I created. However, I changed the instance type to be c3.xlarge instead of m1.medium and also had to tweak the subnets because this instance type is not available in some AWS regions.
I also updated the cloud-tools so this new AMI would be listed as a platform. You can see [the changes required here http://hg.mozilla.org/build/cloud-tools/rev/2bd91d08ce88].
This config was then added to puppet so the AMI should have been created automatically the next time it ran. Also, we had to use invtool to add A and PTR records into DNS for the golden image. The cron job that creates the golden images is stored in Puppet. You can login to the aws manager machine to run the cron jobs manually and speed things up.
Once you have the AMI image up and running, you can [loan yourself a slave https://wiki.mozilla.org/ReleaseEngineering/How_To/Loan_a_Slave#Test_machines] of that type and try to run tests on it on your dev-master. You'll have to change the config in the loan document to match your new slave type, i.e. instead of cloud-tools/configs/tst-linux$arch it should be cloud-tools/configs/tst-emulator64 to follow my example.
Write patches to enable the new platform in buildbot and Puppet.
Example of a change to add a new slave class to puppet. In this example I added two new puppet slave classes because despite them being mapped to the same AWS instance type. This is because of duplicate builder issues in Buildbot if you don't do that.
Here's an example of the buildbot changes required to enable this new slave platform on ash
And then you need to enable the new platform in cloud-tools. Reorder?
And add new masters so they handle the load from the slaves. https://wiki.mozilla.org/ReleaseEngineering/AWS_Master_Setup