Auto-tools/Projects/Infraload 2013Q2
Contents
Goal 2013 Q2
Increase our infrastructure capacity by 10%.
Approach and Milestones
Our infrastructure load has been running deep in the red for the past month and a half. In the last 11 days, Mozilla Inbound has been closed 16% of the time. And while not all of those closures have been due to infrastructure issues (there was a colo loss in there) most of them were. There are many many issues that play into this. Part of it is that our systems cannot keep pace with our rate of use of the infrastructure. Some of it is that we are running everything on every checkin, one process per machine, which is perhaps not the most efficient way to use our resources. There is much we can do, but in order to ensure we are doing the right things, we will measure the load and the usage style first, then take action on the top 5 items that will reduce our infrastructure load.
The project flag for use in bugzilla whiteboard queries is: infraload
- TODO: Add bugzilla bug queries to this by attaching proper whiteboard flags to the relevant bugs that are filed.
Milestone 1 - 1 week (April 8 - 15)
- Measure necessary items and decide on what action needs to be taken
Milestone 2 - 1 week (tentative start: April 15)
- Assign teams their items to either further measure or begin sprinting on (and fill out the rest of the details below)
Milestone 3 - 2 weeks (tentative start: April 22)
- Complete first round of identified sprint items
- April 30: Re-measure current infraload with what has landed, and course correct as needed.
Milestone 3 - 2 weeks (tentative start: May 6)
- Complete second round of identified sprint items
- May 14: Re-measure current infraload with what has landed and course correct as needed
Milestone 4 - 2 weeks (tentative start: May 20)
- Complete third round of identified sprint items
- May 28: Re-measure current infraload. If insubstantial progress has been made, this goal will usurp all other goals for the remainder of the quarter. We'll be in a "all hands on deck" firefight until the end of June to bring our numbers in line.
