|
|
| Line 1: |
Line 1: |
| * NOTE: Source: https://etherpad.mozilla.org/Loads-Current-Status-Aug2014 | | * NOTE 1: Source: https://etherpad.mozilla.org/Loads-Current-Status-Aug2014 |
| * NOTE 2: This is specifically for Loads V1 | | * NOTE 2: This is specifically for Loads V1 |
|
| |
|
| Line 110: |
Line 110: |
| ** Verifier: https://github.com/mozilla/browserid-verifier/issues/50 | | ** Verifier: https://github.com/mozilla/browserid-verifier/issues/50 |
| ** Sync: https://github.com/mozilla-services/server-syncstorage/issues/19 | | ** Sync: https://github.com/mozilla-services/server-syncstorage/issues/19 |
|
| |
|
| |
|
| |
|
| |
| = Loads V2 =
| |
| * What is it?
| |
| * Changes for V2
| |
| * Overview/Slides: http://blog.ziade.org/slides/loadsv2/#/
| |
| * Initial Diagram: http://blog.ziade.org/loads.jpg
| |
| * Initial Look: https://etherpad.mozilla.org/Loadsv2
| |
| * Ben's design work: https://etherpad.mozilla.org/loadsv2-design
| |
|
| |
| == Comparison of Load Test Tools ==
| |
| * Siege: http://www.joedog.org/siege-home/
| |
| * And some others in comparison: http://www.appdynamics.com/blog/devops/load-testing-tools-explained-the-server-side/
| |
| * https://github.com/newsapps/beeswithmachineguns
| |
| * Some of these require large sums of $ in order to run adequate load tests (size/time)
| |
| * Straight HTTP vs. smart tests (that we are sending)
| |
| * Dumb testing vs. smart testing (what we are doing)
| |
| * Some of the off-the-shelf are quite limited - we need to be able to use a programming language to define very specific tests/requirements
| |
| * The Grinder, for example, is not really designed to be deployed on AWS, for example.
| |
| * Tsunami is good at sending a lot of load on web service, but it requires writing XML
| |
|
| |
| == Tasks ==
| |
| * Tarek says:
| |
| * So what I was thinking: I can lead Loads v2 development with the help of QA and Ops and Benjamin for SimplePush, and then slowly transition ownership to the QA and Ops team - because at the end of the day that's the two teams that should benefit the most from this tool.
| |
|
| |
| == New Repos ==
| |
| * https://github.com/loads
| |
| * https://github.com/loads/docs
| |
| * https://github.com/loads/loads-broker
| |
| * https://github.com/loads/loads-tester
| |
| * https://github.com/loads/old-loads-agent
| |
| * https://github.com/loads/old-loads-broker
| |
| * https://github.com/loads/old-loads-base
| |
| * https://github.com/loads/old-loads-web
| |
| * Note: naming is a bit strange right now because the architecture is in transition
| |
|
| |
| == New Documentation ==
| |
| * TBD: for now see https://github.com/loads/docs
| |
|
| |
| == Brown Bag and Info Session ==
| |
| * https://etherpad.mozilla.org/loads-brownbag
| |
| * Note: This will not take place in September, but could take place in December (2014)
| |
|
| |
| = Brainstorming Loads and V2 =
| |
| * What we need going forward
| |
| * What we want going forward
| |
| * Some issues (generalized - see the github issues for details):
| |
| ** 1- very long runs (>10hours) are not really working. This is a design problem.
| |
| ** 2- spinning new slaves to make big tests has not yet been automated. We have 2 slaves boxes that run 10 agents each. This was enough for most of our needs though.
| |
| ** 3- The dashboard is scarce. It'll tell you what;s going on, but we don't have any real reporting features yet.
| |
| ** 4- running a test using another language than Python is a bit of a pain (you need to do some zmq messaging)
| |
|
| |
| * Stealing from Tarek's slide deck:
| |
| ** Day-long runs don't really work
| |
| ** Crappy dashboard
| |
| ** No direct link to Logs/CPU/Mem usage of stressed servers
| |
| ** No automatic slaves deployment yet
| |
| ** Python client only really supported
| |
| ** High bar to implement clients in Haskell/Go
| |
|
| |
| * Also, we have a lot of open bugs that need to get fixed. Some prevent better use of the tool for newer projects/services.
| |
| ** Get Loads "fixed" for Mac 10.9 and XCode 5.1.1: https://bugzilla.mozilla.org/show_bug.cgi?id=1010567
| |
|
| |
| * Figure out how to run loads from personal AWS instances
| |
| * Monitoring
| |
| ** What we currently have for Stage
| |
| ** What do we want/need?
| |
| * Reporting
| |
| * Loads dashboard
| |
| ** What about CPU/memory information (like from atop, tops)
| |
| ** Links to some snapshoted graphs
| |
| ** code version
| |
| ** red/yellow/green states
| |
| ** Deployment bug
| |
| ** Bugs opened
| |
| ** Bugs closed
| |
| * Scaling the cluster dynamically (see V2)
| |
| * Quarterly results/trending
| |
| * Targets
| |
| ** PM targets
| |
| ** expected targets
| |
| ** actual targets
| |
| * Wiki design
| |
| ** One per service?
| |
| ** One per service per deployment?
| |
| * Weekly reporting
| |
| ** What does the QE team want to see
| |
| * Getting the right data/metrics requirements from PMs then extracting that information and displaying on the Loads dashboard and/or in the OPs-built dashboards
| |