Breakpad/Status Meetings/2017-03-08

« previous meeting — index – next week » ^create?

Meeting Info

Breakpad status meetings occur on Wed at 10:00am Pacific Time.

Conference numbers:

   Vidyo: Stability 
   650-903-0800 x92 conf 98200#
   800-707-2533 (pin 369) conf 98200# 

IRC backchannel: #breakpad
Mountain View: Dancing Baby (3rd floor)

Operations Updates

• load testing continues
  • good data from yesterday
  • c4.large nodes can handle 40 submissions/second without much tweaking
  • we're now at the point where an optimized node can handle all prod load
• step up test on single node done, next is auto-scale, then recovery tests where we step up and down

• migration plan for broader socorro (beyond ES)
  • details, details
  • scattered now across too many docs

Project Updates

Deployment Triage

• https://whatsdeployed.io/s-7M7

PR Triage

• http://prs.mozilla.io/mozilla/socorro

Major Projects

Splitting out collector (Antenna)

• (rpapa, miles, willkg) continuing load testing antenna; landed and tested batched metrics changes from last week, added some additional graphs to dashboard, tweaked concurrent_crashmovers configuration variable
• (willkg) worked more on the migration plan and wrote up bugs for missing things
• (rpapa) got the load tests to increment load being sent in steps--this is super awesome!

Current status is that a single Antenna node can handle about 32req/s comfortably (we're using "request" as a proxy for a crash) and we've got an iteration process for loadtest-analyze-theory-fix that works well.

Deprecation rampage

• current status
• (Adrian) slowly progressing, next big step: removing the old Correlations stuff

Processor rewrite

• no update

Upgrading elasticsearch

• (Adrian) no updates
• (Adrian, willkg) Working through the ES PR

Other Business

• end to end tests in an antennae world
  • WillKG has been dockerizing socorro
  • no code, but very similar to what's already in place on antennae

Travel, etc

• Miles AFK Friday
• Lonnen AFK Friday