Breakpad/Status Meetings/2017-02-01

« previous meeting — index – next week » ^create?

Meeting Info

Breakpad status meetings occur on Wed at 10:00am Pacific Time.

Conference numbers:

   Vidyo: Stability 
   650-903-0800 x92 conf 98200#
   800-707-2533 (pin 369) conf 98200# 

IRC backchannel: #breakpad
Mountain View: Dancing Baby (3rd floor)

Operations Updates

• stage submitter problems
  • on wednesday, january 25th, it stopped submitting
    • turned out it was looking at the rabbitmq cluster that Miles took out (it should have been unused)
    • we updated the stage submitter and -prod crashmover configuration to use cloudamqp cluster
  • on tuesday, january 31st, it kicked up disk space alert
    • bunch of orphaned processes again
    • Will tweaked the run_submitter.sh script adding the -once arg for envconsul
    • increased crashes to prod -> increased crashes to stage submitter -> more logging -> more disk used by log files

• crash collection rate in prod increased 3x
  • door hanger to submit old crashes was turned on again
  • this will happen for the first ~5 betas of every cycle
  • see: https://bugzilla.mozilla.org/show_bug.cgi?id=1303067

• access to leeroy instructions are on socorro-dev
  • needs duo push, so it needs the duo app

Project Updates

Deployment Triage

• https://whatsdeployed.io/s-7M7

PR Triage

• http://prs.mozilla.io/mozilla/socorro

Major Projects

Splitting out collector (Antenna)

• (willkg) checked whether data saved by Antenna is the same as the Socorro collector; fixed a couple of issues--should be good to go now
• (willkg, miles) checked -stage environment and dashboards--all set for a load test on -stage
• (willkg, miles, lonnen) worked out two viable ways to get crash siphoning from -prod to -stage working with Antenna; pushing off any work until we need it with the hopes that AWS Lambda supports Python 3 by then

Deprecation rampage

• no update

Upgrading elasticsearch

• migration plan: https://public.etherpad-mozilla.org/p/socorro-es5-migration-plan
• (adrian) Use this opportunity to start using aliases for all our indices?

Other Business

• (mbrandt) Geckodriver