Breakpad/Status Meetings/2017-05-17

From MozillaWiki
Jump to: navigation, search

« previous meeting — index – next week » create?

Meeting Info

Breakpad status meetings occur on Wed at 10:00am Pacific Time.

Conference numbers: 

   Vidyo: Stability 
   650-903-0800 x92 conf 98200#
   800-707-2533 (pin 369) conf 98200#

IRC backchannel: #breakpad
Mountain View: Dancing Baby (3rd floor)

Operations Updates

  • Miles wrote up a new migration document with different details than in past versions. https://docs.google.com/document/d/18xKCDeXQDpoPtVpxJ9BeisAMrxLXyKhFUIfJNWklaYM/edit
    • we're going to "tee" off the infrastructure at Antenna and then build two parallel Socorro universes which should let us figure out all the stuff and then cut over smoothly
  • We saw a weird uptick in processor/elasticsearch save_raw_and_processed times and the symbols api hits.
    • RabbitMQ queues were backing up, so Miles threw more processors at the queue.
    • It's weird because it doesn't look like we're processing more crashes.
    • Adrian fixed some issues in the processor and maybe that'll make things better. We'll find out after -prod deploy and investigate further if need be.
  • System test for Tecken is working again.
    • There are some issues with https urls, but Peter and Miles will take that offline.
  • We have a crontabber thing that runs in SCL3 that needs to send data to postgres for ADI.
    • How to do this in the new upcoming infrastructure?
    • Switch it to push via HTTP to the webapp rather than send stuff to Postgres.
    • Also, that box is "interesting", so it's probably hard to update/deploy/etc.
    • We might be able to get rid of this altogether.
    • Peter will file a bug about this server.
  • Miles asked whether we can ditch crontabber on admin nodes. Probably not, yet.
  • Official ES migration plan involves snapshots and that's the way it's going to go.

Security Updates

Project Updates

Deployment Triage

PR Triage

Major Projects

Deprecation rampage

  • removed the lines
  • removed obsolete cron job code

Processor rewrite

  • (willkg): As part of dockerizing Socorro, I got stackwalker built and running in a python:3.6.1-slim container. That's related, but otherwise, no new progress.
  • We talked about parallelizing code changes to the old processor (e.g. KeyErrors in transform rules)
    • willkg states...
      • don't worry about making changes to the old code
      • don't worry about formalizing annotation about changes to the code

Upgrading elasticsearch

Symbol Server 2017

  • Celery is working in dev
  • Needs to decide a date to go to prod and take over the domain symbols.mozilla.org
    • Requirements to go live
      • OK from Ted that we take over symbols.mozilla.org
      • Start a security review process
      • Start a load testing once it's got a stage or prod environment

Other Business

  • Stickers? Will is working on stuff and will talk about it next week.
  • We should add a QA section to the meeting agenda.

Travel, etc

  • peterbe out May 18-19, May 25-26

Links

Retrieved from "https://wiki.mozilla.org/index.php?title=Breakpad/Status_Meetings/2017-05-17&oldid=1171405"