Breakpad/Status Meetings/2016-08-17

From MozillaWiki
Jump to: navigation, search

« previous meetingindexnext week » create?

Meeting Info

Breakpad status meetings occur on Wed at 10:00am Pacific Time.

Conference numbers:

   Vidyo: Stability 
   650-903-0800 x92 conf 98200#
   800-707-2533 (pin 369) conf 98200# 

IRC backchannel: #breakpad
Mountain View: Dancing Baby (3rd floor)

Operations Updates

P1 Infra Bugs

  • S3 woes. We figured it out in the end. IAM is hard.
    • Problem was that rhelmer used *his* key to configure access to a bucket
    • We need a more formal cleanup of IAMs and policies.
    • Lesson learned: When going to prod TEST the bucket access FIRST!!!
  • Stage submitter PR ( is almost ready to go
    • plan is to build it so it's a NON-admin node, that upgrades on deployments
  • Ready to do a prod deploy now.
  • Python Upgrade?
    • tried over the weekend
    • right python on built machines, but things didn't start
    • peterbe is volunteering to debug python errors in stage with jp
  • Pingdom
    • peterbe, mbrandt, jp receving
    • let's not worry about adding more admins. for now.
    • adding more admins costs more money
  • ElasticSearch monitoring
    • jp talked with "our local ES expert" about things to alert on
    • will do alerting with Datadog (this is what we already do with stage submitter)
  • Status of NewRelic
    • Owned by IT, still
      • will soon be possibly owned by Travis's team
    • Possibly looking at "Synthetics Transactions" as an alternative
    • Still not working,
      • will resume debugging after python 2.7.11 upgrade

Project Updates

  • and emails can now BOTH upload private symbols
  • We're still waiting for word from MacAfee that they're ok with the *output* of symbolication is made public.

Deployment Triage

PR Triage

Major Projects

Migrating off of Persona

  •  :njn can sign in. But there might be bugs related to Nightly and Google Sign-In.

Sending public data spark/presto

Signature generation across crash reporters

  • on hold
  • crash ping will not need signatures iff we can tie to crashid. meeting with legal next week

Splitting out collector

On hold. Will is implementing some metrics gathering code in the collector to track crash report sizes. This is needed for the collector architecture doc because the size of crash reports that we need to handle is one of the requirements for our collector.

Collecting client-side JavaScript errors

Handling more PII data in crashes

Sending stacks for all crashes from the client

  • durst is almost ready to send stacktraces to S3
  • they are not yet in the business of trying to take it from S3 to spark
    • they might piggyback off our solution (S3 -> JVM -> Spark)

Replacing FTPscraper

  • No news from nthomas.
    • peterbe will ping/pester

other business

  • Changing the meeting time while Adrian is abroad?
    • DECIDED on 1pm PST | 4pm EST | 8am AST (Adrian Special Time)
    • First time for this change is Sept 14

Travel, etc

  • Adrian out next week
    • then working the following week
    • then out the week after that
    • then working from far away for ~3 months
  • Willkg on pto friday, august 19th