Breakpad/Status Meetings/2009-Feb-11

From MozillaWiki
Jump to: navigation, search


  • Partitioning post-mortem
  • Quantifying where we need help?
  • Plans for Socorro for rest of Q1
    • bugzilla integration plans
    • deliver on rest of reskin
    • updates to streamline and optimize partitioning
    • security check
    • migrate old data
    • unit tests
  • prod/stage/dev - what do we need? Is this clear?

Partitioning Post-mortem

What went well?

  • It's finished, people worked really hard, we accomplished something that is not easy
  • Awesome that we can do quick restores, moving things to a faster disk drive
  • Getting a copy of the production database so that Lars doesn't have to proxy through Aravind
  • Really liked the screen idea, even though it was suspenseful

What can we do better?

  • Restored wrong database copy, so had to back it out
  • Second time we should have turned off the read logs, etc. so we don't fill up disk space
  • Have a nagios check postgres slave isn't too far behind
  • Try to tackle things with a forward-looking approach first instead of hanging ourselves up on migrating tons of data
  • Testing, testing, tesgting -- getting the environments to do this testing


  • aravind
    • Do agree that we need to have a dev database
    • Staging should not be in the same scale as production
  • Lars
    • OK if we don't have to test things on staging
    • Haven't had a development environment that was like production
    • If we get that, staging being just like production isn't a big problem
    • Would like to set up tools in development environment for a QA person
  • QA
    • need a QA person
    • could Frank be our QA person


  • Staging - ok the way it is
  • Dev - needs to be like production so devs can perform end-to-end tests
  • QA - need to have someone involved from QA to help test things in the development env

What We Need Help With

  • server configuration optimization, hardware optimization
  • determine what type of replication (if any) we need to use moving forward
  • write documentation based on above findings, wrap-up with team
  • a year from now, what will we wish we would have done now? comments on strategy?
  • strategy for migrating old data?