CrashKill/2009-11-02

Misc

Not really enough users/crash data from the 3.6b1 release last week to tell where we are with crash fixing for 3.6. Still under 2000 crashes per day and only 601 signatures to look at. That's around the same that 3.5.x releases get during during their pre-release period.

• day total-crashes number-of-signatures for 3.6b1
• 20091027-crashdata.csv 410 230
• 20091028-crashdata.csv 453 249
• 20091029-crashdata.csv 527 266
• 20091030-crashdata.csv 544 297
• 20091031-crashdata.csv 1049 418
• 20091101-crashdata.csv 1468 500
• 20091102-crashdata.csv 1939 601

• day total-crashes number-of-signatures for 3.5.4
• 20091025-crashdata.csv 1852 908
• 20091026-crashdata.csv 1730 895
• 20091027-crashdata.csv 2825 1310
• 20091028-crashdata.csv 34007 8190
• 20091029-crashdata.csv 79135 15053
• 20091030-crashdata.csv 92431 16607
• 20091031-crashdata.csv 91250 16441
• 20091101-crashdata.csv 105485 18390
• 20091102-crashdata.csv 108838 18666

Breakpad & Socorro

• • metrics goals and priorities and high level goals
  • storage situation and retention policies
    • raw crash dumps/jsons (used as deferred storage)
      • 7 Days (no backups) - currently stored on a freebsd NAS server - disks are redundant, hardware not redundant.
    • storage of processed crash dumps and jsons (processed)
      • The jsons will be stored for 3 years (haven't deleted any yet) - Currently stored on a sun thumper, disks are redundant, hardware not redundant. Not being backed up currently, but we have a bug in place to back these up (should be done something in the next couple of weeks).
      • The processed raw dumps are on the same freebsd NAS as above. My current plan is to wipe them out after a week.
    • disk space for the database server
      • Currently on equallogic storage arrays.
      • The head is not redundant, but the disk is redundant.
      • This is backed up onto another eql storage volume every night, and WAL is played back on the backup server to keep things current. (not dumped to tape).
    • other storage?
  • Future Storage - two minutes talking about the hadoop/hdfs plan
  • socorro roadmap

Bugs