Security/Meetings/lifecycledisc

A small meeting on 2011-08-17 on the topic of one slice of the security lifecycle: getting bugs fixed faster, once they have been identified.

Define the Problem

• We'd like security bugs to get fixed faster.
• Some teams are unresponsive. Can we solve this in a better way than "comment in bugs every week", which is time-consuming (especially when done in a Critsmash meeting) and somewhat adversarial and dosn't scale beyond crits?
• Evolving sec bugs to be actionable by product teams and allow them to self prioritize with regards to the full list of priorities
• It's difficult for teams to discuss security bugs in their public weekly meetings, which is where they do much of their prioritization.
• The prioritization of "work" needs to get better.

Communicating the security team's priorities

• [sg:high P1] or [sg:high][sg:P1] to mean "we cannot ship another release with this bug"
  • Borrowing from http://dbaron.org/log/20090120-bug-priorities
  • But we don't actually get to block releases on security fixes (we do in the extreme: "chemspills")

Moving security bug triage out of the security team

• We want to not be in the relative prioritization and weekly-poking loop for every bug.
  • At most we should be helping with sg ratings and understanding risk of things becoming 0day.

Release triage team

• Urgent security bugs have some things in common with release-risk-tracking (Aurora/beta triage)
  • But maybe not enough similarity to justify using the same flags (e.g. tracking-firefox6+) and meetings.
• These people already have a lot on their plate (three one-hour meetings a week) :/
• These binary "blocking" and "don't care" buckets don't work.
  • Does the security team need to help the release triage team figure out how to do its job?
• Appropriate for security bugs that are recent regressions.

Feature inbox team

• Multidisciplinary security bugs need to be prioritized against other large pieces of work (Feature Inbox team).
• Quality-in-general needs to be prioritized against features-in-general.

Quality-prioritization team

• A new "Quality prioritization" team that is repsonsible for setting consistent relative priorities between critsmash, memshrink, crashkill, gofast?

Team leads

• Understand how hard each bug is to fix, and how much time their team members have.
• Under pressure from many different directions.

Communicating how teams are doing on security

• Weekly sec bug report improvements
  • Bucket components by team
  • Weight so that 5 highs = 1 crit or so, rather than sorting only by crits
    • Current charts have weights like 10, 7.5, 4, 1.5 or 10, 8, 4, 2 (critical, high, moderate, low)
• Make security stats public, creating a shitstorm but aligning incentives
  • Start with new bugs, but specify a time at which we'll do it for all bugs (kind of like what ZDI did)
• Start talking about security bugs in large public meetings

Future conversations and meetings

• A meeting with team leads (roc, dmandelin, joedrew, dolske, dtownsend, dietrich, bsmedberg, josh) and direction setters (johnath, damon, shaver, JP, dria?)
  • Possibly during the September all-hands.
• A meeting with quality-aspect leads: njn (memshrink), smooney (crashkill), cww (support), etc.
  • How they socialize priorities.
  • How they keep bugs moving.
  • Each team puts out one or more weekly reports; perhaps we should all collude on a single "weekly quality report".