QA/Execution/Meetings/2009-11-18/MeasuringQuality

Measuring Quality Notes and Thoughts

At the next qa meeting, lets continue our discussion on some concrete thoughts and ideas on quality measurement going into 2010. Please jot them down below on the wiki or notepad and come prepared to discuss on wednesday. The goal of this discussion is to come up with a collaborative approach to defining quality and a release criteria, and to put it into practice across our projects.

Some questions as a baseline:

• How would you measure quality in the project that you own?
• How would you define a release criteria to ensure you are shipping a quality release?
• What are some ways you can demonstrate more leadership on your project?

Tony

• spending 30mins every 2 weeks with a feature developer, going over a list of fixed bugs and talking about test strategy
• Look for more regression trends across bugs after every 3.6 release. Need to get a feel on how many checkins is a "good number". for example, is 150 bugfixes landing within a week of beta a good thing? how many regressions would that cause? Or maybe it is a good number, but need some price points to compare with.
• figure out a better way to track qawanted. If we're not keeping up with it, developers will not use it. Currently, i see damons using it the most, but then he has to send an seperate ping to get our attention. We need to be more proactive on qawanted bugs.
• I'd like to get developers when giving their component area updates, to also include any hints or areas that could use some extra testing in their area that week. For example, if Layout is making big changes to CSS, then it'd be good for them to mention to check more sites with fullscreen flash or div heavy tables. We can get in the practice of asking people, but the goal is to get them to volunteer information.
• Shifting mindset to not just testing the fix that a developer is asking you, but really ask the questions: "What other areas of exposure? What is the overall impact?" Scan related bugs.

Tracy

• Less manual pre-formatted regression testing (trust automation to do that)
• More proactive hammering on currently unstable areas.
• A stronger QA presence in Bugzilla
  • work mostly in the highest priority areas
  • get Shaver to keep to his promise of returning QA ownership to us