ReleaseReadiness

Problem

While feature work gets a lot of natural attention both internally and externally, an equally important part to any successful product is ensuring that each release meets a consistent bar of quality and user satisfaction. This set of "non-feature work" includes security, stability, usability, performance, localization, and other general product attributes necessarily to consistently deliver products users trust & love to actually use.

The question is how can we effectively prioritize these different sets of non-feature work (and stakeholders) against each other and against feature work. Right now those priorities are set completely independently of each other, making it hard to prioritize work consistently across teams.

A related but different problem is that each group then has to manually chase developers to ensure their priorities are addressed, putting developers in a fundamentally reactive (and defensive) role. We need a way to drive towards a common understanding on the the set of bugs to be addressed early in each release cycle that gives everyone involved a consistent understanding of the intended outcome (i.e. which set of bugs will be addressed for each area for a given window of time).

So the key questions are:

• How do we prioritize a set of issues in context of all the other product priorities in a way that is self-explanatory and consistent across our projects
• Once we set those priorities, how do effectively track and drive progress on those?

Characteristics of a good solution

• Reliable - It results in the outcome (i.e. priorities) we expect
• Predictive - We can use this mechanism to plan our work
• Bidirectional - Priorities can't be set in a vacuum or simply driven top-down, but are really a negotiation between the various stakeholders and the developers who will actually do the work
• Accountable - For a solution to be reliable, predictive and bidirectional people must feel accountable to each other to follow through on requests in a timely manner
• Flexible - Priorities and resources are always in a state of flux, so any solution needs to be predicated on that

Potential tools/approaches

• Release criteria or other targets to define what a quality release should look like
• Evangelizing the strategic importance of each of these non-feature areas
• Feedback mechanisms, including reports and dashboards that raise awareness on organizational & individual levels
• Better ways of tracking priority and state in bugzilla (that hopefully do not rely on comments or arcane keyword-foo)