QA/Data harvesting/Progress Card

From MozillaWiki
Jump to: navigation, search

Progress Card



Feature Prio Delivery Estimated Date of Completion Owner
Additional Comments
Ability to compare data from different projects

Sample from bugzilla here

P2
  v2


This is put on top as it influences the usefulness of all the other featueres. Pretty much every graph described below needs to allow for filtering by project, where project is something that is loosely defined on a case-by-case basis, using these fields:

  • we currently have keywords, product, component, status_whiteboard
  • needs flags, needs target_milestone, needs teams


Ability to compare data from different releases

Sample from bugzilla here

P1
v2
2010-08-27
Michael Kurze

This ties in closely with the feature above. In principle, the same set of field is used to constrain data for a release (especially target_milestone and flags).


Find/fixed Rate-trend Graphs

Sample metrics types here

P1
v1
2010-08-27
Michael Kurze

This will need flags to allow for better selection of releases. Also, DWH data is incomplete (only bugs #560K to about #585K are included).

  • opened/found per week (last 7 weeks): [1]
  • closed/fixed over time (last 7 weeks): [2]



Buckets:grouped by priority

Samples from bugzilla here.

P1
v1


  • Current open bugs by prio for products Firefox and Core: [3]



Buckets:grouped by severity
P1
v2


  • Currently open bugs by severity, for products Firefox and Core: [4]



Buckets:grouped by   component

Sample from bugzilla here

P1
v2


  • Current open bugs by component, for products Firefox and Core: [5]
  • Same, but only the 10 components with the most bugs: [6]
  • Ideally we want to support faceting by combination of component x status so we can have nice bar charts with multiple colors per column (one column per component, one segment/color per status). This is not currently in SOLR, but there is a patch ([7]) that we could use. Can also be done currently with one SOLR query per status.



Buckets:grouped by bug-asignee

Sample metrics types here

P3
v2


  • For the new DWH, this needs team-data to be useful, especially for per-team-dashboards.
  • Get open bugs by assignee (other than nobody), just filtering by product, components etc.: [8]



Buckets:grouped by flags

Sample metrics types here

P1
v2
2010-08-20
Michael Kurze

needs flags


Average age of a bug in each component

Sample metric types here

P1
v2


This can all be filtered by product, component, keyword etc.

  • Average age of the currently open bugs, by component: [9]
  • Average sum of OPEN time (including in re-opened state) per bug [10]
  • Average time a bug is OPEN (or re-opened) before being closed: [11]



Rate/percentage of bugs being reopened/project

Sample metric types here

P2
v1


This can all be filtered by product, component, keyword etc.

  • Currently open bugs, how many are still open for the first time vs. re-opened at least once: [12]


Regression bugs per release/project

Samples from bugzilla here

P1
v1
2010-09-04
Michael Kurze

This can all be filtered by product, component, keyword etc.

  • Of the given bugs, normal bugs vs. regressions [13]


Average age of bugs in unconfirmed status

Sample metrics types here

P1
v2


This can all be filtered by product, component, keyword etc.

  • How long did bugs stay UNCONFIRMED (on average) before being changed, not counting bugs that still are unconfirmed [14]
  • How old are UNCONFIRMED bugs (avg) that still are UNCONFIRMED (hard to do in a single query with the above, but it really is something different). Interestingly, this number is much higher. So if your bug has not been confirmed after 5 days, there is an increasingly high chance that it never will be (this is also due to the inclomplete data set that I have here!). [15]


Tabular reporting P3
CURRENTLY AVAILABLE NA

The metrics dashboard should automatically deliver this with all dashboard items.


Options to-CSV/JSON/Save the graph as an image
P3
v2
NA

If anything of this is buggy or difficult, it is probably best to file a metrics bug.