Confirmed users
137
edits
QA/Firefox/Risk Report: Difference between revisions
Jump to navigation
Jump to search
no edit summary
(Created page with "== QA Risk Analysis Report == === Potential Tracking Bugs === * Status: UNCONFIRMED, NEW, ASSIGNED * Product: Firefox * Classification: Client Software * Blo...") |
No edit summary |
||
| Line 1: | Line 1: | ||
== QA Risk | = The Canary Project = | ||
== QA Risk Indicators == | |||
Project Canary is about identifying potential risks early in the development process so they can be analyzed and addressed before they become bugs in the release version. Ideally these indicators (canaries) can be reported on a wiki page/dashboard for easy reference and be used to help focus our testing efforts on release. | |||
* Step 1: Find the hidden tracking bugs that have been recently updated | |||
* Step 2: Identify unusual amount of code changes in any given release/file. | |||
* Step 3: Display this data in easily accessible page. | |||
=== | === Bugzilla MediaWiki Extension Update === | ||
Add the ability to do client side filtering of data that can not easily be done with the existing REST APIs in the extension. | |||
* Added display type 'filter' | |||
* options | |||
** filter_on='<field_name>' | |||
** filter_op='<[gt,lt,eq,ne]'> | |||
** filter_value='<field value or array size>' | |||
Example: | |||
bugzilla display='filter' filter_on='depends_on' filter_op='gt' filter_value='3' | |||
This will only display bugs that have more that 3 depends_on bugs. | |||
These options are usable with any standard bugzilla extension query | |||
Why modify the extension? | |||
* Not all tracking bugs are marked as such | |||
* Not all tracking bugs are equal. | |||
** There are a lot of bugs with one or two depends on bugs. For the most part these are not representative of a significant amount of work being done. | |||
** There is no means of limiting a Bugzilla query based on the number of tracking bugs | |||
This extension modification allows for a more fine grained approach to querying data outside of that supported by the standard REST APIs used by the extension or in Bugzilla query interface. | |||
=== HG-Metrics === | |||
Analyzes hg logs and tracks the average rate of change for each file in the code base. It then identifies any unusual spikes in activity for each file by checking the percent change for a given release against its average percent change plus its standard deviation to account for normal fluctuations in change rate. | |||
This consists of two parts: | |||
* Back end data collector [https://github.com/mschifer/hg-metrics hg-metrics on github] | |||
* Python script that collects the hg logs and analyzes the data which it then stores in a sqlite database. | |||
* MediaWiki extension for querying the database and display the results based on user specified parameters. | |||
=== The Canary === | |||
Currently the above reports can be found on: | |||
http://10.252.28.88:8888/wiki/index.php/Main_Page/RiskReport | |||