CodeCoverage/Firefly: Difference between revisions

m
 
(6 intermediate revisions by 3 users not shown)
Line 1: Line 1:
Quick links:  [http://people.mozilla.com/~mnandigama/codecoverage_html/ C/C++ coverage]  [http://people.mozilla.com/~mnandigama/jscoverage-report/jscoverage.html JS Coverage]  [https://wiki.mozilla.org/CodeCoverage/Firefly:BugMetrics Bug Metrics]
== Overview ==  
== Overview ==  
=== [[Image:Bigfirefly.png|left|82x110px]]Introduction  ===
=== [[Image:Bigfirefly.png|left|82x110px]]Introduction  ===


Project Firefly is a Mozilla project about using code coverage data, bug data and Hg commit information to identify where we need to create test cases that will have the highest impact on preventing or quick catching regressions.  
Project Firefly is a Mozilla project about using code coverage data, bug data and Hg commit information to identify where we need to create test cases that will have the highest impact on preventing or quickly catching regressions.  


The following coverage cycle best describes our tactical approach.  
The following coverage cycle best describes our tactical approach.  
Line 40: Line 42:
That is what we have done in this exercise. We have generated data points for each file regarding the number of times it changed to number of different kinds of bugs fixed in the file to its coverage numbers in various test modes.
That is what we have done in this exercise. We have generated data points for each file regarding the number of times it changed to number of different kinds of bugs fixed in the file to its coverage numbers in various test modes.


See the full spreadsheet with these bugs stats [http://people.mozilla.org/~timr/CodeCoverage/resultset.xls here].
Download the full spreadsheet with these bugs stats here: [http://people.mozilla.org/~timr/CodeCoverage/resultset.xls results.xls].


=== How: Test Development Workflow  ===
=== How: Test Development Workflow  ===
Line 47: Line 49:


This section describes the work flow for recording information about those priorities and decisions; and then process for starting, tracking, and completing work to build test cases and get them into automated test suites.
This section describes the work flow for recording information about those priorities and decisions; and then process for starting, tracking, and completing work to build test cases and get them into automated test suites.


=== What: Code Coverage Reports and Tools  ===
=== What: Code Coverage Reports and Tools  ===
Line 59: Line 62:
You can also instrument and run code coverage tools on your own builds. Here is how to do that
You can also instrument and run code coverage tools on your own builds. Here is how to do that


*[https://wiki.mozilla.org/QA:CodeCoverage DYI Firefox Instrumentation]
*[https://wiki.mozilla.org/QA:CodeCoverage DIY Firefox Instrumentation]


== Major Accomplishments Timelime==
== Major Accomplishments Timelime==
Line 77: Line 80:
*Show the lists of test cases touching the source files.  
*Show the lists of test cases touching the source files.  
*Create a '''testing->codecoverage''' category in bugzilla to file enhancement requests.
*Create a '''testing->codecoverage''' category in bugzilla to file enhancement requests.
*{{Bug|510232}} Code coverage for Electrolysis
* '''XPINSTALL results should not be that low. There is supposedly a better test suite to test it to provide max coverage.'''
** A mail has been sent to Dave Townsend and Rob String to look at the test run logs located at http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/experimental/codecoverage/codecoverage-mozilla-central-logs.tar
*{{Bug|510489}} filed to investigate the log files decompression-corruption issue.
== TestDev Planning based on Code Coverage Data ==
'''Aug-13-2009''':: '''Tony Chung'''
As discussed in today's QAE meeting, these are the associated .js files
along with the P1 feature ownership thats been defined in Tracy's
feature ownership spreadsheet.  Note that there are more files that
touch these components in .cpp, .h, xul, xml, and other .js files.  But
this is a place to start.
Again, the point of this is to track look over missing areas of code
coverage from the report, and see if there are existing litmus tests
that provide test coverage.  If neither manual or automated test
coverage is available, please work with murali, clint, and the feature
developer to determine what testcases need to be written (either
automated or manual coverage)
Latest JS Coverage report:
http://people.mozilla.org/~mnandigama/jscoverage-report/jscoverage.html
Feature Ownership spreadsheet:
http://spreadsheets.google.com/ccc?key=0AkSa2kZ0OBffdE94NTdKcFFENEdfRE9OcmswUTFyM3c&hl=en
Addons Manager
- Owner: tchung
- extensionsManager.js
Audio Video
- Owner: tchung
- cpp files.  no .js
Awesomebar
- Owner: Tracy
- nsPlacesTransactionService.js, nsPlacesUtil.js
Plugins
- Owner: tchung
- pluginGlue.js
Private Browsing
- Owner: marcia
- nsPrivateBrowsingService, nsHelperAppDlg.js
Security
- Owner: Ashughes/Aakashd
- nsSafebrowsingApplicaiton, BadCertHandler.js
TabbedBrowsing
- Owner: Marcia/Henrik
- nsDragandDrop.js, browser.js
I will work on the P2 and P3 feature coverage breakdown in another email.


== FAQs  ==
== FAQs  ==
Line 142: Line 202:


** http://mkngama.blogspot.com/2009/08/richard-feynmanthinking.html
** http://mkngama.blogspot.com/2009/08/richard-feynmanthinking.html


== Contact  ==
== Contact  ==
2,088

edits