User:Rcampbell/Toronto Summit Notes: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
m (GPHemsley moved page Robcee:Toronto Summit Notes to User:Rcampbell/Toronto Summit Notes without leaving a redirect: User page) |
||
| (3 intermediate revisions by one other user not shown) | |||
| Line 2: | Line 2: | ||
== automated testing == | == automated testing == | ||
=== | === Buildbot functions === | ||
* pull nightly | * pull nightly | ||
* initiate test runs | * initiate test runs | ||
* build given patch | * build given patch | ||
** | ** '''Ben Hearsum doing work with Build team on this''' | ||
* select platforms | * select platforms | ||
* reporting handled by performance testing framework | * reporting handled by performance testing framework | ||
| Line 41: | Line 41: | ||
* store some set period | * store some set period | ||
* retain important historical milestones / benchmarks | * retain important historical milestones / benchmarks | ||
== JS Tests == | |||
* specialized builds | |||
* buildbot vs tinderbox canned builds | |||
** requires js shell, xpcshell, can't use 'off-the-shelf' versions. | |||
== Metrics == | |||
* bug density | |||
** per component | |||
** map bugzilla modules | |||
** checkins churns | |||
* lines of code | |||
* performance thresholds | |||
** links on wiki | |||
* top-crashers | |||
Latest revision as of 00:53, 11 May 2015
These are robcee's bulletized notes from the Toronto QA Development Summit on January 25th-26th, 2007. They are rough and there are large pieces missing. Feel free to fill in the gaps.
automated testing
Buildbot functions
- pull nightly
- initiate test runs
- build given patch
- Ben Hearsum doing work with Build team on this
- select platforms
- reporting handled by performance testing framework
- litmus reporting
- clean up build directory/create cleaner mirror
- get rid of kruft
- drive performance testing
Reporting Tools
Survey of reporting tools
- litmus
- buildbot
- tinderbox
Report aggregation
- RSS
- Atom
- Microformats
- Security
- passworded version
- irc
- report drill-down
- history
- test details
- logs
- store links in database for detailed analysis
Rollups
- graphserver
- buildbot?
History
- store some set period
- retain important historical milestones / benchmarks
JS Tests
- specialized builds
- buildbot vs tinderbox canned builds
- requires js shell, xpcshell, can't use 'off-the-shelf' versions.
Metrics
- bug density
- per component
- map bugzilla modules
- checkins churns
- lines of code
- performance thresholds
- links on wiki
- top-crashers