Confirmed users
3,816
edits
QA/TDAI/TestDev Presentation 2009-09-11: Difference between revisions
Jump to navigation
Jump to search
QA/TDAI/TestDev Presentation 2009-09-11 (view source)
Revision as of 17:21, 11 September 2009
, 11 September 2009→Fennec Log Parser by Joel Maher & Hans Sebastian
No edit summary |
|||
| Line 19: | Line 19: | ||
== Fennec Log Parser by Joel Maher & Hans Sebastian == | == Fennec Log Parser by Joel Maher & Hans Sebastian == | ||
* Notes and Questions | * Notes and Questions | ||
** In the last few weeks, we have wired it to tinderbox that pulls the real data in from tinderbox running on a cron job | |||
** Intermediate post layer to mitigate the performance issues with the large "new failure" query | |||
** Also have the new failures list - tracks new failures that haven't been seen before. | |||
** Think about other types of queries that we can pull out of this data now that we have it. | |||
** Only showing reftest results - some columns have zeros in them, this has to do with tests crashing on maemo and devices hanging etc. | |||
** mochi* aren't running on maemo | |||
** xpcshell tests are running, but the way we differentiate checks and tests are messing up our counts for the logs. | |||
** Need to figure out the proper build step to process the build log, so that we only upload builds that have the full end to end run. | |||
** <Bob> If you're not going to put results up when you have a device crash how do you track that? | |||
*** Aki and releng crew have some means to track them. What we care about here are tracking new failures on tests. So getting partial results in won't help. As far as tracking the devices, we need to find a better way to flash them more frequently | |||
*** the releng team manages keeping the tests alive. The buildteam uses the buildbot waterfall to track failing devices. | |||
*** it would be an interesting piece of data that the devices are running. | |||
** <Henrik>Mozmill results one day - how difficult is it to use the views of couchdb to do the same thing for Mozmill? | |||
*** It's actually easier. The reporting system is already built into mozmill and so there is no real parsing involved. The per test indexes and views have to be written, which isn't hard but isn't done yet. | |||
** <TODO> Modify the log parser to deal with partial logs | |||
** Are we flushing on every write? | |||
*** Yes, mostly. Something to consider | |||
** When you parse the tinderbox data, what do you get? | |||
*** gets the tinderbox logs - which is everything. If it crashes a cell shows up in tinderbox, but we don't get any test runs. We do get the stdout up to that point in time. So the runs that show zero, did those never attempt to run? No, we are parsing the end of the log file and not the stdout so if that end doesn't show up then we will not see those. | |||
** We should modify that to be more accurate. | |||
*** failure is relative concept...whatmikeal said. | |||
== TopCrash Automated Analysis & Reproduction by Bob Clary == | == TopCrash Automated Analysis & Reproduction by Bob Clary == | ||