Confirmed users
675
edits
Socorro/Hang Processing Proposal: Difference between revisions
Socorro/Hang Processing Proposal (view source)
Revision as of 16:14, 14 August 2012
, 14 August 2012→Migration
(be clear about potential storage issues) |
|||
| Line 33: | Line 33: | ||
* Add Firefox support for submitting minidumps for the Flash processes NOTE: this means that most hang reports will end up with *four* minidumps needing to be processed and stored, instead of the current two. | * Add Firefox support for submitting minidumps for the Flash processes NOTE: this means that most hang reports will end up with *four* minidumps needing to be processed and stored, instead of the current two. | ||
* Add crash-stats frontend support for displaying multiple minidumps stacks | * Add crash-stats frontend support for displaying multiple minidumps stacks | ||
== Socorro Implementation Ramifications == | |||
This is a major change to how crashes are collected, processed and displayed. Changes will have to be made in the backend, middleware and UI components. | |||
=== Collector === | |||
Currently the collector reads the binary minidump as a stream. The collector code will have to be modified to read as many streams al are offered in the crash. The minidumps will have to either be named or enumerated so that they can be stored and retrieved individually. | |||
The size of a given crash will increase. In 2009, 8% of crash submissions failed because of timeouts in the crash data upload. We theorized that these were crashes from people on dial up lines. We need to see what that upload failure rate is in 2012 and project ramifications to the increased crash size. | |||
=== Storage === | |||
The interface for the crash storage sub-system will need to be extended to allow for multiple minidumps. For filesystem storage (collector storage), this will mean a new naming scheme to differentiate the minidumps. For HBase this will have to be coordinated with metrics on storage. We need to look at total storage ramifications: how much will this increase the average size of a crash. Minidump compression has been mentioned. | |||
=== Processor === | |||
The processor will be responsible for feeding all minidumps to MDSW. Having multiple minidumps, if submitted serially, will significantly increase the amount of time that it takes to process a crash. However, since MDSW is invoked as a separate process, it would not be too difficult to run them in parallel. | |||
The existing Hang Pair code will have to be excised from the processor. | |||
Signature generation: a signature is generated from the output of MDSW. With multiple minidumps in a single crash, does the processor make a collection of signatures? | |||
=== Middleware === | |||
The API will need extensions to take into account multiple mindumps. The get crash service is the first example that comes to mind. | |||
=== UI === | |||
Display of the results of multiple MDSW output will require revamping the UI. | |||