Breakpad:Current Implementation: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(google code to github)
 
(7 intermediate revisions by 3 users not shown)
Line 4: Line 4:


= Client Side =
= Client Side =
Currently Breakpad work in Mozilla has only progressed on Win32.  An OS X implementation is available in the Breakpad source.  Linux patches are in progress, but not yet available.
Breakpad crash reporting is currently enabled by default on trunk builds of Firefox on Windows, Mac OS X and Linux.


== Symbol Extraction/Upload ==
== Symbol Extraction/Upload ==
Symbol extraction and upload is done at build time.  There are two targets in the top level Makefile, [http://mxr.mozilla.org/seamonkey/source/Makefile.in#140 buildsymbols] and [http://mxr.mozilla.org/seamonkey/source/Makefile.in#158 uploadsymbols].
Symbol extraction and upload is done at build time.  There are two targets in [http://mxr.mozilla.org/mozilla/source/Makefile.in the top level Makefile], buildsymbols and uploadsymbols.


The |buildsymbols| target finds PDB files associated with EXE and DLL files in $(DIST), and processes them with [http://mxr.mozilla.org/seamonkey/source/toolkit/airbag/tools/win32/ dump_syms.exe] to get a SYM file for each, using the [http://mxr.mozilla.org/seamonkey/source/toolkit/airbag/tools/make_symbol_store.pl make_symbol_store.pl] script.  The SYM files and PDB files are both placed in $(DIST)/crashreporter-symbols/$(BUILDID) in the directory structure used for the Microsoft Symbol Server.  A description of this directory structure can be found [http://google-breakpad.googlecode.com/svn/trunk/src/processor/simple_symbol_supplier.h in the Breakpad source].  The symbols are then compressed into a zip file, in a dated directory alongside the top source directory.
== Exception Handler ==
 
The |uploadsymbols| target simply calls the [http://mxr.mozilla.org/seamonkey/source/toolkit/airbag/tools/upload_symbols.sh upload_symbols.sh] script to upload the symbols using scp.  This script relies on certain environment variables being set:
* AIRBAG_SYMBOL_SERVER  : host to upload symbols to
* AIRBAG_SYMBOL_USER    : username on that host
* AIRBAG_SYMBOL_PATH    : path on that host to put symbols in
 
The symbols are copied to the symbol server and unzipped into the specified path.


== Exception Handler ==
The exception handler is set in <code>XRE_Main</code>, so any app using that as an entry point can use it simply by setting some values in application.ini.  [http://bonsai.mozilla.org/cvsblame.cgi?file=/mozilla/browser/app/application.ini&rev=1.10&mark=55-59#55  See the Firefox application.ini] for an example.


The exception handler is set [http://mxr.mozilla.org/seamonkey/source/toolkit/xre/nsAppRunner.cpp#2209 in XRE_Main] by calling [http://mxr.mozilla.org/seamonkey/source/toolkit/airbag/nsAirbagExceptionHandler.cpp#203 SetExceptionHandler()].  The exception handler is not actually installed unless the MOZ_AIRBAG environment variable is set to a non-zero value.  The code in XRE_Main then sets some additional data (currently Vendor, ProductName, Version and BuildID) by calling [http://mxr.mozilla.org/seamonkey/source/toolkit/airbag/nsAirbagExceptionHandler.cpp#342 AnnotateCrashReport].  Once a profile has been found, a "minidumps" directory is located or created in the profile directory, and [http://mxr.mozilla.org/seamonkey/source/toolkit/xre/nsAppRunner.cpp#2622 this path is passed] to [http://mxr.mozilla.org/seamonkey/source/toolkit/airbag/nsAirbagExceptionHandler.cpp#261 SetMinidumpPath()] as the destination for minidump files from the exception handler.
There is an API for adding extra data to the crash report, implemented using the [http://mxr.mozilla.org/mozilla/source/toolkit/crashreporter/nsICrashReporter.idl nsICrashReporter] interface.  Note that the server must be modified to handle additional data.


Upon catching an exception, the handler writes the additional data set using AnnotateCrashReport out to a file with the same base name as the minidump, but with a ".extra" extension. [http://mxr.mozilla.org/seamonkey/source/toolkit/airbag/nsAirbagExceptionHandler.cpp#78 attempts to execute crashreporter.exe] from the same directory as the current executable.  The full path to the minidump file is passed in as a command line parameter.  The application then exits.
Upon handling an exception, a minidump is written, and then the data added using <code>annotateCrashReport</code> is written to a file with a .extra extension alongside the minidump.  The crash reporter client is then launched with the minidump filename as the sole parameter.


== Crash Reporter Client ==
== Crash Reporter Client ==


The [http://mxr.mozilla.org/seamonkey/source/toolkit/airbag/client/ crash reporter client] is currently capable of enabling/disabling crash reporting, and submitting minidumps to a server for processing.
The [http://mxr.mozilla.org/mozilla/source/toolkit/crashreporter/ crash reporter client] will display the contents of the .extra file for the user to view, and allow the user to submit a report.  The client can also restart the crashed application. The client stores reports in a directory structure [http://kb.mozillazine.org/Breakpad#Location_of_crash_reports explained here].  Reports that have not yet been submitted are stored in <code>pending</code>, and successful reports have a text file with information about the report placed in <code>submitted</code>.
 
If run with no parameters, the client simply allows the user to enable or disable crash reporting.
 
If run with parameters, the client first [http://mxr.mozilla.org/seamonkey/source/toolkit/airbag/client/crashreporter_win.cpp#230 checks the Windows registry] to see if it has been disabled.  It looks for the value "Enabled" under the key "Software\Mozilla\Crash Reporter" in both the HKEY_LOCAL_MACHINE and HKEY_CURRENT_USER roots.  If the value exists and is not set to "1", then the client simply exits.  If the value exists and is set to "1" the client proceeds to [http://mxr.mozilla.org/seamonkey/source/toolkit/airbag/client/crashreporter_win.cpp#318 upload the minidump file].  If the minidump is uploaded successfully, then the file is deleted locally.  The client then displays a message box indicating whether the minidump was successfully submitted or not.
 
The crash reporter client follows the lead of the updater software and is potentially localizable by translating strings in the [http://mxr.mozilla.org/seamonkey/source/toolkit/airbag/client/crashreporter.ini crashreporter.ini] file.  Unfortunately, currently the URL of the server to send crash reports to is also stored in the crashreporter.ini file.
 
Win32 trunk nightlies currently include the airbag code, but it's disabled at runtime by default.  To test it, you will need to set the environment variable <code>MOZ_AIRBAG=1</code>. Crash reports are currently collected at:
http://mavra.perilith.com/~luser/airbag-collector/list.py


= Server Side =
= Server Side =


The server side consists of several parts.
== Symbol Store ==
The symbol store is simply a flat file store with a directory structure mirroring that of the Microsoft Symbol Server.  [[#Symbol Extraction/Upload|See above]] for a link to the Breakpad source describing the structure.  The benefit of this is that if we preserve PDB files in this same directory structure, and make the symbol store accessible via HTTP, we can make Windows release builds debuggable.  Building on this, in the future, we may even be able to provide [http://zenit.senecac.on.ca/wiki/index.php/Mozilla_Source_and_Symbol_Server source level debugging of release builds].
You can see [http://mavra.perilith.com/~luser/airbag-symbols/ an example symbol store here].  Note that there is no provision for deleting outdated symbols currently, although the symbol upload script does create a text file listing the symbols contained in each build.
== Crash Report Collector ==
Our server code is named [http://code.google.com/p/socorro/ Socorro] and lives in its own Google code project.  It is implemented in Python on top of [http://pylonshq.com/ Pylons].  Rob Sayre has [http://blog.mozilla.com/rob-sayre/2007/02/23/a-breakpad-server/ some more info on his blog].
The collector currently uses a [https://bugzilla.mozilla.org/attachment.cgi?id=250492 MySQL database] to store crash reports.  Crash reports are processed as they are submitted, which is probably a poor choice as they may take over 10 seconds when provided with full symbols.  Minidumps are passed to the [http://google-breakpad.googlecode.com/svn/trunk/src/processor/minidump_stackwalk.cc minidump_stackwalk program] provided by Breakpad, which supports a "-m" command line flag to get [http://google-breakpad.googlecode.com/svn/trunk/src/processor/testdata/minidump2.stackwalk.machine_readable.out machine-readable output].
== Crash Report Viewer ==


The crash report viewer is probably the least-well-implemented portion of the entire system.  It is very minimal, allowing one to see all crash reports that have been submitted, and to click on a particular crash report to see the details, including a stack trace.  You can [http://mavra.perilith.com/~luser/airbag-collector/list.py?id=19 see a sample here]. Socorro does something similar right now.
Information about the server can be found at [https://github.com/mozilla/socorro/ the Socorro Github page].  The Mozilla instance is located at http://crash-stats.mozilla.com/ .

Latest revision as of 06:52, 24 June 2013

« back to Breakpad main page

This is a summary of the current state of Breakpad integration into Mozilla.

Client Side

Breakpad crash reporting is currently enabled by default on trunk builds of Firefox on Windows, Mac OS X and Linux.

Symbol Extraction/Upload

Symbol extraction and upload is done at build time. There are two targets in the top level Makefile, buildsymbols and uploadsymbols.

Exception Handler

The exception handler is set in XRE_Main, so any app using that as an entry point can use it simply by setting some values in application.ini. See the Firefox application.ini for an example.

There is an API for adding extra data to the crash report, implemented using the nsICrashReporter interface. Note that the server must be modified to handle additional data.

Upon handling an exception, a minidump is written, and then the data added using annotateCrashReport is written to a file with a .extra extension alongside the minidump. The crash reporter client is then launched with the minidump filename as the sole parameter.

Crash Reporter Client

The crash reporter client will display the contents of the .extra file for the user to view, and allow the user to submit a report. The client can also restart the crashed application. The client stores reports in a directory structure explained here. Reports that have not yet been submitted are stored in pending, and successful reports have a text file with information about the report placed in submitted.

Server Side

Information about the server can be found at the Socorro Github page. The Mozilla instance is located at http://crash-stats.mozilla.com/ .

Retrieved from "https://wiki.mozilla.org/index.php?title=Breakpad:Current_Implementation&oldid=669479"