CrashKill: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
 
(303 intermediate revisions by 28 users not shown)
Line 1: Line 1:
__NOTOC__
==General Agenda==
==General Agenda==
Crashkill team works to track and escalate stability issues for both Desktop and Mobile products. Our activities include....
* Track general stability for all our releases and development channels.
* Triage Top Crashes to find owners, set priorities, and identify vender outreach issues.
* Triage Top Crashes to find owners, set priorities, and identify vender outreach issues.
* Ensure that crash bugs are logged and updated with new information.
* Identify top tools issues so we can find & fix faster.
* Identify top tools issues so we can find & fix faster.
* Identify the first thing we want to do to increase developer awareness and commitment to fixing top crashers.
* Identify the first thing we want to do to increase developer awareness and commitment to fixing top crashers.
* Identify the first thing we want to automate.
* Identify the first thing we want to automate.
* Working with the Socorro team to get features on the roadmap that help us find and diagnose crash bugs better.
* Set [[CrashKill/Topcrash|topcrash threshold criteria]].
==Top Crashes==
There are a number of reports that we triage regularly in order to stay on top of issues, most notably the topcrash reports as linked in the front page of Socorro crash-stats (see top links in the Reports & Tools section of this page). Finding reproducible steps and helping us make them actionable will have a direct and in many cases significant impact on our users due to the volume involved. Not all bugs are fixes in our code. Many are 3rd party issues that can be resolved through outreach and/or blocklisting.
== Reports & Tools ==
Mozilla collects crash data which can be easily viewed using Socorro (a web interface to the crash reports). We use Socorro and its numerous reports to monitor stability rates on a daily basis.
* https://crash-stats.mozilla.com/products/Firefox (Firefox Desktop)
* https://crash-stats.mozilla.com/products/FennecAndroid (Firefox for Android)
* https://crash-stats.mozilla.com/products/B2G (Firefox OS)
The [http://arewestableyet.com/ "Are We Stable Yet?" Stability Dashboard] tries to collect all important numbers in one place, with colored indicators for hinting at what they mean.


== Links ==


=== Bug lists ===
We also have a list of additional reports that we have developed to help us understand certain crashes and trends. We plan over time to integrate these into Socorro and most of these are on the roadmap already.
* [http://bit.ly/3K8mhq The CrashKill Bugs]  <small>(Status Whiteboard: [crashkill])</small>
* {{Crash-ThirdParty| [http://bit.ly/1TlEts CrashKill Bugs caused by third parties] }} <small>(Status Whiteboard: [crashkill-thirdparty])</small>
* {{Crash-Blocklist| [http://bit.ly/33K46O Crashkill Bugs where we're considering blocking the addon/dll] }}  <small>(Status Whiteboard: [crashkill-block])</small>
* {{Crash-Fixed| [http://bit.ly/2aFggC CrashKill Bugs where we've provided a fix] of some kind, even if just a wallpaper or skidmark }}  <small>(Status Whiteboard: [crashkill-fix])</small>
* {{Crash-Debug| [http://bit.ly/3qmv2U CrashKill Bugs where we've provide a patch only to help debug the problem] }} <small>(Status Whiteboard: [crashkill-debug])</small>


* [http://bit.ly/Uf9J5 List] of bugs tracking metrics improvements for CrashKill efforts. <small>(Status Whiteboard: [crashkill-metrics])</small>
* https://crash-analysis.mozilla.com/chofmann/
* nominate bugs for the 3.6-only DLL blocklist by adding a dependency to {{bug|525103}}
* https://crash-analysis.mozilla.com/rkaiser/ (see [https://crash-analysis.mozilla.com/rkaiser/0000.overview.html Overview] for always-current links)
** Explosiveness, Weekly bug stats, FxOS crashes by B2G version, Flash topcrashes by version, and other specialty reports
* https://crash-analysis.mozilla.com/bsmedberg/
** https://crash-analysis.mozilla.com/bsmedberg/flash-summary.html (summary breakdown of crashes and hangs by Flash version)
** [[CrashKill/Flash Breakdown Data]]
* Reporting Malware (external link): https://www.techsupportalert.com/content/how-report-malware-or-false-positives-multiple-antivirus-vendors.htm


=== Analysis ===
== Planning ==
* [http://crash-stats.mozilla.com/topcrasher/byversion/Firefox/3.5.5/7 Firefox 3.5.5 top crashes in the last week]
* [[CrashKill/Plan]]
* [http://crash-stats.mozilla.com/topcrasher/byversion/Firefox/3.5.4/7 Firefox 3.5.4 top crashes in the last week]
* We are actively working with the Socorro team on a list of [https://wiki.mozilla.org/CrashKill/Plan/Priorities priorities] for getting features into Socorro that will help crash analysis. We update the list quarterly.
* [http://crash-stats.mozilla.com/topcrasher/byversion/Firefox/3.6b3/7 Firefox 3.6 beta 3 top crashes in the last week]
* [http://people.mozilla.com/crash_analysis/ Correlation Reports]
* [[CrashKill/Signatures|Crash signatures by module]]


* [http://bit.ly/8elnaZ Outreach Bugs] tracking our efforts to contact add on vendors who have products causing crashes that we want to kill.
==Work Weeks==
*[[CrashKill/WorkWeek2012|Stability Work Week 2012]] (Notes)
*[[CrashKill/StabilityWeek2013|Stability Week 2013]] (Notes)


* [http://www.squarefree.com/2009/10/15/531/ How to sort a bug list in topcrash order]
==Weekly Crashkill Meetings==
* [[CrashKill/Mac Flash Identifiers|Mac Flash Identifiers]] - Turns "025105C956638D665850591768FB743D0" into "Flash 10.0.32.18 on PPC"
Up to the end of 2014, weekly meetings were held to talk about stability issues across releases and channels. Those now have been replaced by mailing list communication and ad-hoc meetings before and after releases.


==Meeting logistics==
[[CrashKill/Meetings|Archived info about meetings]]
* 650-903-0800 x92 Conf# 8605 (US/INTL)
* 1-800-707-2533 (pin 369) Conf# 8605 (US)
* *1 to unmute yourself
* join irc.mozilla.org #planning for back channel
* Mountain View conference room "Warp Core"


==Meeting notes==
==Old Stuff==
* [[CrashKill/2009-11-23|Monday, November 23, 3pm PST]]
* [[CrashKill/Historical| Old links, queries and info]]
* [[CrashKill/2009-11-16|Monday, November 16, 3pm PST]]
* [[CrashKill/2009-11-09|Monday, November 9, 3pm PDT]]
* [[CrashKill/2009-11-02|Monday, November 2, 3pm PDT]]
* [[CrashKill/2009-10-26|Monday, October 26, 3pm PDT]]
* [[CrashKill/2009-10-19|Monday, October 19, 3pm PDT]]
* [[CrashKill/2009-10-12|Monday, October 12, 3pm PDT]]
* [[CrashKill/2009-10-05|Monday, October 5, 3pm PDT]]
* [[CrashKill/2009-09-28|Monday, September 28, 3pm PDT]]

Latest revision as of 20:10, 22 December 2014

General Agenda

Crashkill team works to track and escalate stability issues for both Desktop and Mobile products. Our activities include....

  • Track general stability for all our releases and development channels.
  • Triage Top Crashes to find owners, set priorities, and identify vender outreach issues.
  • Ensure that crash bugs are logged and updated with new information.
  • Identify top tools issues so we can find & fix faster.
  • Identify the first thing we want to do to increase developer awareness and commitment to fixing top crashers.
  • Identify the first thing we want to automate.
  • Working with the Socorro team to get features on the roadmap that help us find and diagnose crash bugs better.
  • Set topcrash threshold criteria.

Top Crashes

There are a number of reports that we triage regularly in order to stay on top of issues, most notably the topcrash reports as linked in the front page of Socorro crash-stats (see top links in the Reports & Tools section of this page). Finding reproducible steps and helping us make them actionable will have a direct and in many cases significant impact on our users due to the volume involved. Not all bugs are fixes in our code. Many are 3rd party issues that can be resolved through outreach and/or blocklisting.

Reports & Tools

Mozilla collects crash data which can be easily viewed using Socorro (a web interface to the crash reports). We use Socorro and its numerous reports to monitor stability rates on a daily basis.

The "Are We Stable Yet?" Stability Dashboard tries to collect all important numbers in one place, with colored indicators for hinting at what they mean.


We also have a list of additional reports that we have developed to help us understand certain crashes and trends. We plan over time to integrate these into Socorro and most of these are on the roadmap already.

Planning

  • CrashKill/Plan
  • We are actively working with the Socorro team on a list of priorities for getting features into Socorro that will help crash analysis. We update the list quarterly.

Work Weeks

Weekly Crashkill Meetings

Up to the end of 2014, weekly meetings were held to talk about stability issues across releases and channels. Those now have been replaced by mailing list communication and ad-hoc meetings before and after releases.

Archived info about meetings

Old Stuff