MozillaWiki

Quantum/Release Criteria: Difference between revisions

Page Discussion

Quantum/Release Criteria (view source)

Revision as of 22:52, 6 December 2017

976 bytes added ,  6 December 2017
→‎Bugs Ready To Be Fixed: Only show bugs marked ready to be fixed
(Bugs that block QRC meta bugs and have profiles but are unassigned)
(→‎Bugs Ready To Be Fixed: Only show bugs marked ready to be fixed)
 
(22 intermediate revisions by 5 users not shown)
Line 3: Line 3:


= Bug Workflow =
= Bug Workflow =
A '''QRC''' bug is a bug that indicates Firefox is below some performance target on a QRC benchmark or metric.


QRC bug states:
QRC bug states:
# '''"[QRC_NeedAnalysis]"''' and '''unassigned''' (ready for profile analysis)
# '''"[QRC][QRC_NeedAnalysis]"''' and '''unassigned''' (ready for profile analysis)
# '''"[QRC_NeedAnalysis]"''' and '''assigned''' to an engineer analyzing the profile
# '''"[QRC][QRC_NeedAnalysis]"''' and '''assigned''' to an engineer analyzing the profile
# '''"[QRC_Analyzed][qf]"''' and '''unassigned''' (ready for QF triage)
# '''"[QRC][QRC_Analyzed][qf:p1]"''' and '''unassigned''' (ready to be fixed by a QF engineer)
# '''"[QRC_Analyzed][qf:"''' and '''assigned''' to an engineer fixing the bug
# '''"[QRC][QRC_Analyzed][qf:p1]"''' and '''assigned''' to a QF engineer fixing the bug


# Testers will:
# Testers will:
## File a new bug for any QRC test failures that don't already have a bug.
## File a new bug for any QRC test failures that don't already have a bug.
## Make the new bug block one of the [https://bugzilla.mozilla.org/buglist.cgi?short_desc=Quantum%20Release%20Criteria&resolution=---&query_format=advanced&short_desc_type=allwordssubstr QRC meta bugs].
## Make the new bug block one of the [https://bugzilla.mozilla.org/showdependencytree.cgi?id=1370336&maxdepth=1&hide_resolved=1 QRC meta bugs].
## Attach a [https://perf-html.io/ Gecko Profiler] profile.
## Attach a link to a [https://perf-html.io/ Gecko Profiler] profile.
## Add the whiteboard tag "[QRC_NeedAnalysis]".
## Add the whiteboard tag "[QRC][QRC_NeedAnalysis]".
# Engineers on the [https://docs.google.com/spreadsheets/u/1/d/1KsoE7QcGZPCXJ6GfqzDiuX4QrnBOjG42Kd12wFCsV-s/ "Profiler Overflow" list]:
# Profile analysis engineers on the [https://docs.google.com/spreadsheets/u/1/d/1KsoE7QcGZPCXJ6GfqzDiuX4QrnBOjG42Kd12wFCsV-s/ "Profiler Overflow" list]:
## Select a bug from the "Bugs Ready To Be Analyzed" list below.
## Select a bug from the "Bugs Ready To Be Analyzed" list below.
## If the bug has no profile is attached, then needinfo the reporter and select a new bug.
## Assign the bug to yourself while you are analyzing it.
## Assign the bug to yourself while you are analyzing it.
## If you can't identify the root cause, then unassign yourself and select a new bug. Another engineer will pick it, so leave any useful notes or needinfo someone who may have domain knowledge relevant to the profile.
## If the bug is invalid or not actionable, then needinfo the reporter or close the bug.
## Once you have identified the root cause, then:
##* There might not be obvious, visible problems. The profile analysis does not necessarily require the profile analysis engineer to reproduce the test case, just identify if there are obvious hot spots in the profile.
### Change the whiteboard tag from "[QRC_NeedAnalysis]" to "[QRC_Analyzed][qf]" (so the bug will be seen by Quantum Flow triage).
## If you can't identify the root cause, then unassign yourself. Another profile analysis engineer will pick it from the QRC_NeedAnalysis list, so leave any notes you have or needinfo someone who may have domain knowledge relevant to the profile.
## If you have identified the root cause, then:
### Change the whiteboard tag from "[QRC_NeedAnalysis]" to "[QRC_Analyzed][qf:p1]" (so the bug will be seen by QF engineers).
### Move the bug to an appropriate Bugzilla component.
### Move the bug to an appropriate Bugzilla component.
### Unassign yourself.
### Unassign yourself.


= Bugs Ready To Be Analyzed =
= Resources on "How to do Gecko Profiling"=


QRC bugs that are not QRC_Analyzed:
If you have questions about using the Gecko Profiler, operators are ready standing by to help in the #flow IRC channel. :)


<bugzilla> {
'''Ehsan's videos on how to analyze Gecko profiles'''
  "status_whiteboard": "[QRC",
* [https://air.mozilla.org/gecko-profiler-introduction Introduction to Gecko profiler]
  "f1": "status_whiteboard",
* [https://air.mozilla.org/gecko-and-native-profiler Introduction to Gecko and Native profiler]
  "o1": "notsubstring",
  "v1": "[QRC_Analyzed]",
  "resolution": "---",
  "include_fields": "id, priority, summary, whiteboard, assigned_to",
  "order": "id"
} </bugzilla>


QF bugs that have profiles but are unassigned:
'''Mike's deep dive videos analyzing profiles on real bugs'''
* [https://air.mozilla.org/search/?q=joy+of+profiling Search for "''Joy of Profiling''" on Air Mozilla. '''Effective Dec 4, 2017 - 15 episodes''']
* [https://air.mozilla.org/the-joy-of-profiling-pilot The Joy of Profiling - Pilot]
* [https://air.mozilla.org/the-joy-of-profiling-pilot-2 The Joy of Profiling - Pilot 2]

* [https://air.mozilla.org/the-joy-of-profiling-episode-1-20170519 The Joy of Profiling - Episode 1 Posted on May 19/2017]

* [https://air.mozilla.org/the-joy-of-profiling-episode-2 The Joy of Profiling - Episode 2 Posted on June 10/2017]
* [https://air.mozilla.org/the-joy-of-profiling-episode-3 The Joy of Profiling - Episode 3 Posted on June 17/2017]
* [https://air.mozilla.org/the-joy-of-profiling-episode-4 The Joy of Profiling - Episode 4 Posted on June 24/2017]
* [https://air.mozilla.org/the-joy-of-profiling-episode-5 The Joy of Profiling - Episode 5 Posted on July 8/2017]
* [https://air.mozilla.org/the-joy-of-profiling-episode-6 The Joy of Profiling - Episode 6 Posted on July 15/2017]


<bugzilla> {
= Bugs Ready To Be Analyzed =
  "f1": "status_whiteboard",
  "o1": "anywordssubstr",
  "v1": "[qf:investigate,[qf:p1",
  "f2": "status_whiteboard",
  "o2": "notsubstring",
  "v2": "[QRC_Analyzed]",
  "longdesc": "https:\/\/perfht.ml\/",
  "assigned_to": "nobody@mozilla.org",
  "resolution": "---",
  "include_fields": "id, priority, summary, whiteboard, assigned_to",
  "order": "id"
} </bugzilla>


Bugs that block QRC meta bugs and have profiles but are unassigned:
QRC_NeedAnalysis bugs:


<bugzilla> {
<bugzilla> {
   "f1": "status_whiteboard",
   "status_whiteboard": "[QRC_NeedAnalysis]",
  "o1": "nowordssubstr",
  "v1": "[QRC",
  "longdesc": "https:\/\/perfht.ml\/",
   "assigned_to": "nobody@mozilla.org",
   "assigned_to": "nobody@mozilla.org",
  "blocks":"1339557,1362068,1362625,1363335,1363354,1366777,1366779,1366785,1366786,1366790,1366792,1366793,1366797,1366799,1366801,1366802,1366877,1366881",
   "resolution": "---",
   "resolution": "---",
   "include_fields": "id, priority, summary, whiteboard, assigned_to",
   "include_fields": "id, priority, summary, whiteboard, assigned_to",
   "order": "assigned_to"
   "order": "id"
} </bugzilla>
} </bugzilla>


Line 76: Line 66:
   "f1": "status_whiteboard",
   "f1": "status_whiteboard",
   "o1": "substring",
   "o1": "substring",
   "v1": "[QRC_NeedAnalysis]",
   "v1": "[QRC_NeedAnaly",
   "f2": "assigned_to",
   "f2": "assigned_to",
   "o2": "notequals",
   "o2": "notequals",
   "v2": "nobody@mozilla.org",
   "v2": "nobody@mozilla.org",
   "resolution": "---",
   "resolution": "---",
   "include_fields": "id, priority, summary, whiteboard, assigned_to",
   "include_fields": "id, priority, summary, whiteboard, assigned_to, last_change_time",
   "order": "assigned_to"
   "order": "assigned_to"
} </bugzilla>
} </bugzilla>


= Bugs Ready To Be Assigned =
= Bugs Ready To Be Fixed =


QRC_Analyzed bugs that are ready for QF triage:
QRC_Analyzed bugs that are ready to be fixed by a QF engineer:


<bugzilla> {
<bugzilla> {
   "status_whiteboard": "[QRC_Analyzed]",
   "status_whiteboard": "[QRC_Analyzed],[qf:p1]",
  "f1": "status_whiteboard",
  "o1": "notsubstring",
  "v1": "[qf:",
   "resolution": "---",
   "resolution": "---",
   "include_fields": "id, priority, summary, whiteboard, assigned_to",
   "include_fields": "id, priority, summary, whiteboard, assigned_to",
Line 99: Line 86:
} </bugzilla>
} </bugzilla>


QRC_Analyzed bugs that have been QF triaged and are ready to be fixed:
= Links =


<bugzilla> {
[[Responsibilities of the Quantum Profiling Pool]]
  "status_whiteboard": "[QRC_Analyzed],[qf:",
  "resolution": "---",
  "include_fields": "id, priority, summary, whiteboard, assigned_to",
  "order": "id"
} </bugzilla>
 
= Links =


QRC queries:
QRC queries:
* [https://bugzilla.mozilla.org/buglist.cgi?resolution=---&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=%5BQRC_NeedAnalysis%5D All QRC_NeedAnalysis bugs]
* [https://bugzilla.mozilla.org/buglist.cgi?status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=%5BQRC_NeedAnalysis%5D All QRC_NeedAnalysis bugs]
* [https://bugzilla.mozilla.org/buglist.cgi?resolution=---&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=%5BQRC_Analyzed%5D All QRC_Analyzed bugs]
* [https://bugzilla.mozilla.org/buglist.cgi?status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=%5BQRC_Analyzed%5D All QRC_Analyzed bugs]
* [https://bugzilla.mozilla.org/buglist.cgi?resolution=---&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=%5BQRC%5D All QRC bugs]
* [https://bugzilla.mozilla.org/buglist.cgi?status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=%5BQRC%5D All QRC bugs]
* [https://bugzilla.mozilla.org/buglist.cgi?short_desc=Quantum%20Release%20Criteria&resolution=---&query_format=advanced&short_desc_type=allwordssubstr QRC meta bugs]
* [https://bugzilla.mozilla.org/buglist.cgi?short_desc=Quantum%20Release%20Criteria&resolution=---&query_format=advanced&short_desc_type=allwordssubstr QRC meta bugs]
* [https://bugzilla.mozilla.org/buglist.cgi?f1=status_whiteboard&o1=anywordssubstr&resolution=---&emailtype1=exact&o2=nowordssubstr&query_format=advanced&emailassigned_to1=1&f2=status_whiteboard&longdesc=https%3A%2F%2Fperfht.ml%2F&email1=nobody%40mozilla.org&v1=%5Bqf%3Ainvestigate%2C%5Bqf%3Ap1%5D&v2=%5BQRC_Analy&longdesc_type=allwordssubstr QF bugs that have profiles but are unassigned]
Cpeterson
Confirmed users
3,337

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1172441...1185478"