Latest revision as of 22:52, 6 December 2017

QRC = Quantum Release Criteria
QF = Quantum Flow

Bug Workflow

A QRC bug is a bug that indicates Firefox is below some performance target on a QRC benchmark or metric.

QRC bug states:

"[QRC][QRC_NeedAnalysis]" and unassigned (ready for profile analysis)
"[QRC][QRC_NeedAnalysis]" and assigned to an engineer analyzing the profile
"[QRC][QRC_Analyzed][qf:p1]" and unassigned (ready to be fixed by a QF engineer)
"[QRC][QRC_Analyzed][qf:p1]" and assigned to a QF engineer fixing the bug

Testers will:
1. File a new bug for any QRC test failures that don't already have a bug.
2. Make the new bug block one of the QRC meta bugs.
3. Attach a link to a Gecko Profiler profile.
4. Add the whiteboard tag "[QRC][QRC_NeedAnalysis]".
Profile analysis engineers on the "Profiler Overflow" list:
1. Select a bug from the "Bugs Ready To Be Analyzed" list below.
2. Assign the bug to yourself while you are analyzing it.
3. If the bug is invalid or not actionable, then needinfo the reporter or close the bug.
  - There might not be obvious, visible problems. The profile analysis does not necessarily require the profile analysis engineer to reproduce the test case, just identify if there are obvious hot spots in the profile.
4. If you can't identify the root cause, then unassign yourself. Another profile analysis engineer will pick it from the QRC_NeedAnalysis list, so leave any notes you have or needinfo someone who may have domain knowledge relevant to the profile.
5. If you have identified the root cause, then:
  1. Change the whiteboard tag from "[QRC_NeedAnalysis]" to "[QRC_Analyzed][qf:p1]" (so the bug will be seen by QF engineers).
  2. Move the bug to an appropriate Bugzilla component.
  3. Unassign yourself.

Resources on "How to do Gecko Profiling"

If you have questions about using the Gecko Profiler, operators are ready standing by to help in the #flow IRC channel. :)

Ehsan's videos on how to analyze Gecko profiles

Mike's deep dive videos analyzing profiles on real bugs

Bugs Ready To Be Analyzed

QRC_NeedAnalysis bugs:

Full Query

ID	Priority	Summary	Whiteboard	Assigned to
1363352	--	Amazon homepage start: time to first nonblank is slow	[QRC][QRC_NeedAnalysis]
1394761	P3	Hasal input latency case facebook_ail_scroll_home_1_txt regressed median from 22ms to 44ms on 7/29	[QRC][QRC_NeedAnalysis]

2 Total; 2 Open (100%); 0 Resolved (0%); 0 Verified (0%);

Bugs Now Being Analyzed

QRC_NeedAnalysis bugs:

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);

Bugs Ready To Be Fixed

QRC_Analyzed bugs that are ready to be fixed by a QF engineer:

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);

Links

Responsibilities of the Quantum Profiling Pool

QRC queries:

@@ Line 21: / Line 21: @@
 ## Assign the bug to yourself while you are analyzing it.
 ## If the bug is invalid or not actionable, then needinfo the reporter or close the bug.
+##* There might not be obvious, visible problems. The profile analysis does not necessarily require the profile analysis engineer to reproduce the test case, just identify if there are obvious hot spots in the profile.
 ## If you can't identify the root cause, then unassign yourself. Another profile analysis engineer will pick it from the QRC_NeedAnalysis list, so leave any notes you have or needinfo someone who may have domain knowledge relevant to the profile.
 ## If you have identified the root cause, then:
@@ Line 26: / Line 27: @@
 ### Move the bug to an appropriate Bugzilla component.
 ### Unassign yourself.
+= Resources on "How to do Gecko Profiling"=
+If you have questions about using the Gecko Profiler, operators are ready standing by to help in the #flow IRC channel. :)
+'''Ehsan's videos on how to analyze Gecko profiles'''
+* [https://air.mozilla.org/gecko-profiler-introduction Introduction to Gecko profiler]
+* [https://air.mozilla.org/gecko-and-native-profiler Introduction to Gecko and Native profiler]
+'''Mike's deep dive videos analyzing profiles on real bugs'''
+* [https://air.mozilla.org/search/?q=joy+of+profiling Search for "''Joy of Profiling''" on Air Mozilla. '''Effective Dec 4, 2017 - 15 episodes''']
+* [https://air.mozilla.org/the-joy-of-profiling-pilot The Joy of Profiling - Pilot]
+* [https://air.mozilla.org/the-joy-of-profiling-pilot-2 The Joy of Profiling - Pilot 2]
+* [https://air.mozilla.org/the-joy-of-profiling-episode-1-20170519 The Joy of Profiling - Episode 1 Posted on May 19/2017]
+* [https://air.mozilla.org/the-joy-of-profiling-episode-2 The Joy of Profiling - Episode 2 Posted on June 10/2017]
+* [https://air.mozilla.org/the-joy-of-profiling-episode-3 The Joy of Profiling - Episode 3 Posted on June 17/2017]
+* [https://air.mozilla.org/the-joy-of-profiling-episode-4 The Joy of Profiling - Episode 4 Posted on June 24/2017]
+* [https://air.mozilla.org/the-joy-of-profiling-episode-5 The Joy of Profiling - Episode 5 Posted on July 8/2017]
+* [https://air.mozilla.org/the-joy-of-profiling-episode-6 The Joy of Profiling - Episode 6 Posted on July 15/2017]
 = Bugs Ready To Be Analyzed =
@@ Line 32: / Line 52: @@
 <bugzilla> {
-   "status_whiteboard": "[QRC_NeedAnaly",
+   "status_whiteboard": "[QRC_NeedAnalysis]",
    "assigned_to": "nobody@mozilla.org",
    "resolution": "---",
@@ Line 51: / Line 71: @@
    "v2": "nobody@mozilla.org",
    "resolution": "---",
-   "include_fields": "id, priority, summary, whiteboard, assigned_to",
+   "include_fields": "id, priority, summary, whiteboard, assigned_to, last_change_time",
    "order": "assigned_to"
 } </bugzilla>
@@ Line 66: / Line 86: @@
 } </bugzilla>
-QRC_Analyzed bugs that are missing the [qf:p1] whiteboard tag:
+= Links =
-<bugzilla> {
-  "status_whiteboard": "[QRC_Analy",
-  "f1": "status_whiteboard",
-  "o1": "notsubstring",
-  "v1": "[qf:p1]",
-  "resolution": "---",
-  "include_fields": "id, priority, summary, whiteboard, assigned_to",
-  "order": "id"
-} </bugzilla>
-= Links =
+[[Responsibilities of the Quantum Profiling Pool]]
 QRC queries:
-* [https://bugzilla.mozilla.org/buglist.cgi?resolution=---&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=%5BQRC_NeedAnalysis%5D All QRC_NeedAnalysis bugs]
+* [https://bugzilla.mozilla.org/buglist.cgi?status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=%5BQRC_NeedAnalysis%5D All QRC_NeedAnalysis bugs]
-* [https://bugzilla.mozilla.org/buglist.cgi?resolution=---&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=%5BQRC_Analyzed%5D All QRC_Analyzed bugs]
+* [https://bugzilla.mozilla.org/buglist.cgi?status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=%5BQRC_Analyzed%5D All QRC_Analyzed bugs]
-* [https://bugzilla.mozilla.org/buglist.cgi?resolution=---&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=%5BQRC%5D All QRC bugs]
+* [https://bugzilla.mozilla.org/buglist.cgi?status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=%5BQRC%5D All QRC bugs]
 * [https://bugzilla.mozilla.org/buglist.cgi?short_desc=Quantum%20Release%20Criteria&resolution=---&query_format=advanced&short_desc_type=allwordssubstr QRC meta bugs]
 * [https://bugzilla.mozilla.org/buglist.cgi?f1=status_whiteboard&o1=anywordssubstr&resolution=---&emailtype1=exact&o2=nowordssubstr&query_format=advanced&emailassigned_to1=1&f2=status_whiteboard&longdesc=https%3A%2F%2Fperfht.ml%2F&email1=nobody%40mozilla.org&v1=%5Bqf%3Ainvestigate%2C%5Bqf%3Ap1%5D&v2=%5BQRC_Analy&longdesc_type=allwordssubstr QF bugs that have profiles but are unassigned]

Quantum/Release Criteria: Difference between revisions