Quantum/Release Criteria

From MozillaWiki
Jump to: navigation, search
  • QRC = Quantum Release Criteria
  • QF = Quantum Flow

Bug Workflow

A QRC bug is a bug that indicates Firefox is below some performance target on a QRC benchmark or metric.

QRC bug states:

  1. "[QRC][QRC_NeedAnalysis]" and unassigned (ready for profile analysis)
  2. "[QRC][QRC_NeedAnalysis]" and assigned to an engineer analyzing the profile
  3. "[QRC][QRC_Analyzed][qf:p1]" and unassigned (ready to be fixed by a QF engineer)
  4. "[QRC][QRC_Analyzed][qf:p1]" and assigned to a QF engineer fixing the bug
  1. Testers will:
    1. File a new bug for any QRC test failures that don't already have a bug.
    2. Make the new bug block one of the QRC meta bugs.
    3. Attach a link to a Gecko Profiler profile.
    4. Add the whiteboard tag "[QRC][QRC_NeedAnalysis]".
  2. Profile analysis engineers on the "Profiler Overflow" list:
    1. Select a bug from the "Bugs Ready To Be Analyzed" list below.
    2. Assign the bug to yourself while you are analyzing it.
    3. If the bug is invalid or not actionable, then needinfo the reporter or close the bug.
      • There might not be obvious, visible problems. The profile analysis does not necessarily require the profile analysis engineer to reproduce the test case, just identify if there are obvious hot spots in the profile.
    4. If you can't identify the root cause, then unassign yourself. Another profile analysis engineer will pick it from the QRC_NeedAnalysis list, so leave any notes you have or needinfo someone who may have domain knowledge relevant to the profile.
    5. If you have identified the root cause, then:
      1. Change the whiteboard tag from "[QRC_NeedAnalysis]" to "[QRC_Analyzed][qf:p1]" (so the bug will be seen by QF engineers).
      2. Move the bug to an appropriate Bugzilla component.
      3. Unassign yourself.

Resources on "How to do Gecko Profiling"

If you have questions about using the Gecko Profiler, operators are ready standing by to help in the #flow IRC channel. :)

Ehsan's videos on how to analyze Gecko profiles

Mike's deep dive videos analyzing profiles on real bugs

Bugs Ready To Be Analyzed

QRC_NeedAnalysis bugs:

Full Query
ID Priority Summary Whiteboard Assigned to
1363352 -- Amazon homepage start: time to first nonblank is slow [QRC][QRC_NeedAnalysis]

1 Total; 1 Open (100%); 0 Resolved (0%); 0 Verified (0%);

Bugs Now Being Analyzed

QRC_NeedAnalysis bugs:

Full Query
ID Priority Summary Whiteboard Assigned to Last change time
1371641 -- YouTube navigation start profile [QRC][QRC_NeedAnalysis] Matt Woodrow (:mattwoodrow) 2017-10-04T20:34:51Z
1363364 -- STR Amazon search first result [QRC][QRC_NeedAnalysis] Naveed Ihsanullah [:naveed] 2018-03-27T21:23:26Z
1362039 P3 Facebook home navigation start [QRC][QRC_NeedAnalysis] Randell Jesup [:jesup] 2018-03-05T17:18:28Z
1394761 P3 Hasal input latency case facebook_ail_scroll_home_1_txt regressed median from 22ms to 44ms on 7/29 [QRC][QRC_NeedAnalysis][qf:p3] Sean Stangl [:sstangl] 2017-10-07T00:03:29Z

4 Total; 4 Open (100%); 0 Resolved (0%); 0 Verified (0%);

Bugs Ready To Be Fixed

QRC_Analyzed bugs that are ready to be fixed by a QF engineer:

Full Query
ID Priority Summary Whiteboard Assigned to
1375346 P3 Facebook group page load [qf:p1][QRC][QRC_Analyzed]

1 Total; 1 Open (100%); 0 Resolved (0%); 0 Verified (0%);


Responsibilities of the Quantum Profiling Pool

QRC queries: