Performance/Triage: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(→‎Triage process: Added some tips for understanding a performance profile)
No edit summary
Line 1: Line 1:
{{DISPLAYTITLE:Performance Triage}}
{{DISPLAYTITLE:Performance Triage}}
{{message/box|If you have any feedback/suggestions/questions regarding the performance triage process, you can share them in {{matrix|perf-triage}}, or reach out to {{people|davehunt|Dave Hunt}} or {{people|frankd|Frank Doty}}.
= Nomination =
= Nomination =
== Bugzilla ==
== Bugzilla ==
Line 182: Line 185:


== What if things don't go as expected? ==
== What if things don't go as expected? ==
Don't panic! The triage process is not expected to be perfect, and can improve with your feedback. Maybe the result of the triage calculator doesn't feel right, or you find a scenario that's not covered in these guidelines. In this case we recommend that you bring it up in [https://chat.mozilla.org/#/room/#perf-triage:mozilla.org #perf-triage], or consider scheduling a short meeting with some triage leads (you can see some recent leads in the [[#Triage rotation|triage rotation]]). If in doubt, leave a comment on the bug with your thoughts and move on. There's a chance someone will respond, but if not the next performance triage sheriffs may have some other ideas.
Don't panic! The triage process is not expected to be perfect, and can improve with your feedback. Maybe the result of the triage calculator doesn't feel right, or you find a scenario that's not covered in these guidelines. In this case we recommend that you bring it up in {{matrix|perf-triage}}, or consider scheduling a short meeting with some triage leads (you can see some recent leads in the [[#Triage rotation|triage rotation]]). If in doubt, leave a comment on the bug with your thoughts and move on. There's a chance someone will respond, but if not the next performance triage sheriffs may have some other ideas.


== How do I determine the performance impact project flag? ==
== How do I determine the performance impact project flag? ==

Revision as of 11:16, 31 October 2023


{{message/box|If you have any feedback/suggestions/questions regarding the performance triage process, you can share them in #perf-triage, or reach out to Dave Hunt or Frank Doty.

Nomination

Bugzilla

To (re)nominate a bug for triage, set the Performance Impact flag in Bugzilla to ?

This can be found by clicking Show Advanced Fields followed by Set bug flags when entering a new bug:

Bugzilla performance nomination on new bug form.png

Or by expanding the Tracking section when editing an existing bug:

Screenshot 2022-02-24 at 19.53.54.png

GitHub

To nominate a bug for triage, add the Performance label to an issue. This can be done by filing an new issue with the "Performance issue" template:

Screenshot of file a "Performance issue" template on GitHub

Or by opening an existing issue on GitHub and selecting the label from the right-hand bar:

Screenshot of adding a performance label on GitHub

Currently, only the following GitHub repositories are supported:

Queries

Performance triage

Full Query
ID Summary Status
1771902 Compositor CSS animations are not paused in fully occluded windows NEW
1777875 Lenovo Privacy Guard with "Enable this feature when typing passwords" causing excessive UI responsiveness delays (jank) NEW
1931717 [meta] High OOM rate and CC time in YouTube NEW
1941716 On a fresh profile, Negative heap-unclassified on "My Bugs" query on b.m.o NEW
1959837 openstreetmap iD editor - slow zooming NEW
1966492 JS isn't minified in Fenix release builds NEW
1988776 MutationObserver's getReceiverFor is slow when there are a lot of observers NEW
1994817 Firefox CPU usage spikes to 100%+ on some sites, particularly when opening new tabs UNCONFIRMED
2002795 Horrible new tab bar REOPENED
2008927 Choppy/stuttering scrolling with touchpad UNCONFIRMED

10 Total; 10 Open (100%); 0 Resolved (0%); 0 Verified (0%);


Performance triage (pending needinfo)

Full Query
ID Summary Status
1939354 RAM too much UNCONFIRMED
1943419 Fluidd memory leak UNCONFIRMED
1946913 Consume too much memory UNCONFIRMED
1988608 Tabs in Firefox refresh when multitasking UNCONFIRMED
1993778 Firefox causes MacBook battery to drain fast UNCONFIRMED
1994231 Search with bookmark keywords doesn't work when browser's lagging UNCONFIRMED
2008564 UI freezes on long React/DOM sessions despite moderate CPU usage (main-thread JS long tasks) UNCONFIRMED

7 Total; 7 Open (100%); 0 Resolved (0%); 0 Verified (0%);


Recently opened bugs with performance keywords in the summary

Full Query
ID Summary Status
2013232 Enabling JS Execution Tracing (tracing) in profiler disables native frames and memory tracing NEW
2013404 shader-f16 failing to render in WebGPU NEW
2013634 Firefox fails WPT /css/css-view-transitions/capture-with-offscreen-child-translated.html NEW
2013655 Add SimpleURLPattern component for faster compression dictionary use NEW
2013721 Highlight text selection is slow/inconsistent when Windows 11 taskbar is hidden. ASSIGNED
2013749 Testcase creating N empty text nodes is 380x slower in Nightly. NEW
2013750 Testcase creating N empty textnodes and then appending them in bulk is 60x slower in Firefox NEW
2013766 Firefox seems to have memory leaks when watching YouTube lives UNCONFIRMED
2013886 Opening about:support janks the parent process for 600ms getting WebGL info NEW
2013937 firefox memory leak when minimised UNCONFIRMED
2013986 Refactor variations of nsLayoutUtils::IsAncestorFrame NEW
2014160 Cache2 I/O thread 100% CPU due to O(n) linear search in CacheIndexIterator::RemoveRecord with large cache UNCONFIRMED
2014262 Surface 'Translate' option higher in menu when selecting foreign text NEW
2014280 RTC.IceConnectionStateChange callbacks can be slow NEW
2014284 Remove leading slash from NewTabAttributionService endpoint variable NEW
2014367 [Sample] Bug used as sample for testing file bug functionality in Perfherder NEW
2014418 Clear SharedSubResourceCache on memory pressure? ASSIGNED
2014454 Minor but annoying bug in translation triggering. Prompted to translate a JSON API consisting of almost entirely numbers into norwegian. NEW
2014525 CPU goes to 100%, OffscreenCanvasRenderingContext2D.fillText is calling gfxFontGroup::EnsureFontList & gfxFontGroup::AddFamilyToFontList on every call. UNCONFIRMED
2014620 When the translation panel appears and I click the "Not now" button, when I refresh the site, this panel is no longer displayed. NEW
2014639 Autocad Web crashes in firefox if webGPU is enabled. NEW
2014677 Google Slides rendering unusably slow UNCONFIRMED
2014840 Reduce memory allocations for filling simple paths (PathBuilder ops vector, Path) NEW
2014930 ./mach lint -l android-fenix is slow (runs heavy Gradle tasks / build steps) NEW
2014947 Firefox uses excessive memory and freezes entire laptop UNCONFIRMED
2014989 Prototype DOM Tree Translation API NEW
2014998 WebGPU error in Construct 3 on macOS UNCONFIRMED
2015006 Testcase creating Map keys is 4x slower, and iterating them is 2x slower NEW
2015040 Add Performance tests for security layer ASSIGNED
2015083 Text size cannot be increased for Firefox-translations NEW
2015166 Drawing a WebGPU canvas to a 2D canvas does a readback NEW
2015175 Improve Memory summary verbosity and detail inclusion NEW
2015256 Testcases with json operations are 10x-2x slower in Firefox NEW
2015295 The "Translate Selection" dialog needs a "Replace button" NEW
2015300 [wpt-sync] Sync PR 57640 - Don't divide by zero on perfect matches NEW
2015308 Modified codepen demo (https://codepen.io/robbobfrh84/pen/qBZJbYz?editors=0010) is much slower in Firefox NEW
2015309 Modified codepen demo (https://codepen.io/shubniggurath/pen/RNaWvdm?editors=0010) is slower in Firefox NEW
2015312 memory leak in nsCocoaWindow.mm PopupWindow shadowParameters ASSIGNED
2015313 potential memory leak in nsCocoaWindow.mm acceptsFirstMouse ASSIGNED
2015334 memory leak in Cocoa widget touchbar support ASSIGNED
2015337 potential memory leaks in Cocoa widgets text input handling ASSIGNED
2015355 fillRect with createLinearGradient becomes much slower after many iterations UNCONFIRMED
2015358 Battery API doesn't exist UNCONFIRMED
2015364 potential out of bounds write in GPU data arrays on macOS ASSIGNED
2015370 Link previews becomes stuck with constant high CPU usage until process killed UNCONFIRMED
2015414 [Gemini translation] I am using Firefox Android Beta, and it shows that I am logged in; however, the app crashes when I click "Sign Out." UNCONFIRMED
2015473 Expose additional network timings to TRR performance tests NEW
2015508 Improve Gemini and Qwen's memories retrieval performance NEW
2015513 Include a performance tests for ml security layer allow path ASSIGNED
2015602 Significant performance issue with complex SVG at blowfish.page UNCONFIRMED
2015693 Update WebGPU CTS to new version eb9091d141e126fd9479a62aeceb927b73b6af47 from 2026-02-06 06:03:53 ASSIGNED
2015700 Neither about:translations's source text area nor target language selector focused on page load NEW
2015727 Quick action for Translate appears even for words that don’t match exactly NEW
2015734 Translation boxes are not responsive and stack vertically when the window height is reduced NEW
2015794 The l-beam cursor should not be displayed in the translation box NEW
2015796 Ensure reference target observers are removed from labeled descendants of nearest ancestor label in nsLabeledNodeList UNCONFIRMED
2015839 100% CPU usage when playing youtube video, all cores used - windows 11 UNCONFIRMED
2015887 llama.cpp leaks ~500mb for each link preview performed until threads are exhausted NEW
2015935 Adding Adaptive Performance Prefs to Nimbus Manifest ASSIGNED
2015955 Large event jank from synchronous decoding of images in Kotlin-wasm application NEW
2015988 Firefox using too much memory UNCONFIRMED
2016103 Refactor CommonPerformPromiseCombinator to isolate iterator-specific logic ASSIGNED
2016129 [Linux] NTP scrolls up and down fast with productivity widgets enabled and lower vertical view for Firefox window NEW
2016239 Memory leak: Compose SharedElement animation retains destroyed HomeActivity and HomeFragment via InputMethodManager → Recomposer → snapshotInvalidations NEW
2016281 Finish responsiveness work on sidebar NEW
2016304 in the search bar the space bar doesnt work unless i press it twice slowly the it puts a single space UNCONFIRMED

66 Total; 66 Open (100%); 0 Resolved (0%); 0 Verified (0%);


Triage process

Introduction

The goal of performance triage is to identify the extent to which bugs impact the performance of our products, and to move these bugs towards an actionable state. The goal is not to diagnose or fix bugs during triage. We triage bugs that have been nominated for triage and bugs in the Core::Performance component that do not have the performance impact project flag set.

During triage we may do any/all of the following:

  • Request further information from the reporter (such as a profile)
  • Set the performance impact project flag
  • Add performance keywords
  • Move the bug to a more appropriate component

Who is responsible for triage?

Everyone is welcome to take part in triage. By default, everyone on the performance team is enrolled in triage rotation, but we also have participants from outside the team.

How do I schedule a triage meeting?

If you are on triage duty, you will receive an invitation as a reminder to schedule the triage meeting on the shared performance calendar with the nominated sheriffs invited at a time that works for them. The responsibility of scheduling the meeting falls to the lead sheriff. Once a triage meeting has been scheduled, it’s a good idea to remove the reminder event from the calendar to avoid confusion. It’s a good idea to use the shared calendar, as this increases the visibility of the performance triage and allows other members of the team to contribute or observe the process.

What if a sheriff is unavailable?

The rotation script is not perfect, and doesn’t know when people are on PTO or otherwise unavailable. If the lead sheriff is available, it is their responsibility to either schedule the triage with the remaining available sheriff or to identify a suitable substitute for the unavailable sheriff(s). If the lead sheriff is unavailable, this responsibility passes onto the remaining available sheriffs.

How do I run a triage meeting?

The following describes the triage process to follow during the meeting:

  1. Ask if others would prefer you to share your screen. This can be especially helpful for those new to triage.
  2. Open the first triage query to show bugs nominated for triage or in the Core::Performance component without the performance impact project flag set. The bugs are sorted from oldest to newest. For each bug in the list, follow these steps:
    • Bugs that look like tasks that were filed by members of the Performance team will generally need to be moved to the Core::Performance Engineering component.
    • For defects: Determine if the bug is reproducible and actionable. If not, add a needinfo for the reporter asking for more information and move onto the next bug. We have a template that you can modify as needed.
    • For all bugs (including enhancements):
  3. Open the second triage query to show bugs that have open needinfo requests. The bugs are sorted from oldest to newest. For each bug in the list, follow these steps:
    • If the needinfo was set less than 2 weeks ago, move onto the next bug.
    • If the needinfo was set more than 2 weeks ago but less than 2 months ago, consider adding a needinfo for either: another reporter of the issue, someone with access to the appropriate platform(s) to attempt to reproduce the issue, or a relevant subject matter expert.
    • If the open needinfo was set more than 2 months ago, close the bug as inactive. You can modify the inactive bug template as needed.
  4. If time permits, open the third triage query to show recently opened bugs with performance related keywords in the summary. If any of these look like performance bugs, they can either be triaged the same way as bugs in the initial query or they can be nominated for triage in a subsequent meeting.

What if things don't go as expected?

Don't panic! The triage process is not expected to be perfect, and can improve with your feedback. Maybe the result of the triage calculator doesn't feel right, or you find a scenario that's not covered in these guidelines. In this case we recommend that you bring it up in #perf-triage, or consider scheduling a short meeting with some triage leads (you can see some recent leads in the triage rotation). If in doubt, leave a comment on the bug with your thoughts and move on. There's a chance someone will respond, but if not the next performance triage sheriffs may have some other ideas.

How do I determine the performance impact project flag?

The performance impact project flag is used to indicate a bug’s relationship to the performance of our products. It can be applied to all bugs, and not only defects. The triage calculator should be used to help determine the most appropriate value for this flag. In addition to setting the performance impact project flag, make sure to use the “Copy Bugzilla Comment” button and paste this as a comment on the bug.

For more information about what this flag, and it's settings mean see this blog post.

How do I determine the performance keywords?

There are several performance related keywords, which can be helpful to understand how our performance issues are distributed, or whenever there’s a concerted effort to improve a particular aspect of our products. The triage calculator may recommend keywords to set, and by typing “perf:” in the keywords field in Bugzilla, you will see the available options. Select all that apply to the bug.

How do I determine the correct Bugzilla component?

Ideally we would only have bugs in the Core::Performance component that are the responsibility of the engineers in the performance team. For performance bugs to have the best chance of being fixed, it's important to assign them to the correct component. In some cases the correct component will be obvious from the bug summary, description, or steps to reproduce. In other cases, you may need to do a bit more work to identify the component. For example, if there's a profile associated with the bug, you could see where the majority of time is being spent using the category annotations.

How do I read a performance profile?

It's useful to be able to understand a profile generated by the Firefox Profiler, and hopefully someone in the triage meeting will be able to help. If you find an interesting profile, or just want to understand how to use them to analyse a performance problem, we encourage you to post a link to the profile (or bug) in #joy-of-profiling where someone will be happy to help. The profile may even be analysed during one of the regular "Joy of Profiling" open sessions that can be found on the Performance Office Hours calendar.

Triage calculator

The Performance Impact Calculator was developed to assist in identifying and applying the performance impact project flag and performance keywords consistently. If you have feedback or would like to suggest changes to this tool, please share these in the #perf-triage Matrix channel.

Triage rotation

The sheriffs are allocated on a weekly basis, which is published here. The rotation is generated by this script.

Templates

New bug

This template is included in the description for new bugs opened in the Core::Performance component. If a bug is opened in another component and then moved to Core::Performance, this template can be used as needed to request additional information from the reporter.

### Basic information

Steps to Reproduce:


Expected Results:


Actual Results:


---

### Performance recording (profile)

Profile URL:
(If this report is about slow performance or high CPU usage, please capture a performance profile by following the instructions at https://profiler.firefox.com/. Then upload the profile and insert the link here.)

#### System configuration:

OS version:
GPU model:
Number of cores: 
Amount of memory (RAM): 

### More information

Please consider attaching the following information after filing this bug, if relevant:

 - Screenshot / screen recording
 - Anonymized about:memory dump, for issues with memory usage
 - Troubleshooting information: Go to about:support, click "Copy text to clipboard", paste it to a file, save it, and attach the file here.

---

Thanks so much for your help.

Moved to Core::Performance

This bug was moved into the Performance component. Reporter, could you make sure the following information is on this bug?

 - For slowness or high CPU usage, capture a profile with http://profiler.firefox.com/ , upload it and share the link here.
 - For memory usage issues, capture a memory dump from about:memory and attach it to this bug.
 - Troubleshooting information: Go to about:support, click "Copy raw data to clipboard", paste it into a file, save it, and attach the file here.

Thank you.

No longer able to reproduce

This bug doesn’t seem to happen anymore in current versions of Firefox. Please reopen or file a new bug if you see it again.

No response from reporter

With no answer from the reporter, we don’t have enough data to reproduce and/or fix this issue. Please reopen or file a new bug with more information if you see it again.

Expected behaviour

This is expected behavior. Please reopen or file a new bug if you think otherwise.

Website issue

According to the investigation, this is a website issue. Please reopen or file a new bug if you think otherwise.