MozillaWiki

Project Fission/Memory: Difference between revisions

Page Discussion

Project Fission/Memory (view source)

Revision as of 15:39, 29 June 2020

5,013 bytes added ,  29 June 2020
DMD mode
(Initial draft of fission memory project page.)
 
(DMD mode)
 
(20 intermediate revisions by 4 users not shown)
Line 1: Line 1:
== Goals ==
== Goals ==


Due to the drastic increase in the number of processes required to support [[Project Fission]] we must focus on reducing the per process overhead of each content process. As a baseline we are working on reducing the amount of memory used to load an about:blank page.
Due to the drastic increase in the number of processes required to support [[Project Fission]] we must focus on reducing the per-process overhead of each content process. As a baseline, we are working on reducing the amount of memory used to load an about:blank page.
 
We should also look at per-content process overhead in the parent. One example of this is IPDL protocols. We are not currently  measuring this.
 
== Measurement tools ==
=== about:memory ===
This is the default tool for looking at memory usage. One approach to finding targets for memory reduction is to look at the about:memory report for a small content process (for instance, example.com) and see what is using up memory. In addition to using it to guide memory reduction work, we can work to reduce the heap-unclassified number (memory allocated by Firefox but not reported in about:memory) in tests of interest. Adding new reporters often leads to finding things to improve.
 
=== DMD ===
[https://developer.mozilla.org/en-US/docs/Mozilla/Performance/DMD DMD] (aka the Dark Matter Detector) complements about:memory. When DMD mode is enabled, Firefox tracks the address and allocation stacks of all live blocks of memory and can save that information to a log for offline analysis by dmd.py. The primary use is investigating heap-unclassified, but with alternate modes enabled it can be used for other tasks like investigating leaks.
 
=== GC logs ===
GC logs, which can be saved via the about:memory page, record information about live JS objects. As JS is a major source of memory overhead, having a detailed understanding of where the memory going is very useful. There are a [https://github.com/amccreight/heapgraph/tree/master/g number of scripts available] to help analyze these log files.
 
One interesting variation of this is to run with MOZ_GC_LOG_SIZE=1, which leverages the DevTools UbiNode work to record the size of individual objects in the GC log. You can then use those logs with the dominator tree based analysis in [https://github.com/amccreight/heapgraph/blob/master/g/dom_tree.py dom_tree.py] to get fine-grained information about the memory overhead of chrome JS, down to the individual function level.


== Metrics ==
== Metrics ==
Line 17: Line 31:
Our main focus for Fission is the simpler about:blank ''SY-e10s(ab)'' test. This gives a less noisy baseline metric that allows us to focus on incremental improvements. This is essentially our ''best case'' metric.
Our main focus for Fission is the simpler about:blank ''SY-e10s(ab)'' test. This gives a less noisy baseline metric that allows us to focus on incremental improvements. This is essentially our ''best case'' metric.


===== Metrics =====
'''Resident Unique''' is our main measurement of success. It's the total amount of memory that the content process is using that's not shared with other processes:
'''Resident Unique''' is our main measurement of success. It's the total amount of memory that the content process is using that's not shared with other processes:
* [https://treeherder.mozilla.org/perf.html#/graphs?timerange=31536000&series=mozilla-central,1684862,1,4 Resident Unique Memory - Linux]
* [https://treeherder.mozilla.org/perf.html#/graphs?timerange=31536000&series=mozilla-central,1684862,1,4 Resident Unique Memory - Linux]
Line 26: Line 41:
* [https://treeherder.mozilla.org/perf.html#/graphs?timerange=31536000&series=mozilla-central,1685137,1,4 JavaScript Memory - Windows]
* [https://treeherder.mozilla.org/perf.html#/graphs?timerange=31536000&series=mozilla-central,1685137,1,4 JavaScript Memory - Windows]
* [https://treeherder.mozilla.org/perf.html#/graphs?timerange=31536000&series=mozilla-central,1684850,1,4 JavaScript Memory - OSX]
* [https://treeherder.mozilla.org/perf.html#/graphs?timerange=31536000&series=mozilla-central,1684850,1,4 JavaScript Memory - OSX]
* [https://treeherder.mozilla.org/perf.html#/graphs?series=mozilla-inbound,1890241,1,4&series=mozilla-inbound,1684808,1,4&series=autoland,1684872,1,4&series=mozilla-inbound,1944123,1,4&series=mozilla-central,1684866,1,4&series=autoland,1885459,1,4&series=mozilla-central,1939409,1,4&series=mozilla-central,1888424,1,4&series=autoland,1925415,1,4 JavaScript Memory - Regression view]<br/>This is useful for tracking down where a regression started by breaking out integration branches. It's limited to just Linux, but JS is generally platform agnostic.
===== Testing your changes =====
To run locally use:
./mach awsy-test testing/awsy/awsy/test_base_memory_usage.py
To run on try for comparisons use:
hg up base_revision
./mach try -b o -p linux64 -u awsy-base-e10s -t none --rebuild 5
hg up new_revision
./mach try -b o -p linux64 -u awsy-base-e10s -t none --rebuild 5


==== AWSY (sy) ====
==== AWSY (sy) ====
The original test suite, ''SY-e10s(sy)'', is a stress test that loads 100 pages into 30 tabs 3 times and measures memory at various points and is useful for detecting regressions in startup memory usage, longer term leaks, leaked windows, etc. This is essentially our ''worst case'' metric.
The original test suite, ''SY-e10s(sy)'', is a stress test that loads 100 pages into 30 tabs 3 times and measures memory at various points and is useful for detecting regressions in startup memory usage, longer term leaks, leaked windows, etc. This is essentially our ''worst case'' metric.


===== Metrics =====
'''after tabs opened''' is one of several data points gathered, it is the most relevant to this project:
'''after tabs opened''' is one of several data points gathered, it is the most relevant to this project:
* [https://treeherder.mozilla.org/perf.html#/graphs?timerange=31536000&series=mozilla-central,1651963,1,4 Resident Memory - Linux]
* [https://treeherder.mozilla.org/perf.html#/graphs?timerange=31536000&series=mozilla-central,1651963,1,4 Resident Memory - Linux] [https://treeherder.mozilla.org/perf.html#/graphs?timerange=31536000&series=mozilla-central,1651963,1,4&series=mozilla-inbound,1643517,1,4&series=autoland,1651004,1,4&series=mozilla-central,1893061,1,4&series=mozilla-inbound,1893225,1,4&series=autoland,1885896,1,4 [regression view]]
* [https://treeherder.mozilla.org/perf.html#/graphs?timerange=31536000&series=mozilla-central,1653935,1,4 Resident Memory - Windows]
* [https://treeherder.mozilla.org/perf.html#/graphs?timerange=31536000&series=mozilla-central,1653935,1,4 Resident Memory - Windows] [https://treeherder.mozilla.org/perf.html#/graphs?timerange=31536000&series=mozilla-central,1784212,1,4&series=mozilla-inbound,1784162,1,4&series=autoland,1784052,1,4&series=mozilla-central,1893325,1,4&series=mozilla-inbound,1893275,1,4&series=autoland,1885410,1,4 [regression view]]
* [https://treeherder.mozilla.org/perf.html#/graphs?timerange=31536000&series=mozilla-central,1651894,1,4 Resident Memory - OSX]
* [https://treeherder.mozilla.org/perf.html#/graphs?timerange=31536000&series=mozilla-central,1651894,1,4 Resident Memory - OSX] [https://treeherder.mozilla.org/perf.html#/graphs?timerange=31536000&series=mozilla-central,1651894,1,4&series=mozilla-inbound,1642686,1,4&series=autoland,1650713,1,4 [regression view]]


===== Testing your changes =====
To run locally use:
./mach awsy-test


== Meetings ==
To run on try for comparisons use:
Meetings happen every Tuesday at 12:30pm Pacific time ([http://arewemeetingyet.com/Los%20Angeles/2018-09-18/12:30/w/Fission%20MemShrink%20Meeting countdown in your timezone])
hg up base_revision
{{conf|8394}}
./mach try -b o -p linux64 -u awsy-e10s -t none --rebuild 5
* Vidyo: MemShrink
hg up new_revision
* IRC: [irc://irc.mozilla.org/memshrink #memshrink]
./mach try -b o -p linux64 -u awsy-e10s -t none --rebuild 5
 
==== AWSY (tp6) ====
An updated test suite, ''SY-e10s(tp6)'', is a test that loads the [https://searchfox.org/mozilla-central/source/testing/awsy/tp6-pageset.manifest entire tp6 pageset] and measures memory at various points. The tp6 pageset uses mitmproxy to simulate live connections which will allow us to properly test out-of-process iframes.
 
===== Metrics =====
Currently we're not tracking these metrics, they'll be more interesting once we enable fission.
 
===== Testing your changes =====
To run locally use:
./mach awsy-test --tp6 --preferences testing/awsy/conf/tp6-prefs.json
 
To run on try for comparisons use:
hg up base_revision
./mach try fuzzy --rebuild 5 -q "'linux64/opt-awsy-tp6"
hg up new_revision
./mach try fuzzy --rebuild 5 -q "'linux64/opt-awsy-tp6"


== Bug Tracking ==
== Bug Tracking ==
Line 47: Line 93:


=== Triage ===
=== Triage ===
We use the '''[overhead]''' whiteboard tag to flag items for triage. Additionally any bug in the {{Bug|1436250}} dependency tree is generally triaged. The triage process attempts estimate the impact a bug will have on reducing the overhead of a content process. If we think a bug will reduce per process memory usage by 30KB then update the tag with the expected win: '''[overhead:30K]'''. We try to use a reasonable guess for this value, it doesn't need to be exact.
We use the '''[overhead]''' whiteboard tag to flag items for triage. Additionally any bug in the {{Bug|1436250}} dependency tree is generally triaged. The triage process attempts to estimate the impact a bug will have on reducing the overhead of a content process. If we think a bug will reduce per-process memory usage by 30KB then update the tag with the expected win: '''[overhead:30K]'''. We try to use a reasonable guess for this value, it doesn't need to be exact.


=== Progress ===
=== Progress ===
Line 62: Line 108:
|-
|-
| Q4 2018 || 20.7MB || 5.3MB
| Q4 2018 || 20.7MB || 5.3MB
|-
| Q4.5 2018 || 17.8MB || 4.6MB
|-
| Q1 2019* || 19.5MB || 5.0MB
|-
| Q2 2019 || 19.5MB || 4.1MB
|-
| Q3 2019 || 19.5MB || 4.1MB
|}
|}


Line 67: Line 121:
{| class="wikitable"
{| class="wikitable"
|-
|-
! Quarter !! Resident Unique !! JS
! Quarter !! Resident Unique !! JS !! Resident Unique (WebRender)
|-
| Q2 2018 || 21.5MB || 8.5MB || N/A
|-
| Q3 2018 || 20.8MB || 8.0MB || N/A
|-
| Q4 2018 || 15.0MB || 5.3MB || 17.1MB
|-
| Q4.5 2018 || 15.0MB || 4.8MB || 17.1MB
|-
|-
| Q2 2018 || 21.5MB || 8.5MB
| Q1 2019* || 17.8MB || 5.1MB || 14.5MB
|-
|-
| Q3 2018 || 20.8MB || 8.0MB
| Q2 2019 || 15.1MB || 4.1MB || 14.8MB
|-
|-
| Q4 2018 || 15.0MB || 5.3MB
| Q3 2019 || 15.1MB || 4.1MB || 14.8MB
|}
|}


Line 86: Line 148:
|-
|-
| Q4 2018 || 21.4MB || 5.3MB
| Q4 2018 || 21.4MB || 5.3MB
|-
| Q4.5 2018 || 20.4MB || 4.6MB
|-
| Q1 2019* || 25.2MB || 5.0MB
|-
| Q2 2019 || 24.6MB || 4.1MB
|-
| Q3 2019 || 24.6MB || 4.1MB
|}
|}
* ''Q1 2019 note'': We switched to VMs with GPUs to better simulate real-world conditions. This resulted in apparent regressions across the board.
Amccreight
Confirmed users
628

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1201373...1228590"