Firefox OS/MappedArrayBuffer

Memory-mapped array buffer for XHR response

The feature allows to read data in packaged app by XHR with array buffer type as memory-mapped. The memory-mapped array buffer is created as copy-on-write.

Use Case

The feature helps to save RAM memory usage, especially for big data files in a packaged app. For example, when enabled, the word suggestion(auto correction) or the IME database in the keyboard APP no longer consumes RAM memory. To use this feature, the data file in the packaged app must be uncompressed.

• To determine whether to use it, you may want to consider these factors:
  • RAM size
  • Flash size
  • File size
  • File compression ratio

Support status

• Supported platform
  • Linux
  • Firefox OS(except Windows simulator)

Windows support follow up at https://bugzilla.mozilla.org/show_bug.cgi?id=988813

• Parent process and child process
• Main thread and worker thread
• Preference
  • Use "dom.mapped_arraybuffer.enabled" to enable/disable this feature
  • Enabled for Firefox OS by default

Requirements

• Target file data offset in a packaged app must be
  • Uncompressed
  • Aligned to
    • 8 bytes minimum, and page size[1] recommended[2]
    • After [3] landed, page size aligment is mandatory
• If requirements not meet
  • Fall back to be normal array buffer

[1] Page size on 32-bits Linux is 4096 bytes.

[2] Page alignment of target file in the zip may increase the zip package size a little, but it saves you one page heap memory overhead.

[3] Bug 855669 - Figure out how emscripten &co can decommit memory

OOP support

See https://bugzilla.mozilla.org/show_bug.cgi?id=988816 for detail.

Q & A

• How to make the file uncompressed and aligned in packaged app?

For keyboard app, put the file |metadata.json| in gaia/apps/keyboard to indicate the file to be memory-mapped, below is an example.

 {
   "external": false,
   "zip": {
     "mmap_files": [
       "js/imes/latin/dictionaries/en_us.dict",
       "js/imes/jszhuyin/data/database.data"
     ]  
   }
 }

 $ unzip -v gaia/profile/webapps/keyboard.gaiamobile.org/application.zip | grep Stored
 1451390  Stored  1451390   0% 1970-01-01 08:00 3d22f787  js/imes/latin/dictionaries/en_us.dict
 4541832  Stored  4541832   0% 1970-01-01 08:00 426a7f8e  js/imes/jszhuyin/data/database.data

For more information, please see https://bugzilla.mozilla.org/show_bug.cgi?id=957497

• How to know if the array buffer is memory-mapped in an XHR response?

The Content-Type header is "application/mem-mapped" in such response.

• How to check memory usage?

You can check the memory usage by the following command:

 $ tools/get_about_memory.py --no-gc-cc-log

Below is an example for English and JSZhuyin IME, which means they are not taking any RAM memory.

 │   ├──5.13 MB (27.03%) -- worker(./js/imes/jszhuyin/lib/worker.js, 0xb22dd400)
 │   │  ├──4.57 MB (24.10%) -- zone(0xb104c800)
 │   │  │  ├──4.45 MB (23.46%) -- compartment(web-worker)
 │   │  │  │  ├──4.36 MB (22.99%) -- objects
 │   │  │  │  │  ├──4.33 MB (22.83%) -- non-heap
 │   │  │  │  │  │  ├──4.33 MB (22.83%) ── elements/mapped
 │   │  │  │  │  │  └──0.00 MB (00.00%) ── code/asm.js
 │   │  │  │  │  └──0.03 MB (00.16%) ++ (2 tiny)
 │   │  │  │  └──0.09 MB (00.48%) ++ (2 tiny)
 │   │  │  └──0.12 MB (00.63%) ++ (4 tiny)
 │   │  ├──0.29 MB (01.54%) ++ runtime
 │   │  ├──0.23 MB (01.24%) -- gc-heap
 │   │  │  ├──0.21 MB (01.11%) ── unused-arenas
 │   │  │  └──0.02 MB (00.12%) ++ (2 tiny)
 │   │  └──0.03 MB (00.16%) ++ zone(0xb104bc00)
 │   └──2.02 MB (10.64%) -- worker(js/imes/latin/worker.js, 0xb0d02800)
 │      ├──1.68 MB (08.86%) -- zone(0xb0d0d000)
 │      │  ├──1.56 MB (08.21%) -- compartment(web-worker)
 │      │  │  ├──1.48 MB (07.82%) -- objects
 │      │  │  │  ├──1.38 MB (07.30%) -- non-heap
 │      │  │  │  │  ├──1.38 MB (07.30%) ── elements/mapped
 │      │  │  │  │  └──0.00 MB (00.00%) ── code/asm.js
 │      │  │  │  └──0.10 MB (00.53%) ++ (2 tiny)
 │      │  │  └──0.07 MB (00.39%) ++ (2 tiny)
 │      │  └──0.12 MB (00.65%) ++ (4 tiny)
 │      ├──0.29 MB (01.54%) ++ runtime
 │      └──0.05 MB (00.25%) ++ (2 tiny)

Memory-mapped array buffer is reported at non-heap/elements/mapped

Normal array buffer is reported at malloc-heap/elements/non-asm.js