Media/multichannel
Overview
Current Overall Status
- Multiple channel support for cubeb on Windows is implemented
- Mixing module for cubeb is implemented and mixing test is added
- Have a WIP patch to make multiple channel works in Firefox Windows
- Surveying a more robust way to test the multiple channel support
Roadmap
- Implement multiple channel support in cubeb on Windows
- Support multiple channel for Firefox Windows based on previous cubeb's work
- Repeat the work for OSX, Linux, ... and other platforms.
Timeline
Evaluate dates here.
Status tracking
Cubeb
| ID | Summary | Product | Component | Resolution | Assigned to | Depends on | Blocks | Whiteboard | Target milestone |
|---|---|---|---|---|---|---|---|---|---|
| 1619726 | Use the new Rust audio mixer in cubeb windows | Core | Audio/Video: cubeb | C.M.Chang[:chunmin] | 1073786, 1368938 | --- | |||
| 1300024 | Support audio 5.1 on Android | Core | Audio/Video: cubeb | C.M.Chang[:chunmin] | 1073786, 1286101 | --- | |||
| 1368938 | Support audio 5.1 with non-SMPTE layout devices | Core | Audio/Video: cubeb | Paul Adenot (:padenot) | 1619726 | 1073786, 1325023 | --- | ||
| 1627827 | Audio goes wrong channel with Motu 828 mkii | Core | Audio/Video: cubeb | C.M.Chang[:chunmin] | 1628132 | 1073786 | [media-audio] | --- |
4 Total; 4 Open (100%); 0 Resolved (0%); 0 Verified (0%);
Playback
Insert bug list for playback here
Multiple Channel Support
Supported channels
To support more channels beyond Stereo(who has left and right channels), we need to define what channels we use:
| Code | Channel Name |
|---|---|
| Ｍ | Mono |
| L | Left(Front Left) |
| R | Right(Front Right) |
| C | Center(Front Center) |
| LS | Left Surround(Side Left) |
| RS | Right Surround(Side Right) |
| RLS | Rear Left Surround(Back Left) |
| RRC | Rear Center(Back Center) |
| RRS | Rear Right Surround(Bake Right) |
| LFE | Low Frequency Effects |
Channel Layout
Channel layout specifies the order of input/output channel data in audio buffer. For example, if the layout is stereo, then we have two channel data. The first data is for left channel, the second one is right channel.
Channel layout has various definitions, but SMPTE's format is most common:
| Name | Channels | |||||||
|---|---|---|---|---|---|---|---|---|
| DUAL-MONO | L | R | ||||||
| DUAL-MONO-LFE | L | R | LFE | |||||
| MONO | M | |||||||
| MONO-LFE | M | LFE | ||||||
| STEREO | L | R | ||||||
| STEREO-LFE | L | R | LFE | |||||
| 3F | L | R | C | |||||
| 3F-LFE | L | R | C | LFE | ||||
| 2F1 | L | R | RC | |||||
| 2F1-LFE | L | R | LFE | RC | ||||
| 3F1 | L | R | C | RC | ||||
| 3F1-LFE | L | R | C | LFE | RC | |||
| 2F2 | L | R | LS | RS | ||||
| 2F2-LFE | L | R | LFE | LS | RS | |||
| 3F2 | L | R | C | LS | RS | |||
| 3F2-LFE | L | R | C | LFE | LS | RS | ||
| 3F3R-LFE | L | R | C | LFE | RC | LS | RS | |
| 3F4-LFE | L | R | C | LFE | RLS | RRS | LS | RS |
Mixing
When the number of audio input channels is different from the number of audio output channels, we need to convert the audio input data to fit the audio output's configuration.
Downmix
When input channels is larger than output channels, we need to compress the audio input data. The conversion is called downmix(downward mixing). The table 2 in ITU-R BS.775-3 defines equations to convert audio from 3F2 to 1F(mono), 2F(stereo), 3F, 2F1, 3F1 and 2F2. We can simply add a LFE value to expand the downmix matrix. It will allow it to convert audio from 3F2-LFE(5.1 surround sound) to 1F(mono), 2F(stereo), 3F, 2F1, 3F1 and 2F2 and their LFEs. Here is the code example.
Upmix
When input channels is smaller than output channels, we need to expand the audio input data.
Bypass
When input channels is equal to output channels but they have different layout, we need to bypass the audio input data.