Confirmed users
325
edits
Media/WebRTC Audio Issues: Difference between revisions
Jump to navigation
Jump to search
Add a bunch of bug numbers
No edit summary |
(Add a bunch of bug numbers) |
||
| Line 2: | Line 2: | ||
* '''Sampling Rate issues''' | * '''Sampling Rate issues''' | ||
** 44/44.1KHz mismatch -- {{bug|886886}}<br>This causes a 0.23% drift in audio and a buffer buildup in MediaStreamGraph (delay). This is only an issue when a sampling frequency of 44100 is used, in particular on Windows when the user/default has that selected. Note some laptops and other devices come configured this way, or may have been reconfigured by the user for 44100Hz | ** 44/44.1KHz mismatch -- <strike>{{bug|886886}}</strike><br>This causes a 0.23% drift in audio and a buffer buildup in MediaStreamGraph (delay). This is only an issue when a sampling frequency of 44100 is used, in particular on Windows when the user/default has that selected. Note some laptops and other devices come configured this way, or may have been reconfigured by the user for 44100Hz | ||
*** We believe this issue is also affecting B2G | *** We believe this issue is also affecting B2G | ||
*** Since we're requesting 16000 currently (and perhaps/probably 32000 or 48000 in the future), it makes sense to do the resample at the 44/44.1->16000 point, not a second resample. Bug 886886 has a patch to handle this using the Speex resampler in our tree. There are also patches to port upcoming work from webrtc.org using their sinc resampler; however those are far more extensive and not upliftable. | *** Since we're requesting 16000 currently (and perhaps/probably 32000 or 48000 in the future), it makes sense to do the resample at the 44/44.1->16000 point, not a second resample. Bug 886886 has a patch to handle this using the Speex resampler in our tree. There are also patches to port upcoming work from webrtc.org using their sinc resampler; however those are far more extensive and not upliftable. | ||
| Line 13: | Line 13: | ||
*** The "L16" pseudo-codec in GIPS only supports 8000/16000/32000 sampling rates (including in 3.30) | *** The "L16" pseudo-codec in GIPS only supports 8000/16000/32000 sampling rates (including in 3.30) | ||
* '''MediaStreamGraph fundamental latency''' | * '''MediaStreamGraph fundamental latency''' | ||
** Because MediaStreamGraph reclocks and plays out from the MSG, it has to keep a minimal buffering level to avoid underflow. This adds 15-30ish ms of input-side latency (and output latency, though output clocking MSG will reduce that). Note that correcting this may require different audio streams for internal versus PeerConnection/"realtime" streams. | ** Because MediaStreamGraph reclocks and plays out from the MSG, it has to keep a minimal buffering level to avoid underflow. This adds 15-30ish ms of input-side latency (and output latency, though output clocking MSG will reduce that). Note that correcting this may require different audio streams for internal versus PeerConnection/"realtime" streams. See patch on {{bug|884365}} | ||
* '''Need to add a TrackUnion to streams output from PeerConnection''' | * '''Need to add a TrackUnion to streams output from PeerConnection''' | ||
** We can get persistent delay if the output of a PeerConnection gets blocked. The patch for this has been r-'d and needs a re-design. | ** We can get persistent delay if the output of a PeerConnection gets blocked. The patch for this has been r-'d and needs a re-design. {{Bug|832881}} | ||
* '''AEC location & quality''' | * '''AEC location & quality''' | ||
** The AEC should be in getUserMedia() to have the option of cancelling audio from multiple PeerConnections (so A doesn't head the echo of B (and vice-versa) when both are talking to C). Also this will allow other audio from the browser to be cancelled. Currently the AEC only cancels audio in the same PeerConnection. | ** The AEC should be in getUserMedia() to have the option of cancelling audio from multiple PeerConnections (so A doesn't head the echo of B (and vice-versa) when both are talking to C). Also this will allow other audio from the browser to be cancelled. Currently the AEC only cancels audio in the same PeerConnection. {{Bug|694814}} | ||
*** Google apparently has not moved the AEC yet either. | *** Google apparently has not moved the AEC yet either. | ||
** The pre-AEC resampler could be higher quality (currently it's linear) | ** The pre-AEC resampler could be higher quality (currently it's linear) | ||
*** Google tells us the pre-AEC resampler quality doesn't matter much as it works only on the far-end sound, and in practice causes little cancellation-quality loss. | *** Google tells us the pre-AEC resampler quality doesn't matter much as it works only on the far-end sound, and in practice causes little cancellation-quality loss. | ||
* '''Dynamic input/output changes''' | * '''Dynamic input/output changes''' | ||
** We need to support hot-(un)plugging headsets, and preferably not requiring it to be unplugged to send audio to speakers/etc | ** We need to support hot-(un)plugging headsets, and preferably not requiring it to be unplugged to send audio to speakers/etc. {{Bug|827146}} | ||
** We need to support audio output routing (at least to support "ringing" from main speakers while in-call/video/etc audio goes to headset). | ** We need to support audio output routing (at least to support "ringing" from main speakers while in-call/video/etc audio goes to headset). | ||
* '''Investigate any remaining latency issues''' | * '''Investigate any remaining latency issues''' | ||
** Identifying any additional issues ASAP is critical | ** Identifying any additional issues ASAP is critical | ||
* '''Find some way to test audio quality and delay in automated testing''' | * '''Find some way to test audio quality and delay in automated testing''' | ||