Confirmed users
325
edits
Media/WebRTC Audio Issues: Difference between revisions
Jump to navigation
Jump to search
no edit summary
No edit summary |
No edit summary |
||
| Line 12: | Line 12: | ||
** Basic sample rate low<br>Right now, we're clocking everything at 16000Hz; we should be using higher clockrates. | ** Basic sample rate low<br>Right now, we're clocking everything at 16000Hz; we should be using higher clockrates. | ||
*** The "L16" pseudo-codec in GIPS only supports 8000/16000/32000 sampling rates (including in 3.30) | *** The "L16" pseudo-codec in GIPS only supports 8000/16000/32000 sampling rates (including in 3.30) | ||
* '''MediaStreamGraph fundamental latency | * '''Audio Latency''' -- {{Bug|785584}} | ||
** Because MediaStreamGraph reclocks and plays out from the MSG, it has to keep a minimal buffering level to avoid underflow. This adds 15-30ish ms of input-side latency (and output latency, though output clocking MSG will reduce that). Note that correcting this may require different audio streams for internal versus PeerConnection/"realtime" streams. See patch on {{bug|884365}} | ** Increases in delay and loss of sync -- {{bug|879213}} | ||
*** Clock-domain mismatches need a resampler to avoid possible latency buildup -- {{bug|884365}} | |||
*** Underflows in MSG cause MSG to "slip" the stream, such that later data is permanently delayed. {{bug|901831}}, {{bug|901539}}. A resampler can treat just slips as clock jitter -- {{bug|884365}}, though if the underflow is serious enough we may need to simply drop audio. | |||
*** Reducing load on MSG in callbacks (NotifyPush(), NotifyQueuedTrackChanges()) will reduce the odds of MSG underflowing. See {{bug|884365}} for reduction in the largest CPU consumer (Opus Encode + AEC). Also see {{bug|901831}} for an odd windows-only two-browser interaction. | |||
** MediaStreamGraph fundamental latency | |||
*** Because MediaStreamGraph reclocks and plays out from the MSG, it has to keep a minimal buffering level to avoid underflow. This adds 15-30ish ms of input-side latency (and output latency, though output clocking MSG will reduce that). Note that correcting this may require different audio streams for internal versus PeerConnection/"realtime" streams. See patch on {{bug|884365}} | |||
** Investigate any remaining latency issues | |||
* '''Need to add a TrackUnion to streams output from PeerConnection''' | * '''Need to add a TrackUnion to streams output from PeerConnection''' | ||
** We can get persistent delay if the output of a PeerConnection gets blocked. The patch for this has been r-'d and needs a re-design. {{Bug|832881}} | ** We can get persistent delay if the output of a PeerConnection gets blocked. The patch for this has been r-'d and needs a re-design. {{Bug|832881}} | ||
| Line 24: | Line 30: | ||
** We need to support hot-(un)plugging headsets, and preferably not requiring it to be unplugged to send audio to speakers/etc. {{Bug|827146}} | ** We need to support hot-(un)plugging headsets, and preferably not requiring it to be unplugged to send audio to speakers/etc. {{Bug|827146}} | ||
** We need to support audio output routing (at least to support "ringing" from main speakers while in-call/video/etc audio goes to headset). | ** We need to support audio output routing (at least to support "ringing" from main speakers while in-call/video/etc audio goes to headset). | ||
* '''Find some way to test audio quality and delay in automated testing''' | * '''Find some way to test audio quality and delay in automated testing''' | ||