Media/WebRTC/Status Post IETF-84

Many thanks to EKR for providing the initial draft of this writeup!

Note: This plan does not include mozGetUserMedia(), which has already landed in mozilla-central and is being tested. We can't ship mozGetUserMedia() or WebRTC until we have UX to go along with mozGetUserMedia(), but I (Maire ) am hoping we will have UX completed in time for Firefox 18.

CURRENT STATUS

We have basic calling functionality working in a standalone mode within one browser, and it's probably a day or two of effort away from working between browsers. This includes support for:

• JSEP signaling
• Audio and video
• ICE/STUN (but not TURN)
• DTLS-SRTP at the transport layer (but not yet signaling)
• Prototype identity integration
• DataChannels (but not over the ICE/DTLS channel)

Our objective is to have a compliant, at least partially interoperable implementation in m-c by the time Chrome ships with WebRTC in around 12 weeks from today (release M23) and to have a publicly deployed version by end-of-year (target: Firefox 18). The first step in this process is to have a plausible, compliant implementation at all. The remainder of this message is focused on that.

________________________________________________________

PRIORITY 0: MINIMAL FUNCTIONALITY

The first priority is to finish the functionality that is indicated as partly finished above. This comprises the following tasks.

• Overall:
  • Verify that a call between two endpoints on different machines works. Fix as necessary. [ekr, anant].

  [Note: I tested this on a machine and a VM guest and that worked, so it will probably work. -- EKR]

• JSEP:
  • Wire up DTLS-SRTP fingerprints to the SDP [ekr, emannion]
  • Allow some minimal variation in the SDP. At minimum this means ignoring new identity attributes added by Anant. [emannion, ehugg, w/ ekr, fluffy]

• Media:
  • Remove the video time distortion (video now seems to play out at a very slow rate). This appears to be a result of the advertised frame rate for the source media stream. [crypt, ekr; bug 782299]

 [Note: made a temporary hack here, so this actually appears to work -- EKR]

• Data Channels
  • Move data channels to run over ICE/DTLS [jesup]

• Identity
  • Real in-chrome dialog for relying party [anant]
  • Wire up identity to DTLS-SRTP fingerprint checks [ekr]

• Stability
  • Make sure the system can run multiple calls, stop and start, and exit without crashing [all]

If we do the above, we will have something that people could at least download, play with, and demo. Also, we can't interoperate with Chrome without some of this stuff working.

________________________________________________________

PRIORITY 1: SPECIFICATION COMPLIANCE

Our second objective should be specification compliance. Obviously this is something of a work-in-progress and interacts with interop (below), but there are a number of known points of nonconformance which we need to address. Note: tasks below marked with ∧ are ones that we believe are needed for real interop with Chrome.

These tasks generally fall into two categories:

1. Places where we are noncompliant with everything (e.g., we advertise rtcp-mux but don't do it.)
2. Implement stuff we are required to (or at least really should) implement by WebRTC but don't (e.g., bundle).

Here's the known list:

• JSEP
  • Implement rtcp-mux. Currently, we have support for this in the SDP but it's not implemented in the transport layer. [ekr, emannion]
  • Implement bundle [ehugg, emannion, (ekr for transport piece?)]
  • Make sure that the VP8 and Opus SDP is correct [emannion] ∧
  • Implement/spec SDP for DataChannels [jesup, emannion]
  • Implement hints [emannion, ehugg]
  • Decide on which changes the JS can make to the SDP. Modify SIPCC to accept those changes. [ehugg, emannion] ∧ [Note from Maire: Are we blocked on the working group making a decision here?]
  • Allow audio w/o video, unidirection audio/video [emannion, ehugg]
  • Allow >1 of each type of media stream [emannion, ehugg, crypt, ekr]
  • Enforce state transition rules in SIPCC [emannion, ehugg]

• Media Transport
  • SSRC filtering for bundle [crypt, ekr]
  • Receive trickle ICE [ekr, emannion] ∧
  • Send trickle ICE (lower priority) [ekr, emannion]
  • Implement per-flow STUN configuration [anant, ekr]
  • Implement/wire up TURN. This is already in nICEr but may be obsolete/broken [ekr]

• Media
  • Implement Opus [derf, rillian]
  • Wire up all codecs in VCM [crypt]
  • Audio/video synchronization. GIPS has some support for this, but it needs to be hooked up and may require MediaStreams changes [crypt]

• Identity
  • Implement generic identity as specified in draft-ietf-rtcweb-security-arch, not just browserid [ekr, anant]

________________________________________________________

PRIORITY 2: INTEROPERABILITY WITH CHROME

As noted above, fixing noncompliance is intertwined with interop testing with Chrome. However, we can start doing interop testing as soon as we have fixed the really high order specification compliance issues.

The plan for interop testing is still evolving, but my thought was to have two levels of testing:

• A command line test in the mode of peerconnection_client
• End-to-end browser tests

The first of these is harder to stand up but better for debugging. The second is good for a demo and easy to stand up, especially since we need a roughly equivalent JS test harness for testing our own browser-to-browser mode. EKR is already working on the command line version, but we may start browser-to-browser testing first.