Platform/Features/VideoConferencing

Please use "Edit with form" above to edit this page.

Status

Video (and Audio) Conferencing
Stage	Planning
Status	In progress
Release target	`
Health	OK
Status note	`

{{#set:Feature name=Video (and Audio) Conferencing

|Feature stage=Planning |Feature status=In progress |Feature version=` |Feature health=OK |Feature status note=` }}

Team

Product manager	Christopher Blizzard
Directly Responsible Individual	Christopher Blizzard
Lead engineer	Randell Jesup
Security lead	Curtis Koenig
Privacy lead	Sid Stamm
Localization lead	`
Accessibility lead	TBD
QA lead	TBD
UX lead	Jennifer Boriss
Product marketing lead	TBD
Operations lead	`
Additional members	Media Team

{{#set:Feature product manager=Christopher Blizzard

|Feature feature manager=Christopher Blizzard |Feature lead engineer=Randell Jesup |Feature security lead=Curtis Koenig |Feature privacy lead=Sid Stamm |Feature localization lead=` |Feature accessibility lead=TBD |Feature qa lead=TBD |Feature ux lead=Jennifer Boriss |Feature product marketing lead=TBD |Feature operations lead=` |Feature additional members=Media Team }}

Open issues/risks

• Privacy UI for this feature will be difficult to work out since it involves video and audio as part of the platform. Please see the IETF Security Considerations doc as an example.
• Security will be difficult due to the wish for applications to be able to provide useful interfaces, in conflict with the need to guarantee users are safe with regards to access to their camera and microphones.
• Data transport has yet to be speced and the IETF may or may not pick it up.
• Overall system latency (delay) might be an issue; if it is architectural changes may be needed. Testing will need to be done.
• VP8 is the only open video codec under consideration, but it's likely to face patent/licensing challenges

Stage 1: Definition

1. Feature overview

This feature will allow people using the web platform to include video and audio conferencing as part of their applications. The main attributes include:

• Using the web as the setup end point for a "call." This is different than other calling systems, which use SIP, XMPP or other signaling systems.
• Once capabilities are exchanged, a point-to-point connection is established between browsers when possible. This includes NAT traversal.
• Video is displayed as a primary object in the browser and can be part of content.
• Data can be exchanged along with with the video and audio.

Note that this page does not contain other, further use cases for audio and video - connecting with devices, multi-party conferencing, etc - those are different shippable units although early design choices should reflect these further architectural goals.

2. Users & use cases

• Call a friend on Facebook. From the chat/online window on Facebook you can click on a person and start a video call with them in the browser.

• Video conferencing / sharing slides from inside an app. You call someone from inside an app and you're able to share files, images, slides, etc through the P2P connection as you're talking via the data channel.

• Remote assistance/co-pilot - you "look over the shoulder" of someone as they browse (optionally controlling the mouse/keyboard), while also talking with them over audio or audio+video.

For reference only:

• IETF Use Case Document
• WebRTC meeting @ W3C
• Cullen's Slides

3. Dependencies

This requires the integration of the newly released WebRTC code. It also requires integration of Robert's Media API work.

4. Requirements

Must Have:

• Security design review
• Security implementation review

• The ability to make a call from a web page using a machine's built-in mic and camera.
• Should be able to work with most (80-90%) of NAT setups across all geographies. Note that certain configurations will require a relay.
• Support for royalty-free voice-friendly audio codecs
• Support for royalty-free video codecs
• Audio/Video/Data must be secure (note that no security is absolute)
• A Web API that's been sent to the public working groups for feedback.
• A Web API and transport for data that accompanies the audio and video.
• Automatic bandwidth adjustment (congestion control)
• Ability to add/remove video

• Support on Mobile for voice calling.

Nice to Have:

• Video resolution switching
• If hardware support exists for encoding & decoding VP8, support for video on Mobile.

Non-goals

As stated above, we are not trying to initially support multi-party connections, capturing images or other things not explicitly listed here. This page is focused on the first shippable unit.

Stage 2: Design

5. Functional specification

• Audio codecs planned: G.711 & Opus. Possible: iLBC, iSAC and G.722 (all royalty/license-free)
• Video codecs planned: VP8

API Docs:

• Robert's MediaStream Processing API Proposal
• Mozilla's RTC API Proposal

6. User experience design

`

Stage 3: Planning

7. Implementation plan

Initial demo and release will be an add-on, with some supporting elements added or updated in core if needed. After we have more experience with it and it gains stability, confidence in security (and the ietf/w3 spec gains stability) more or all of the functionality will be moved into core.

8. Reviews

Security review

It's important to realize that this is the first step in a plan to bring video to the platform. We'll eventually be adding multi-party video conferencing and other ways to interact with audio and video. From both a security and privacy standpoint this means we'll need to be reviewing it with an eye towards an eye towards future use cases as well as what's scoped in this one page.

Privacy review

`

Localization review

`

Accessibility

`

Quality Assurance review

`

Operations review

`

Stage 4: Development

9. Implementation

Set of items that need doing with phases.

Stage 5: Release

10. Landing criteria

` {{#set:Feature open issues and risks=* Privacy UI for this feature will be difficult to work out since it involves video and audio as part of the platform. Please see the IETF Security Considerations doc as an example.

• Security will be difficult due to the wish for applications to be able to provide useful interfaces, in conflict with the need to guarantee users are safe with regards to access to their camera and microphones.
• Data transport has yet to be speced and the IETF may or may not pick it up.
• Overall system latency (delay) might be an issue; if it is architectural changes may be needed. Testing will need to be done.
• VP8 is the only open video codec under consideration, but it's likely to face patent/licensing challenges

|Feature overview=This feature will allow people using the web platform to include video and audio conferencing as part of their applications. The main attributes include:

• Using the web as the setup end point for a "call." This is different than other calling systems, which use SIP, XMPP or other signaling systems.
• Once capabilities are exchanged, a point-to-point connection is established between browsers when possible. This includes NAT traversal.
• Video is displayed as a primary object in the browser and can be part of content.
• Data can be exchanged along with with the video and audio.

Note that this page does not contain other, further use cases for audio and video - connecting with devices, multi-party conferencing, etc - those are different shippable units although early design choices should reflect these further architectural goals. |Feature users and use cases=* Call a friend on Facebook. From the chat/online window on Facebook you can click on a person and start a video call with them in the browser.

• Video conferencing / sharing slides from inside an app. You call someone from inside an app and you're able to share files, images, slides, etc through the P2P connection as you're talking via the data channel.

• Remote assistance/co-pilot - you "look over the shoulder" of someone as they browse (optionally controlling the mouse/keyboard), while also talking with them over audio or audio+video.

For reference only:

• IETF Use Case Document
• WebRTC meeting @ W3C
• Cullen's Slides

|Feature dependencies=This requires the integration of the newly released WebRTC code. It also requires integration of Robert's Media API work. |Feature requirements=Must Have:

• Security design review
• Security implementation review

• The ability to make a call from a web page using a machine's built-in mic and camera.
• Should be able to work with most (80-90%) of NAT setups across all geographies. Note that certain configurations will require a relay.
• Support for royalty-free voice-friendly audio codecs
• Support for royalty-free video codecs
• Audio/Video/Data must be secure (note that no security is absolute)
• A Web API that's been sent to the public working groups for feedback.
• A Web API and transport for data that accompanies the audio and video.
• Automatic bandwidth adjustment (congestion control)
• Ability to add/remove video

• Support on Mobile for voice calling.

Nice to Have:

• Video resolution switching
• If hardware support exists for encoding & decoding VP8, support for video on Mobile.

|Feature non-goals=As stated above, we are not trying to initially support multi-party connections, capturing images or other things not explicitly listed here. This page is focused on the first shippable unit. |Feature functional spec=* Audio codecs planned: G.711 & Opus. Possible: iLBC, iSAC and G.722 (all royalty/license-free)

• Video codecs planned: VP8

API Docs:

• Robert's MediaStream Processing API Proposal
• Mozilla's RTC API Proposal

|Feature ux design=` |Feature implementation plan=Initial demo and release will be an add-on, with some supporting elements added or updated in core if needed. After we have more experience with it and it gains stability, confidence in security (and the ietf/w3 spec gains stability) more or all of the functionality will be moved into core. |Feature security review=It's important to realize that this is the first step in a plan to bring video to the platform. We'll eventually be adding multi-party video conferencing and other ways to interact with audio and video. From both a security and privacy standpoint this means we'll need to be reviewing it with an eye towards an eye towards future use cases as well as what's scoped in this one page. |Feature privacy review=` |Feature localization review=` |Feature accessibility review=` |Feature qa review=` |Feature operations review=` |Feature implementation notes=Set of items that need doing with phases. |Feature landing criteria=` }}

Feature details

Priority	P1
Rank	999
Theme / Goal	Connect
Roadmap	Gecko
Secondary roadmap	Mozilla Identity
Feature list	Platform
Project	`
Engineering team	Media

{{#set:Feature priority=P1

|Feature rank=999 |Feature theme=Connect |Feature roadmap=Gecko |Feature secondary roadmap=Mozilla Identity |Feature list=Platform |Feature project=` |Feature engineering team=Media }}

Team status notes

	status	notes
Products	`	`
Engineering	`	Initial testing & deployment target is an add-on, followed by moving the feature(s) into the core.
Security	`	`
Privacy	`	`
Localization	`	`
Accessibility	`	`
Quality assurance	`	`
User experience	`	`
Product marketing	`	`
Operations	`	`

{{#set:Feature products status=`

|Feature products notes=` |Feature engineering status=` |Feature engineering notes=Initial testing & deployment target is an add-on, followed by moving the feature(s) into the core. |Feature security status=` |Feature security health=` |Feature security notes=` |Feature privacy status=` |Feature privacy notes=` |Feature localization status=` |Feature localization notes=` |Feature accessibility status=` |Feature accessibility notes=` |Feature qa status=` |Feature qa notes=` |Feature ux status=` |Feature ux notes=` |Feature product marketing status=` |Feature product marketing notes=` |Feature operations status=` |Feature operations notes=` }}