Simulcast

What is Simulcast?

Simulcast is a useful technique for broadcasting videos during conferences on which participants are in different network conditions and use various devices.

During a typical conference, media is sent from the sender to the Selective Forwarding Unit (SFU), and then to all conference participants. The bandwidth of each of the receivers is often different due to network conditions and device capability issues. Using nonscalable streams, the sender transmits a media stream with a bit rate adjusted to the participant that has the lowest bandwidth. Therefore, all other participants receive lower quality, even if they have good network conditions.

Simulcast in the scope of WebRTC calls, introduced in SDK 2.1, solves this problem. It allows the sender to simultaneously send multiple video streams with different resolutions (HD, MD, SD). Therefore, the SFU adjusts the bit rate according to various receivers’ bandwidth limitations. Based on the received Real-time Transport Control Protocol (RTCP) feedback packets, the SFU forwards the most suitable stream to each receiver. If one conference participant has lower bandwidth, that receiver receives lower-quality streams without impacting other conference participants. Therefore, higher bandwidth users can enjoy higher quality.

Mechanism

Automatic and manual stream selection

The SDK 2.1 provides APIs with the ability to choose the quality of the received stream. The user can, for example, choose to receive a lower quality stream through a wider bandwidth to display a small video during a group conference call. In case of a situation when the bit rate of the selected stream is beyond the receiver's available bandwidth, the system automatically selects a lower-quality stream that can be properly received.

If a user joins a conference where one or more clients send Simulcast streams, the system automatically adapts to the best quality stream that suits the receiver's network condition. It does not require any additional programming or configuration on the client side.

Resolution and bitrate

For the sender

The sender can transmit specific Simulcast streams listed in the following table.

StreamsResolution
HD, MD, SD1280 x 720
HD, MD, SD960 x 540
MD, SD640 x 360
MD, SD480 x 270
SD320 x 180

The stream selection is based on the RTCP feedback packets sent by SFU, which can be either:

  • REMB packets, in cases where the sender's network bandwidth estimation is performed by SFU and sent back to the sender
  • TransportCC packets, in cases where the sender's network bandwidth estimation is performed by the sender, based on TrasnportCC packet data

The sender's network bandwidth estimation cannot be higher than 2 Mbps per stream. The number of layers (one, two, or three) is selected based on the WebRTC algorithm using the bandwidth estimation.

For the receiver

Conference participants receive the highest-quality streams, based on three variables:

  • The stream quality sent by the sender.
  • The stream quality requested by the receiver.
  • The stream quality estimated based on the receiver's available bandwidth.

The stream selection first estimates the available bandwidth of all conference participants using the REMB packets. Then, it subtracts from these values the bit rate that is needed for transmitting standard streams (such as screen sharing and non-Simulcast video transmission). The results of this calculation are divided by the number of Simulcast participants. Based on these results, the highest quality streams are selected.

The user cannot receive a video if his available bandwidth is lower than a bit rate of the sent SD Simulcast stream.

Available APIs

The SDK 2.1 introduces new APIs for Simulcast configuration.

APIs for desktop web platforms

To send Simulcast streams, clients should configure the join method with optional JoinOptions parameters. The Simulcast parameter turns the Simulcast transmission on and off, even at the start of the call.

A simplest example of the joinOptions:

let options = {simulcast: true}

A more complex example of video constraints:

const constraints = {
  audio: true,
  video: {
    width: {
      min: "320",
      max: "1280"
    },
    height: {
      min: "240",
      max: "720"
    }
  }
};

options = {constraints: constraints}

VoxeetSDK.conference.join(conference, options)
  .then((info) => {
  })
  .catch((error) => {
  });

To receive Simulcast streams and, therefore, limit or cap the stream quality, the client can use the Simulcast API. It includes the ParticipantQuality (either HD, MD, or SD ) with the participant's ID and quality parameters, allowing selection of stream quality whenever it is needed:


  "participantId": Quality
}]);

APIs for mobile platforms (Android and iOS)

To receive Simulcast streams, mobile clients should use the following APIs asking for a specific stream quality:

These APIs allow stream selection any time it is needed.

Requirements

Currently, only desktop web users can send video Simulcast streams. (See the following table.) All users are able to receive a specific quality of video streams.

Operating system Supported browser Version Sending Simulcast Receiving selected quality stream
Windows 10 Google Chrome 78, 79 Yes Yes
Firefox 72, 73 No Yes
macOS 10.14, 10.15 Safari 13, 12 No Yes
Google Chrome 78, 79 Yes Yes
Firefox 72, 73 No Yes
iOS 13 Mobile Safari Latest No Yes
Native iOS SDK 2.1 No Yes
Android 9, 10 Mobile Chrome Latest No Yes
Native Android SDK 2.1 No Yes

The SDK 2.1 supports only the H.264 codec.