Video Composer

A Video Composer is a Media Extension that composes a Programmable Video Room's participants' videos in an adaptive grid layout, with the option for a presentation mode.

The Video Composer joins a Twilio Video Room as a participant. It formats the videos of participants who have published tracks in an adaptive grid format with their identities written as an overlay on the HTML Video elements. It then sends the Room's video and audio content to a PlayerStreamer. By default the resolution of the livestream that is encoded is 720p with landscape aspect ratio, however the resolution can be configured and portrait aspect ratio is also supported for mobile first livestreams.

The Video Composer will display up to nine videos in the grid layout. If there are more than nine participants in the Room, the Video Composer will render the videos for the last nine active speakers. The Video Composer will highlight the currently active speaker's video with a green border. To hide a participant with published tracks from the grid, refer to the Track Subscription API documentation..

The Video Composer also supports presentation mode, where one video track is highlighted as the largest track on the screen, and the list of other speakers are displayed as thumbnails on the right of the main video track. The list of speakers on the right will display the five most recently active speakers, and the list will change as the active speakers change.

If the Video Room that this Video Composer is connected to ends (if its status is set to COMPLETED or it ends from a failure), the MediaProcessor will continue running. The Video Composer will disconnect from the Video Room if it is the only participant in the room for five minutes.

You will need to explicitly end the MediaProcessor when you are finished with the livestream or configure the MediaProcessor's maximum duration so that the stream automatically terminates after a specified amount of time. See Billing and Resource Management for more information.

Get Started

To use a Video Composer in a Twilio Live application, specify the Video Composer extension name and extension context parameters when creating a MediaProcessor.

Extension Names

Below are the recommended Video Composer extension names. To know more about released extension names, refer to the Video Composer Changelog:

Extension Context

The snippet below describes the Video Composer extension context parameters. The MediaProcessor uses these extension context parameters to create and start the Video Composer application. For example, the room parameter tells the Video Composer which Video Room to connect to, and the outputs parameter specifies where the composed video stream should be sent.

{
  // Room Parameters (required)
  "room": {
      // The Programmable Video Room unique name or SID (required)
      "name": "RM123"
  },

  // Identity that the Extension uses when joining the Room (optional, video-composer-v2 by default)
  "identity": "video-composer-v2",

  // An array of media destinations represented as PlayerStreamer SIDs (required)
  "outputs": ["VJXXX"],

  // Audio bitrate in Kbps (optional, defaults to 128)
  "audioBitrate": 128,

    // Resolution (optional, defaults to "1280x720")
  "resolution": "1920x1080"
}

Extension Environment

The snippet below describes the Video Composer extension environment parameters. Extension environment parameters are injected as environment variables into the Video Composer JavaScript application, and used to configure specific behaviors in the application.

{
  // Prevents participant identities from being displayed in the Video Composer (optional, false by default)
  "hideParticipantIdentities": true,

  // Layout orientation of the video tiles, when not provided it defaults to "landscape"
  "videoTileOrientation": "portrait"

  // Set a HEX color for the background of the Video Composer (optional, #121c2d by default)
  "backgroundColor": "#121c2d"
}

Video Room creation

The Video Composer joins a Twilio Video Room as a participant. The default maximum Participant duration is 4 hours, but can be configured for up to 24 hours. To increase the maximum Participant duration for a Twilio Video Room, please visit the Video documentation and follow the instructions. You can configure the maximum Participant duration via the REST API or within the Twilio Console.

If Client-side Room Creation is enabled for your account and a Video Room with the specified unique name or SID does not already exist, the Video Composer will create one with this value in the unique name field and then join that Video Room. You can configure Client-side Room Creation in the Twilio Console.

Client-side Room Creation is enabled for accounts by default.

If Client-side Room Creation is disabled and you provide a Video Room name or SID that does not exist, the Video Composer will still start but will not create or connect to a Video Room.

Note that the Video Composer only tries to connect to a Video Room once. If the Video Room does not exist and Client-side Room Creation is disabled, the Video Composer will not connect to a Video Room. It will also not connect if a Video Room with the same unique name or SID is eventually created after the Video Composer has started.

Creating a MediaProcessor with a Video Composer

The snippet below demonstrates an example curl request for creating a MediaProcessor with a Video Composer Media Extension:

curl -X POST 'https://media.twilio.com/v1/MediaProcessors' \
  -u $TWILIO_API_KEY_SID:$TWILIO_API_KEY_SECRET \
  -d 'Extension="video-composer-v2"' \
  -d 'ExtensionContext={"room": {"name": "RM123"}, "outputs": ["VJXXX"]}'

Presentation Mode

To enable presentation mode, you can publish a video track with the name video-composer-presentation. Once you publish a video track with this name, that track will become the largest video on the screen, and the videos for other speakers will move to the right of the presentation. The list of other speakers will include the five most recently active speakers. If there are more than five other speakers in the stream, this list will update as the active speakers change.

The Video Composer only supports one video track with the name video-composer-presentation at a time. If you publish multiple video tracks with this name, only the first one will be rendered as the main video in the presentation. Audio or data tracks with the name video-composer-presentation will also not change the Video Composer's layout.

The Video Composer will return to the adaptive grid layout after you unpublish the video track named video-composer-presentation.