Audio Mixer

An Audio Mixer is a Media Extension that mixes a Programmable Video Room's participants' audio and makes it available for streaming.

The Audio Mixer joins a Twilio Video Room as a participant. It mixes the other participants' audio and then sends the mixed audio to a PlayerStreamer. The Audio Mixer also sends Timed Metadata about the Room's participants to the connected PlayerStreamer every 500ms.

If the Video Room that this Audio Mixer is connected to ends (if its status is set to COMPLETED or it ends from a failure), the MediaProcessor will continue running. While the Room is active, the Audio Mixer will stay connected as a participant even if there are no other participants in the Video Room. You will need to explicitly end the MediaProcessor when you are finished with the livestream; see Billing and Resource Management for more information.

Get Started

To use an Audio Mixer in a Twilio Live application, specify the Audio Mixer extension name and extension context parameters when creating a MediaProcessor.

Extension Names

Below are the recommended Audio Mixer extension names. To know more about released extension names, refer to the Audio Mixer Changelog:

Extension Context

The snippet below describes the Audio Mixer extension context parameters.

{
  // Room Parameters (required)
  "room": {
      // The Programmable Video Room unique name or SID (required)
      "name": "RM123"
  },

  // Identity that the Extension uses when joining the Room (optional, audio-mixer-v2 by default)
  "identity": "audio-mixer-v2",

  // An array of media destinations represented as PlayerStreamer SIDs (required)
  "outputs": ["VJXXX"],

  // Audio bitrate in Kbps (optional, defaults to 128)
  "audioBitrate": 128
}

Video Room creation

The Audio Mixer joins a Twilio Video Room as a participant. The default maximum Participant duration is 4 hours, but can be configured for up to 24 hours. To increase the maximum Participant duration for a Twilio Video Room, please visit the Video documentation and follow the instructions. You can configure the maximum Participant duration via the REST API or within the Twilio Console.

If Client-side Room Creation is enabled for your account and a Video Room with the specified unique name or SID does not already exist, the Audio Mixer will create one with this value in the unique name field and then join that Video Room. You can configure Client-side Room Creation in the Twilio Console.

Client-side Room Creation is enabled for accounts by default.

If Client-side Room Creation is disabled and you provide a Video Room name or SID that does not exist, the Audio Mixer will still start but will not create or connect to a Video Room.

Note that the Audio Mixer only tries to connect to a Video Room once. If the Video Room does not exist and Client-side Room Creation is disabled, the Audio Mixer will not connect to a Video Room. It will also not connect if a Video Room with the same unique name or SID is eventually created after the Audio Mixer has started.

Creating a MediaProcessor with an Audio Mixer

The snippet below demonstrates an example curl request for creating a MediaProcessor with an Audio Mixer Media Extension:

curl -X POST 'https://media.twilio.com/v1/MediaProcessors' \
  -u $TWILIO_API_KEY_SID:$TWILIO_API_KEY_SECRET \
  -d 'Extension="audio-mixer-v2"' \
  -d 'ExtensionContext={"room": {"name": "RM123"}, "outputs": ["VJXXX"]}'

Timed Metadata

The Audio Mixer sends information about a Room's participants every 500ms to the connected PlayerStreamer as a JSON object. That Timed Metadata can then be accessed with the Player SDK in either a web or mobile application.

The Timed Metadata payload includes a p object that contains information about each participant and an s object that represents a sequence ID. Below is an example payload for a room that contains two participants:

{
  "s": 23,
  "p": {
    "alice" : {"v":300},
    "bob" : {"v":-1} // bob is muted
  }
}

Each key within the p participant object represents a participant's identity, and the value of that key is another JSON object that contains a v key. The v key represents the participant's volume level; -1 indicates that the participant is muted.

The s sequence ID is provided to help with ordering the Timed Metadata payloads. If you access the Timed Metadata in your client-side application via the Player SDK, you should inspect the payload's sequence ID to make sure you are receiving the payloads in order. We recommend that you discard any message payload that you receive out of sequence.