Audio Mixer
We are no longer allowing new customers to onboard to Twilio Live. Effective November 30, 2023, Twilio Live will End of Life. We have created this Migration Guide to help you identify an alternative solution for your use case.
An Audio Mixer is a Media Extension that mixes a Programmable Video Room's participants' audio and makes it available for streaming.
The Audio Mixer joins a Twilio Video Room as a participant. It mixes the other participants' audio and then sends the mixed audio to a PlayerStreamer. The Audio Mixer also sends Timed Metadata about the Room's participants to the connected PlayerStreamer every 500ms.
If the Video Room that this Audio Mixer is connected to ends (if its status is set to COMPLETED
or it ends from a failure), the MediaProcessor will continue running. While the Room is active, the Audio Mixer will stay connected as a participant even if there are no other participants in the Video Room. You will need to explicitly end the MediaProcessor when you are finished with the livestream; see Billing and Resource Management for more information.
Get Started
To use an Audio Mixer in a Twilio Live application, specify the Audio Mixer extension name and extension context parameters when creating a MediaProcessor.
Extension Names
Below are the recommended Audio Mixer extension names. To know more about released extension names, refer to the Audio Mixer Changelog:
Name | Description |
---|---|
audio-mixer-v2 |
The latest stable Audio Mixer extension. |
Extension Context
The snippet below describes the Audio Mixer extension context parameters.
{
// Room Parameters (required)
"room": {
// The Programmable Video Room unique name or SID (required)
"name": "RM123"
},
// Identity that the Extension uses when joining the Room (optional, audio-mixer-v2 by default)
"identity": "audio-mixer-v2",
// An array of media destinations represented as PlayerStreamer SIDs (required)
"outputs": ["VJXXX"],
// Audio bitrate in Kbps (optional, defaults to 128)
"audioBitrate": 128
}
Name | Description |
---|---|
room | A required object that results in the Audio Mixer being injected with all the required parameters to connect to a Programmable Video Room. The room object should contain the unique name or SID for the Programmable Video Room you would like to stream. See Video Room creation for more information about how the Audio Mixer joins the Room. |
identity | The identity associated with the Audio Mixer. Defaults to the extension name and major version (eg. audio-mixer-v2 ) |
outputs | An array of media destinations represented as PlayerStreamer SIDs. |
audioBitrate | The audio bitrate in Kbps. Can be either 128 or 192. Defaults to 128. |
Video Room creation
The Audio Mixer joins a Twilio Video Room as a participant. The default maximum Participant duration is 4 hours, but can be configured for up to 24 hours. To increase the maximum Participant duration for a Twilio Video Room, please visit the Video documentation and follow the instructions. You can configure the maximum Participant duration via the REST API or within the Twilio Console.
If Client-side Room Creation is enabled for your account and a Video Room with the specified unique name or SID does not already exist, the Audio Mixer will create one with this value in the unique name field and then join that Video Room. You can configure Client-side Room Creation in the Twilio Console.
Client-side Room Creation is enabled for accounts by default.
If Client-side Room Creation is disabled and you provide a Video Room name or SID that does not exist, the Audio Mixer will still start but will not create or connect to a Video Room.
Note that the Audio Mixer only tries to connect to a Video Room once. If the Video Room does not exist and Client-side Room Creation is disabled, the Audio Mixer will not connect to a Video Room. It will also not connect if a Video Room with the same unique name or SID is eventually created after the Audio Mixer has started.
Creating a MediaProcessor with an Audio Mixer
The snippet below demonstrates an example curl request for creating a MediaProcessor with an Audio Mixer Media Extension:
curl -X POST 'https://media.twilio.com/v1/MediaProcessors' \
-u $TWILIO_API_KEY_SID:$TWILIO_API_KEY_SECRET \
-d 'Extension="audio-mixer-v2"' \
-d 'ExtensionContext={"room": {"name": "RM123"}, "outputs": ["VJXXX"]}'
Timed Metadata
The Audio Mixer sends information about a Room's participants every 500ms to the connected PlayerStreamer as a JSON object. That Timed Metadata can then be accessed with the Player SDK in either a web or mobile application.
The Timed Metadata payload includes a p
object that contains information about each participant and an s
object that represents a sequence ID. Below is an example payload for a room that contains two participants:
{
"s": 23,
"p": {
"alice" : {"v":300},
"bob" : {"v":-1} // bob is muted
}
}
Each key within the p
participant object represents a participant's identity, and the value of that key is another JSON object that contains a v
key. The v
key represents the participant's volume level; -1 indicates that the participant is muted.
The s
sequence ID is provided to help with ordering the Timed Metadata payloads. If you access the Timed Metadata in your client-side application via the Player SDK, you should inspect the payload's sequence ID to make sure you are receiving the payloads in order. We recommend that you discard any message payload that you receive out of sequence.
Need some help?
We all do sometimes; code is hard. Get help now from our support team, or lean on the wisdom of the crowd by visiting Twilio's Stack Overflow Collective or browsing the Twilio tag on Stack Overflow.