Audio Mixer Changelog

2.2.0 (July 14th, 2022)

We made a change to improve the performance of the audio-mixer.

2.1.0 (June 8th, 2022)

The audio-mixer now uses a new API for Inband Timed Metadata.

2.0.0 (June 8th, 2022)

Breaking change: Previously, the audio-mixer would disconnect and then reconnect from a Room after 4 hours (the default maximum Participant duration). The audio-mixer will now no longer reconnect after reaching the maximum Participant duration, and the default maximum Participant duration is now configurable. The default maximum Participant duration is still 4 hours, but can be configured for up to 24 hours.

Previously, audio-mixer would disconnect and reconnect from the Room every 3h 55m, because a Participant could be connected to the Room for a maximum of 4 hours. Following recent changes to Twilio Video Rooms, a Room's owner can now set this time limit to up to 24 hours.

To increase the maximum Participant duration for a Room, please visit the Video documentation and follow the instructions. You can configure the maximum Participant duration via the REST API or within the Twilio Console.

1.0.0 (October 20th, 2021)

The 1.0.0-beta.1 Audio Mixer has been promoted to 1.0.0 and is now Generally Available! Thank you for all the feedback from our beta customers. For more details about the Audio Mixer, please visit the documentation.

1.0.0-beta.1 (October 13th, 2021)

A Media Extension that mixes a Programmable Video Room's participants' audio tracks.

Features

• Automatically connects and disconnects to a Programmable Video Room.
• Mixes all Participants' audio tracks into a Twilio Live Stream.
• Inserts TimedMetadata that can be used for building a rich application UI. See TimedMetadata section for more details.

Getting Started

To use an Audio Mixer in a Twilio Live application, specify the audio mixer extension name and extension context parameters when creating a MediaProcessor.

curl -X POST 'https://media.twilio.com/v1/MediaProcessors' \
  -u 'SKXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX:your_api_key_secret' \
  -d 'Extension="audio-mixer-v1"' \
  -d 'ExtensionContext={"room": {"name": "RM123"}, "outputs": ["VJXXX"]}'

Extension Context

The snippet below describes the Audio Mixer extension context parameters.

{
  // Room Parameters (required)
  "room": {
      "name": "RM123", // required
  },

  // Extension Identity (optional, audio-mixer-v1 by default)
  "identity": "audio-mixer-v1",

  // Media destinations (required)
  "outputs": ["VJXXX"],

  // Audio bitrate (optional)
  "audioBitrate": 128,
}

Timed Metadata

The audio mixer gets a Room's participants every 500 ms and inserts this information into the stream via the TimedMetadata API as a JSON object. The payload includes a "p" object that stands for participants and an "s" object that represents a sequence ID. Each key within the "p" object represents a participant’s identity and the value contains a JSON object. The JSON object contains a "v" key which represents the participant’s volume level. This option saves space via abbreviated keys but also allows for additions to the payload. Applications are encouraged to inspect a payload’s sequence ID and discard a message payload received out-of-order. The snippet below describes an example payload with two participants.

{
  "s": 23, // sequence id
  "p": {
    "alice" : {"v":300},
    "bob" : {"v":-1} // bob is muted
  }
}

To process this TimedMetadata payload in your client application, observe TimedMetadata events in the Player SDK and parse the TimedMetadata payload as a JSON object.