Skip to contentSkip to navigationSkip to topbar
Rate this page:
On this page

TwiML™️ Voice: <Stream>


(warning)

Warning

This Twiml Verb is not currently available when using Twilio Regions Ireland (IE1) or Australia (AU1). This is currently only supported with the default US1 region. A full list of unsupported products and features with Twilio Regions is documented here.

The <Stream> instruction allows you to receive raw audio streams from a live phone call over WebSockets in near real-time.

The most basic use of <Stream>:


_10
<?xml version="1.0" encoding="UTF-8"?>
_10
<Response>
_10
<Start>
_10
<Stream url="wss://mystream.ngrok.io/audiostream" />
_10
</Start>
_10
</Response>

This TwiML will instruct Twilio to fork the audio stream of the current call and send it in real-time over WebSocket to wss://mystream.ngrok.io/audiostream.

The <Start> verb starts the audio <Stream> asynchronously and immediately continues with the next TwiML instruction. If there is no instruction, the call will be disconnected. In order to avoid this, provide a TwiML instruction to continue the call.

If you'd prefer a synchronous bi-directional stream, you should use the <Connect> verb.

There is a one to one mapping of a stream to a WebSocket connection, therefore there will be at most one call being streamed over a single WebSocket connection. Metadata will be provided so that you can handle multiple inbound connections and manage the mappings between the unique stream identifier or StreamSid.

(information)

Info

If communication issues are encountered with your WebSocket server, they will be reported in the Twilio Debugger with additional information about the failure.

(warning)

Warning

There are a maximum of 4 forked streams allowed per call. Each track, inbound or outbound, is a forked stream.


Attributes

attributes page anchor

<Stream> supports the following attributes:

Attribute NameAllowed ValuesDefault Value
urlrelative or absolute URLnone
nameOptional. Unique name for the Streamnone
trackOptional. inbound_track, outbound_track, both_tracksinbound_track
statusCallbackOptional. Relative or absolute URLnone
statusCallbackMethodOptional. GET or POSTPOST

url

attributes-url page anchor

The url attribute accepts a relative or absolute url. On successful execution, a WebSocket connection to the url will be established and audio will start streaming.wss is the only supported protocol.

The url does not support query string parameters. To pass custom key value pairs to the WebSocket, make use of Custom Parameters instead.

Providing a name will allow you to reference the stream directly. This name must be unique per Call. For instance, stopping a Stream by name.

The track attribute allows you to optionally request to receive a specific track of a call. On any given active call, there are inbound and outbound tracks. Inbound represents the audio Twilio receives from the call and outbound represents the audio generated by Twilio to the call.

By default, Twilio always streams the inbound track of a call. When a Stream is started using the synchronous bi-directional <Connect> verb, this is your only option.

However, if you are using an asynchronous Stream by using the <Start> verb, you can request the audio it generates by choosing outbound_track. To receive both tracks of a call use both_tracks. If both_tracks is used, you will receive both the inbound media event and outbound media event.

The statusCallback attribute takes an absolute or relative URL as value. Whenever a stream is started or stopped, Twilio will make a request to this URL with the following parameters:

ParameterDescription
AccountSidThe unique identifier of the Account responsible for this Stream.
CallSidThe unique identifier of the Call
StreamSidThe unique identifier for this Stream
StreamNameIf defined, this is the unique name of the Stream. Defaults to the StreamSid
StreamEventOne of stream-started, stream-stopped, or stream-error (see StreamError for the message)
StreamErrorIf an error has occurred, this will contain a detailed error message.
TimestampThe time of the event in ISO 8601 format(link takes you to an external page)

The HTTP method to use when requesting the statusCallback URL. Default is POST.


It is possible to include additional key value pairs that will passed along with your stream. You can do this by using the nested <Parameter> TwiML noun.


_10
<?xml version="1.0" encoding="UTF-8"?>
_10
<Response>
_10
<Start>
_10
<Stream url="wss://mystream.ngrok.io/example" >
_10
<Parameter name="FirstName" value ="Jane"/>
_10
<Parameter name="LastName" value ="Doe" />
_10
<Parameter name="RemoteParty" value ="Bob" />
_10
</Stream>
_10
</Start>
_10
</Response>

These values will be sent along with the Start WebSocket message.

(warning)

Warning

There is a currently a size limitation in the amount of data that can be sent from custom parameters. Aim to keep the total characters of your parameters (both name and value) under 500 characters.


It is possible to stop an asynchronous stream by name. For instance by naming the Stream my_first_stream.


_10
<Start>
_10
<Stream name="my_first_stream" url="wss://mystream.ngrok.io/audiostream" />
_10
</Start>

You can later use the unique name of my_first_stream to stop the stream.


_10
<Stop>
_10
<Stream name="my_first_stream" />
_10
</Stop>


WebSocket Messages - From Twilio

websocket-messages---from-twilio page anchor

There are separate types of events that occur during the Stream's life cycle. These events are represented via WebSocket Messages: Connected, Start, Media, Stop, and the bi-directional only Mark.

Each message sent is a JSON string. You can determine which type of event is occurring by using the event property of every JSON object.

The following messages are sent from Twilio to your application:

The first message sent once a WebSocket connection is established is the Connected event. This message describes the protocol to expect in the following messages.

ParameterDescription
eventThe value of connected
protocolDefines the protocol for the WebSocket connections lifetime. eg: "Call"
versionSemantic version of the protocol.

Example Connected Message

example-connected-message page anchor

_10
{
_10
"event": "connected",
_10
"protocol": "Call",
_10
"version": "1.0.0"
_10
}

This message contains important metadata about the stream and is sent immediately after the Connected message. It is only sent once at the start of the Stream.

ParameterDescription
eventThe value of start
sequenceNumberNumber used to keep track of message sending order. First message starts with "1" and then is incremented.
startAn object containing Stream metadata
start.streamSidThe unique identifier of the Stream
start.accountSidThe Account identifier that created the Stream
start.callSidThe Call identifier from where the Stream was started.
start.tracksAn array of values that indicates what media flows to expect in subsequent messages. Values include inbound, outbound.
start.customParametersAn object that represents the Custom Parameters that where set when defining the Stream
start.mediaFormatAn object containing the format of the payload in the Media Messages.
start.mediaFormat.encodingThe encoding of the data in the upcoming payload. Value will always be audio/x-mulaw.
start.mediaFormat.sampleRateThe Sample Rate in Hertz of the upcoming audio data. Value is always 8000
start.mediaFormat.channelsThe number of channels in the input audio data. Value will always be 1
streamSidThe unique identifier of the Stream

_24
{
_24
"event": "start",
_24
"sequenceNumber": "2",
_24
"start": {
_24
"streamSid": "MZ18ad3ab5a668481ce02b83e7395059f0",
_24
"accountSid": "AC123",
_24
"callSid": "CA123",
_24
"tracks": [
_24
"inbound",
_24
"outbound"
_24
],
_24
"customParameters": {
_24
"FirstName": "Jane",
_24
"LastName": "Doe",
_24
"RemoteParty": "Bob",
_24
},
_24
"mediaFormat": {
_24
"encoding": "audio/x-mulaw",
_24
"sampleRate": 8000,
_24
"channels": 1
_24
}
_24
},
_24
"streamSid": "MZ18ad3ab5a668481ce02b83e7395059f0"
_24
}

This message type encapsulates the raw audio data.

ParameterDescription
eventThe value of media
sequenceNumberNumber used to keep track of message sending order. First message starts with "1" and then is incremented for each message.
mediaAn object containing media metadata and payload
media.trackOne of inbound or outbound
media.chunkThe chunk for the message. The first message will begin with "1" and increment with each subsequent message.
media.timestampPresentation Timestamp in Milliseconds from the start of the stream.
media.payloadRaw audio in encoded in base64
streamSidThe unique identifier of the Stream
Outbound Track
message-outbound-media page anchor

_11
{
_11
"event": "media",
_11
"sequenceNumber": "3",
_11
"media": {
_11
"track": "outbound",
_11
"chunk": "1",
_11
"timestamp": "5",
_11
"payload": "no+JhoaJjpzSHxAKBgYJDhtEopGKh4aIjZm7JhILBwYIDRg1qZSLh4aIjJevLBUMBwYHDBUsr5eMiIaHi5SpNRgNCAYHCxImu5mNiIaHipGiRBsOCQYGChAf0pyOiYaGiY+e/x4PCQYGCQ4cUp+QioaGiY6bxCIRCgcGCA0ZO6aSi4eGiI2YtSkUCwcGCAwXL6yVjIeGh4yVrC8XDAgGBwsUKbWYjYiGh4uSpjsZDQgGBwoRIsSbjomGhoqQn1IcDgkGBgkPHv+ej4mGhomOnNIfEAoGBgkOG0SikYqHhoiNmbsmEgsHBggNGDWplIuHhoiMl68sFQwHBgcMFSyvl4yIhoeLlKk1GA0IBgcLEia7mY2IhoeKkaJEGw4JBgYKEB/SnI6JhoaJj57/Hg8JBgYJDhxSn5CKhoaJjpvEIhEKBwYIDRk7ppKLh4aIjZi1KRQLBwYIDBcvrJWMh4aHjJWsLxcMCAYHCxQptZiNiIaHi5KmOxkNCAYHChEixJuOiYaGipCfUhwOCQYGCQ8e/56PiYaGiY6c0h8QCgYGCQ4bRKKRioeGiI2ZuyYSCwcGCA0YNamUi4eGiIyXrywVDAcGBwwVLK+XjIiGh4uUqTUYDQgGBwsSJruZjYiGh4qRokQbDgkGBgoQH9KcjomGhomPnv8eDwkGBgkOHFKfkIqGhomOm8QiEQoHBggNGTumkouHhoiNmLUpFAsHBggMFy+slYyHhoeMlawvFwwIBgcLFCm1mI2IhoeLkqY7GQ0IBgcKESLEm46JhoaKkJ9SHA4JBgYJDx7/no+JhoaJjpzSHxAKBgYJDhtEopGKh4aIjZm7JhILBwYIDRg1qZSLh4aIjJevLBUMBwYHDBUsr5eMiIaHi5SpNRgNCAYHCxImu5mNiIaHipGiRBsOCQYGChAf0pyOiYaGiY+e/x4PCQYGCQ4cUp+QioaGiY6bxCIRCgcGCA0ZO6aSi4eGiI2YtSkUCwcGCAwXL6yVjIeGh4yVrC8XDAgGBwsUKbWYjYiGh4uSpjsZDQgGBwoRIsSbjomGhoqQn1IcDgkGBgkPHv+ej4mGhomOnNIfEAoGBgkOG0SikYqHhoiNmbsmEgsHBggNGDWplIuHhoiMl68sFQwHBgcMFSyvl4yIhoeLlKk1GA0IBgcLEia7mY2IhoeKkaJEGw4JBgYKEB/SnI6JhoaJj57/Hg8JBgYJDhxSn5CKhoaJjpvEIhEKBwYIDRk7ppKLh4aIjZi1KRQLBwYIDBcvrJWMh4aHjJWsLxcMCAYHCxQptZiNiIaHi5KmOxkNCAYHChEixJuOiYaGipCfUhwOCQYGCQ8e/56PiYaGiY6c0h8QCgYGCQ4bRKKRioeGiA=="
_11
} ,
_11
"streamSid": "MZ18ad3ab5a668481ce02b83e7395059f0"
_11
}


_12
_12
{
_12
"event": "media",
_12
"sequenceNumber": "4",
_12
"media": {
_12
"track": "inbound",
_12
"chunk": "2",
_12
"timestamp": "5",
_12
"payload": "no+JhoaJjpzSHxAKBgYJDhtEopGKh4aIjZm7JhILBwYIDRg1qZSLh4aIjJevLBUMBwYHDBUsr5eMiIaHi5SpNRgNCAYHCxImu5mNiIaHipGiRBsOCQYGChAf0pyOiYaGiY+e/x4PCQYGCQ4cUp+QioaGiY6bxCIRCgcGCA0ZO6aSi4eGiI2YtSkUCwcGCAwXL6yVjIeGh4yVrC8XDAgGBwsUKbWYjYiGh4uSpjsZDQgGBwoRIsSbjomGhoqQn1IcDgkGBgkPHv+ej4mGhomOnNIfEAoGBgkOG0SikYqHhoiNmbsmEgsHBggNGDWplIuHhoiMl68sFQwHBgcMFSyvl4yIhoeLlKk1GA0IBgcLEia7mY2IhoeKkaJEGw4JBgYKEB/SnI6JhoaJj57/Hg8JBgYJDhxSn5CKhoaJjpvEIhEKBwYIDRk7ppKLh4aIjZi1KRQLBwYIDBcvrJWMh4aHjJWsLxcMCAYHCxQptZiNiIaHi5KmOxkNCAYHChEixJuOiYaGipCfUhwOCQYGCQ8e/56PiYaGiY6c0h8QCgYGCQ4bRKKRioeGiI2ZuyYSCwcGCA0YNamUi4eGiIyXrywVDAcGBwwVLK+XjIiGh4uUqTUYDQgGBwsSJruZjYiGh4qRokQbDgkGBgoQH9KcjomGhomPnv8eDwkGBgkOHFKfkIqGhomOm8QiEQoHBggNGTumkouHhoiNmLUpFAsHBggMFy+slYyHhoeMlawvFwwIBgcLFCm1mI2IhoeLkqY7GQ0IBgcKESLEm46JhoaKkJ9SHA4JBgYJDx7/no+JhoaJjpzSHxAKBgYJDhtEopGKh4aIjZm7JhILBwYIDRg1qZSLh4aIjJevLBUMBwYHDBUsr5eMiIaHi5SpNRgNCAYHCxImu5mNiIaHipGiRBsOCQYGChAf0pyOiYaGiY+e/x4PCQYGCQ4cUp+QioaGiY6bxCIRCgcGCA0ZO6aSi4eGiI2YtSkUCwcGCAwXL6yVjIeGh4yVrC8XDAgGBwsUKbWYjYiGh4uSpjsZDQgGBwoRIsSbjomGhoqQn1IcDgkGBgkPHv+ej4mGhomOnNIfEAoGBgkOG0SikYqHhoiNmbsmEgsHBggNGDWplIuHhoiMl68sFQwHBgcMFSyvl4yIhoeLlKk1GA0IBgcLEia7mY2IhoeKkaJEGw4JBgYKEB/SnI6JhoaJj57/Hg8JBgYJDhxSn5CKhoaJjpvEIhEKBwYIDRk7ppKLh4aIjZi1KRQLBwYIDBcvrJWMh4aHjJWsLxcMCAYHCxQptZiNiIaHi5KmOxkNCAYHChEixJuOiYaGipCfUhwOCQYGCQ8e/56PiYaGiY6c0h8QCgYGCQ4bRKKRioeGiA=="
_12
},
_12
"streamSid": "MZ18ad3ab5a668481ce02b83e7395059f0"
_12
}

A stop message will be sent when the Stream is either <Stop>ped or the Call has ended.

ParameterDescription
eventThe value of stop
sequenceNumberNumber used to keep track of message sending order. First message starts with "1" and then is incremented for each message.
stopAn object containing Stream metadata
stop.accountSidThe Account identifier that created the Stream
stop.callSidThe Call identifier that started the Stream
streamSidThe unique identifier of the Stream

_10
{
_10
"event": "stop",
_10
"sequenceNumber": "5",
_10
"stop": {
_10
"accountSid": "AC123",
_10
"callSid": "CA123"
_10
},
_10
"streamSid": "MZ18ad3ab5a668481ce02b83e7395059f0"
_10
}

A DTMF message will be sent when someone presses a touch-tone number key (such as the "1" key in the example message below) in the inbound stream, typically in response to a prompt in the outbound stream.

ParameterDescription
eventThe value of dtmf
streamSidThe unique identifier of the Stream
sequenceNumberNumber used to keep track of message sending order. First message starts with "1" and then is incremented for each message.
dtmf.track"inbound_track"
dtmf.digitthe number-key tone detected

_10
_10
{
_10
"event": "dtmf",
_10
"streamSid":"MZ18ad3ab5a668481ce02b83e7395059f0",
_10
"sequenceNumber":"5",
_10
"dtmf": {
_10
"track":"inbound_track",
_10
"digit": "1"
_10
}
_10
}

Th mark event is sent only during bi-directional streaming by using the <Connect> verb. It is used to track, or label, when media has completed.

ParameterDescription
eventThe value of mark
sequenceNumberNumber used to keep track of message sending order. First message starts with "1" and then is incremented for each message.
markAn object containing the mark metadata
mark.nameThe value specified when creating the mark message to Twilio

_10
{
_10
"event": "mark",
_10
"sequenceNumber": "4",
_10
"streamSid": "MZ18ad3ab5a668481ce02b83e7395059f0",
_10
"mark": {
_10
"name": "my label"
_10
}
_10
}

WebSocket Messages - To Twilio

websocket-messages---to-twilio page anchor

The events that you can send back to Twilio are Media, Mark, and Clear.

(information)

Info

In order to send data back to Twilio, your stream must have been initialized using the <Connect> TwiML verb. This will give you a bi-directional Stream which allows you to pipe audio and control the flow.

To send media back to Twilio, you must provide a similarly formattedmedia message. The payload must be encoded audio/x-mulaw with a sample rate of 8000 and base64 encoded. The audio can be of any size.

The media messages will be buffered and played in the order received. If you'd like interrupt the buffered audio, see the clear event message.

(warning)

Warning

The media payload should not contain audio file type header bytes. Providing header bytes will cause the media to be streamed incorrectly.

ParameterDescription
eventThe value of media
streamSidThe SID of the Stream that should play back the audio
mediaAn object containing media metadata and payload
media.payloadRaw mulaw/8000 audio in encoded in base64

_10
{
_10
"event": "media",
_10
"streamSid": "MZ18ad3ab5a668481ce02b83e7395059f0",
_10
"media": {
_10
"payload": "a3242sadfasfa423242... (a base64 encoded string of 8000/mulaw)"
_10
}
_10
}

Send a mark event message after sending a media event message to be notified when the audio that you have sent has been completed. You'll receive a mark event with a matching name from Twilio when the audio ends (or if there is no audio buffered).

You will also receive an incoming mark event message if the buffer was cleared using the clear event message.

ParameterDescription
eventThe value of mark
streamSidThe SID of the Stream that should receive the mark
markAn object containing mark metadata and payload
mark.nameA name specific to your needs that will assist in recognizing future received mark event

_10
{
_10
"event": "mark",
_10
"streamSid": "MZ18ad3ab5a668481ce02b83e7395059f0",
_10
"mark": {
_10
"name": "my label"
_10
}
_10
}

Send the clear event message if you would like to interrupt the audio that has been sent various media event messages. This will empty all buffered audio and cause any mark event messages to be sent back to you.

ParameterDescription
eventThe value of clear
streamSidThe SID of the Stream that should receive the mark

_10
{
_10
"event": "clear",
_10
"streamSid": "MZ18ad3ab5a668481ce02b83e7395059f0",
_10
}


Start a new asynchronous MediaStream named "Example Audio Stream". The stream will begin sending messages to the url specified over a WebSocket connection.

Node.js
Python
C#
Java
PHP
Ruby

_10
const VoiceResponse = require('twilio').twiml.VoiceResponse;
_10
_10
const response = new VoiceResponse();
_10
const start = response.start();
_10
start.stream({
_10
name: 'Example Audio Stream',
_10
url: 'wss://mystream.ngrok.io/audiostream'
_10
});
_10
_10
console.log(response.toString());

Output

_10
<?xml version="1.0" encoding="UTF-8"?>
_10
<Response>
_10
<Start>
_10
<Stream name="Example Audio Stream" url="wss://mystream.ngrok.io/audiostream" />
_10
</Start>
_10
</Response>

You can send additional custom information by using the <Parameter> TwiML noun. These values will be delivered in the Start WebSocket message in the customParameters section.

Provide Custom Parameters to a MediaStream on creation

provide-custom-parameters-to-a-mediastream-on-creation page anchor
Node.js
Python
C#
Java
PHP
Ruby

_17
const VoiceResponse = require('twilio').twiml.VoiceResponse;
_17
_17
const response = new VoiceResponse();
_17
const start = response.start();
_17
const stream = start.stream({
_17
url: 'wss://mystream.ngrok.io/example'
_17
});
_17
stream.parameter({
_17
name: 'FirstName',
_17
value: 'Jane'
_17
});
_17
stream.parameter({
_17
name: 'LastName',
_17
value: 'Doe'
_17
});
_17
_17
console.log(response.toString());

Output

_10
<?xml version="1.0" encoding="UTF-8" ?>
_10
<Response>
_10
<Start>
_10
<Stream url="wss://mystream.ngrok.io/example">
_10
<Parameter name="FirstName" value="Jane" />
_10
<Parameter name="LastName" value="Doe" />
_10
</Stream>
_10
</Start>
_10
</Response>


Bi-directional Media Streams

bi-directional-media-streams page anchor

If you want to send media back to the call, the Stream *must* be bi-directional. To do this initialize the stream using the <Connect> TwiML verb as opposed to the <Start> verb. The <Stream> noun's url attribute must be set to a secure websocket server (wss).

Connect call to a bi-directional MediaStream

connect-call-to-a-bi-directional-mediastream page anchor

The <Connect> verb sets up a synchronous bi-directional stream

Node.js
Python
C#
Java
PHP
Ruby

_10
const VoiceResponse = require('twilio').twiml.VoiceResponse;
_10
_10
const response = new VoiceResponse();
_10
const connect = response.connect();
_10
connect.stream({
_10
url: 'wss://mystream.ngrok.io/audiostream'
_10
});
_10
_10
console.log(response.toString());

Output

_10
<?xml version="1.0" encoding="UTF-8"?>
_10
<Response>
_10
<Connect>
_10
<Stream url="wss://mystream.ngrok.io/audiostream" />
_10
</Connect>
_10
</Response>


Media Streams must communicate with Twilio's cloud in order to function. Listed below are the IP address ranges used to communicate with Twilio's cloud. If necessary, use this information to configure your firewall to enable communication with Twilio.

(warning)
EdgeLocationServer IP Address Range
ashburnUS East Coast (Virginia)34.203.254.0/24 3.235.111.128/25

Traffic is secure web socket over TCP


Rate this page: