Menu

Expand
Rate this page:

Thanks for rating this page!

We are always striving to improve our documentation quality, and your feedback is valuable to us. How could this documentation serve you better?

TwiML™️ Voice: <Stream> (Beta)

The <Stream> instruction allows you to receive raw audio streams from a live phone call over WebSockets in near real-time.

The most basic use of <Stream>:

<?xml version="1.0" encoding="UTF-8"?>
<Response>
   <Start>
       <Stream url="wss://mystream.ngrok.io/audiostream" />
   </Start>
</Response>

This TwiML will instruct Twilio to fork the audio stream of the current call and send it in real-time over WebSocket to wss://mystream.ngrok.io/audiostream.

<Stream> starts the audio stream asynchronously and immediately continues with the next TwiML instruction. If there is no instruction, the call will be disconnected. In order to avoid this, provide a TwiML instruction to continue the call.

There is a one to one mapping of a stream to a websocket connection, therefore there will be at most one call being streamed over a single websocket connection. Metadata will be provided so that you can handle handle multiple inbound connections and manage the mappings between the unique stream identifier or StreamSid.

If communication issues are encountered with your WebSocket server, they will be reported in the Twilio Debugger with additional information about the failure.

Attributes

<Stream> supports the following attributes:

Attribute Name Allowed Values Default Value
url relative or absolute URL none
name Optional. Unique name for the Stream none
track Optional. inbound_track, outbound_track, both_tracks inbound_track
statusCallback Optional. Relative or absolute URL none
statusCallbackMethod Optional. GET or POST POST

name

Providing a name will allow you to reference the stream directly. This name must be unique per Call.

For instance by naming the Stream my_first_stream.

<Start>
    <Stream name="my_first_stream" url="wss://mystream.ngrok.io/audiostream" />
</Start>

You can later use the unique name of my_first_stream to stop the stream.

<Stop>
   <Stream name="my_first_stream" />
</Stop>

url

The url attribute accepts a relative or absolute url. On successful execution, a WebSocket connection to the url will be established and audio will start streaming.wss is the only supported protocol.

The url does not support query string parameters. To pass custom key value pairs to the WebSocket, make use of Custom Parameters instead.

statusCallback

The statusCallback attribute takes an absolute or relative URL as value. Whenever a stream is started or stopped, Twilio will make a request to this URL with the following parameters:

Parameter Description
AccountSid The unique identifier of the Account responsible for this Stream.
CallSid The unique identifier of the Call
StreamSid The unique identifier for this Stream
StreamName If defined, this is the unique name of the Stream. Defaults to the StreamSid
StreamEvent One of stream-started, stream-stopped, or stream-error (see StreamError for the message)
StreamError If an error has occurred, this will contain a detailed error message.
Timestamp The time of the event in ISO 8601 format

statusCallbackMethod

The HTTP method to use when requesting the statusCallback URL. Default is POST.

Custom Parameters

It is possible to include additional key value pairs that will passed along with your stream. You can do this by using the nested <Parameter> TwiML noun.

<?xml version="1.0" encoding="UTF-8"?>
<Response>
   <Start>
     <Stream url="wss://mystream.ngrok.io/example" >
        <Parameter name="FirstName" value ="Jane"/>
        <Parameter name="LastName" value ="Doe" />
        <Parameter name="RemoteParty" value ="Bob" />
      </Stream>
    </Start>
</Response>

These values will be sent along with the Start WebSocket message.

WebSocket Messages

There are separate types of events that occur during the Stream's life cycle. These events are represented via WebSocket Messages: Connected, Start, Media, Stop. Each message sent is a JSON string. You can determine which type of event is occurring by using the event property of every JSON object.

Connected Message

The first message sent once a WebSocket connection is established is the Connected event. This message describes the protocol to expect in the following messages.

Parameter Description
event The value of connected
protocol Defines the protocol for the WebSocket connections lifetime. eg: "Call"
version Semantic version of the protocol.

Example Connected Message

{ 
 "event": "connected",  
 "protocol": "Call", 
 "version": "0.2.0"
}

Start Message

This message contains important metadata about the stream and is sent immediately after the Connected message. It is only sent once at the start of the Stream.

Parameter Description
event The value of start
sequenceNumber Number used to keep track of message sending order. First message starts with "1" and then is incremented.
start An object containing Stream metadata
start.streamSid The unique identifier of the Stream
start.accountSid The Account identifier that created the Stream
start.callSid The Call identifier from where the Stream was started.
start.tracks An array of values that indicates what media flows to expect in subsequent messages. Values include inbound, outbound.
customParameters An object that represents the Custom Parameters that where set when defining the Stream
mediaFormat An object containing the format of the payload in the Media Messages.
mediaFormat.encoding The encoding of the data in the upcoming payload. Default is audio/x-mulaw.
mediaFormat.sampleRate The Sample Rate in Hertz of the upcoming audio data. Default value is 8000
mediaFormat.channels The number of channels in the input audio data. Default value is 1

Example Start Message

{ 
 "event": "start",  
 "sequenceNumber": "2", 
 "start": { 
   "streamSid": "MZ18ad3ab5a668481ce02b83e7395059f0", 
   "accountSid": "AC123", 
   "callSid": "CA123", 
   "tracks": [ 
     "inbound", 
     "outbound" 
   ],
   "customParameters": {
     "FirstName": "Jane",
     "LastName": "Doe",
     "RemoteParty": "Bob", 
   },
   "mediaFormat": { 
     "encoding": "audio/x-mulaw", 
     "sampleRate": 8000, 
     "channels": 1 
   } 
 }
}

Media Message

This message type encapsulates the raw audio data.

Parameter Description
event The value of media
sequenceNumber Number used to keep track of message sending order. First message starts with "1" and then is incremented for each message.
media An object containing media metadata and payload
media.track One of inbound or outbound
media.chunk The chunk for the message. The first message will begin with "1" and increment with each subsequent message.
media.timestamp Presentation Timestamp in Milliseconds from the start of the stream.
media.payload Raw audio in encoded in base64

Example Media Messages

Outbound
{ 
 "event": "media",
 "sequenceNumber": "3", 
 "media": { 
   "track": "outbound", 
   "chunk": "1", 
   "timestamp": "5",
   "payload": "no+JhoaJjpzSHxAKBgYJDhtEopGKh4aIjZm7JhILBwYIDRg1qZSLh4aIjJevLBUMBwYHDBUsr5eMiIaHi5SpNRgNCAYHCxImu5mNiIaHipGiRBsOCQYGChAf0pyOiYaGiY+e/x4PCQYGCQ4cUp+QioaGiY6bxCIRCgcGCA0ZO6aSi4eGiI2YtSkUCwcGCAwXL6yVjIeGh4yVrC8XDAgGBwsUKbWYjYiGh4uSpjsZDQgGBwoRIsSbjomGhoqQn1IcDgkGBgkPHv+ej4mGhomOnNIfEAoGBgkOG0SikYqHhoiNmbsmEgsHBggNGDWplIuHhoiMl68sFQwHBgcMFSyvl4yIhoeLlKk1GA0IBgcLEia7mY2IhoeKkaJEGw4JBgYKEB/SnI6JhoaJj57/Hg8JBgYJDhxSn5CKhoaJjpvEIhEKBwYIDRk7ppKLh4aIjZi1KRQLBwYIDBcvrJWMh4aHjJWsLxcMCAYHCxQptZiNiIaHi5KmOxkNCAYHChEixJuOiYaGipCfUhwOCQYGCQ8e/56PiYaGiY6c0h8QCgYGCQ4bRKKRioeGiI2ZuyYSCwcGCA0YNamUi4eGiIyXrywVDAcGBwwVLK+XjIiGh4uUqTUYDQgGBwsSJruZjYiGh4qRokQbDgkGBgoQH9KcjomGhomPnv8eDwkGBgkOHFKfkIqGhomOm8QiEQoHBggNGTumkouHhoiNmLUpFAsHBggMFy+slYyHhoeMlawvFwwIBgcLFCm1mI2IhoeLkqY7GQ0IBgcKESLEm46JhoaKkJ9SHA4JBgYJDx7/no+JhoaJjpzSHxAKBgYJDhtEopGKh4aIjZm7JhILBwYIDRg1qZSLh4aIjJevLBUMBwYHDBUsr5eMiIaHi5SpNRgNCAYHCxImu5mNiIaHipGiRBsOCQYGChAf0pyOiYaGiY+e/x4PCQYGCQ4cUp+QioaGiY6bxCIRCgcGCA0ZO6aSi4eGiI2YtSkUCwcGCAwXL6yVjIeGh4yVrC8XDAgGBwsUKbWYjYiGh4uSpjsZDQgGBwoRIsSbjomGhoqQn1IcDgkGBgkPHv+ej4mGhomOnNIfEAoGBgkOG0SikYqHhoiNmbsmEgsHBggNGDWplIuHhoiMl68sFQwHBgcMFSyvl4yIhoeLlKk1GA0IBgcLEia7mY2IhoeKkaJEGw4JBgYKEB/SnI6JhoaJj57/Hg8JBgYJDhxSn5CKhoaJjpvEIhEKBwYIDRk7ppKLh4aIjZi1KRQLBwYIDBcvrJWMh4aHjJWsLxcMCAYHCxQptZiNiIaHi5KmOxkNCAYHChEixJuOiYaGipCfUhwOCQYGCQ8e/56PiYaGiY6c0h8QCgYGCQ4bRKKRioeGiA=="
 } 
}
Inbound
{ 
 "event": "media",
 "sequenceNumber": "4",
 "media": { 
   "track": "inbound", 
   "chunk": "1", 
   "timestamp": "5",
   "payload": "no+JhoaJjpzSHxAKBgYJDhtEopGKh4aIjZm7JhILBwYIDRg1qZSLh4aIjJevLBUMBwYHDBUsr5eMiIaHi5SpNRgNCAYHCxImu5mNiIaHipGiRBsOCQYGChAf0pyOiYaGiY+e/x4PCQYGCQ4cUp+QioaGiY6bxCIRCgcGCA0ZO6aSi4eGiI2YtSkUCwcGCAwXL6yVjIeGh4yVrC8XDAgGBwsUKbWYjYiGh4uSpjsZDQgGBwoRIsSbjomGhoqQn1IcDgkGBgkPHv+ej4mGhomOnNIfEAoGBgkOG0SikYqHhoiNmbsmEgsHBggNGDWplIuHhoiMl68sFQwHBgcMFSyvl4yIhoeLlKk1GA0IBgcLEia7mY2IhoeKkaJEGw4JBgYKEB/SnI6JhoaJj57/Hg8JBgYJDhxSn5CKhoaJjpvEIhEKBwYIDRk7ppKLh4aIjZi1KRQLBwYIDBcvrJWMh4aHjJWsLxcMCAYHCxQptZiNiIaHi5KmOxkNCAYHChEixJuOiYaGipCfUhwOCQYGCQ8e/56PiYaGiY6c0h8QCgYGCQ4bRKKRioeGiI2ZuyYSCwcGCA0YNamUi4eGiIyXrywVDAcGBwwVLK+XjIiGh4uUqTUYDQgGBwsSJruZjYiGh4qRokQbDgkGBgoQH9KcjomGhomPnv8eDwkGBgkOHFKfkIqGhomOm8QiEQoHBggNGTumkouHhoiNmLUpFAsHBggMFy+slYyHhoeMlawvFwwIBgcLFCm1mI2IhoeLkqY7GQ0IBgcKESLEm46JhoaKkJ9SHA4JBgYJDx7/no+JhoaJjpzSHxAKBgYJDhtEopGKh4aIjZm7JhILBwYIDRg1qZSLh4aIjJevLBUMBwYHDBUsr5eMiIaHi5SpNRgNCAYHCxImu5mNiIaHipGiRBsOCQYGChAf0pyOiYaGiY+e/x4PCQYGCQ4cUp+QioaGiY6bxCIRCgcGCA0ZO6aSi4eGiI2YtSkUCwcGCAwXL6yVjIeGh4yVrC8XDAgGBwsUKbWYjYiGh4uSpjsZDQgGBwoRIsSbjomGhoqQn1IcDgkGBgkPHv+ej4mGhomOnNIfEAoGBgkOG0SikYqHhoiNmbsmEgsHBggNGDWplIuHhoiMl68sFQwHBgcMFSyvl4yIhoeLlKk1GA0IBgcLEia7mY2IhoeKkaJEGw4JBgYKEB/SnI6JhoaJj57/Hg8JBgYJDhxSn5CKhoaJjpvEIhEKBwYIDRk7ppKLh4aIjZi1KRQLBwYIDBcvrJWMh4aHjJWsLxcMCAYHCxQptZiNiIaHi5KmOxkNCAYHChEixJuOiYaGipCfUhwOCQYGCQ8e/56PiYaGiY6c0h8QCgYGCQ4bRKKRioeGiA=="                        
 } 
}

Stop Message

A stop message will be sent when the Stream is either <Stop>ped or the Call has ended.

Example Stop Message

{ 
 "event": "stop",
 "sequenceNumber": "5"
}
Parameter Description
event The value of media
sequenceNumber Number used to keep track of message sending order. First message starts with "1" and then is incremented for each message.
Rate this page:

Need some help?

We all do sometimes; code is hard. Get help now from our support team, or lean on the wisdom of the crowd browsing the Twilio tag on Stack Overflow.