Skip to contentSkip to navigationSkip to topbar
Page tools
Useful for sharing or LLM
Accelerate development with AI

On this page
Looking for more inspiration?Visit the

Provider failover for Conversation Relay


Learn how to configure provider failover so a Conversation Relay call continues when its text-to-speech (TTS) or speech-to-text (STT) provider fails. When the provider that a call uses fails, Conversation Relay switches to a backup provider. The caller stays on the line, and your application keeps the same WebSocket connection.

For details about the <ConversationRelay> attributes and WebSocket messages that this guide uses, see Related reference documentation.

All Conversation Relay guides


What provider failover does

what-provider-failover-does page anchor

Provider failover works as follows:

  • TTS and STT fail over independently: If only the TTS provider fails, STT continues on its current provider. If only the STT provider fails, TTS continues on its current provider.
  • Failover works at setup and during a call: Conversation Relay handles a provider that fails to connect when the call starts in the same way as a provider that fails partway through the call.
  • Specify your backups: Name the backup provider, voice, and model for each language so the behavior is predictable. If you omit the TTS or STT backup for a language, Conversation Relay uses Twilio's default backup for that language and primary provider.
  • Pending text is still spoken: The backup voice speaks any text from your agent that hasn't been spoken yet, in the order the agent sent it.
  • You can monitor failover: Subscribe to the config-updated event to see which provider the call uses.

To use failover, add <Failover> to your <ConversationRelay> TwiML. After that, failover is automatic. Your application doesn't need to detect or react to the outage.


Add failover to your TwiML

add-failover-to-your-twiml page anchor

Add one <Failover> element inside <ConversationRelay>, next to the <Language> elements. The following sections show the ways to write it.

Use default backups for every language

use-default-backups-for-every-language page anchor

An empty <Failover/> element applies to every language in the session. That includes the language of <ConversationRelay> and each <Language> element. Conversation Relay uses Twilio's default backups.

1
<Response>
2
<Connect>
3
<ConversationRelay url="wss://example.com/socket" language="en-US" events="config-updated">
4
<Language code="es-ES" ttsProvider="elevenlabs" transcriptionProvider="deepgram" />
5
<Failover />
6
</ConversationRelay>
7
</Connect>
8
</Response>

If Twilio has no default backup for any of these languages, Conversation Relay rejects the call with error 64101. To avoid this error, list only the languages that you want to protect.

Use failover for specific languages

use-failover-for-specific-languages page anchor

Add a nested <Language> element for each language that you want to protect. Languages that you omit have no failover.

1
<Response>
2
<Connect>
3
<ConversationRelay url="wss://example.com/socket" language="en-US" events="config-updated">
4
<Failover>
5
<Language code="en-US" />
6
</Failover>
7
</ConversationRelay>
8
</Connect>
9
</Response>

This example uses Twilio's default backups for both TTS and STT for en-US only. Other languages in the session have no failover.

Specify backups explicitly

specify-backups-explicitly page anchor

Specify the backup TTS provider, the backup STT provider, or both. You can also specify the voice or model. Conversation Relay uses Twilio's default backup for anything that you omit. Twilio recommends specifying backups explicitly.

1
<Response>
2
<Connect>
3
<ConversationRelay url="wss://example.com/socket" language="en-US" events="config-updated">
4
<Language code="es-ES" ttsProvider="elevenlabs" transcriptionProvider="deepgram" />
5
<Failover>
6
<Language code="en-US" ttsProvider="google" voice="en-US-Neural2-J" transcriptionProvider="google" speechModel="telephony" />
7
<Language code="es-ES" transcriptionProvider="google" speechModel="telephony" />
8
</Failover>
9
</ConversationRelay>
10
</Connect>
11
</Response>

The en-US entry specifies both TTS and STT. The es-ES entry specifies only STT, so its TTS uses Twilio's default backup.

A <Language> element inside <Failover> supports the code, ttsProvider, voice, transcriptionProvider, and speechModel attributes. Conversation Relay ignores other attributes, such as hints. For the full attribute descriptions, see <Failover> element.

Keep these rules in mind:

  • The code must be a full language code, such as en-US (not en), and must match a language in the session.
  • If you specify a provider without a voice or model, Conversation Relay uses that provider's default voice or model, as it does for a regular <Language> element. To avoid unexpected defaults, specify the voice or model.

Conversation Relay uses a backup that you specify in <Failover> exactly as you write it. If you omit the TTS or STT backup, Conversation Relay uses Twilio's default backup. The default depends on the language and the primary provider, and it always differs from the provider that failed.

For example, the default backups for en-US are:

TypePrimary providerBackup providerBackup voice or model
TTSElevenLabs or AmazonGoogleen-US-Neural2-J
TTSGoogleElevenLabsUgBBYS2sOqTuMpoF3BR0 (Mark)
STTGoogleDeepgramnova-3-general
STTDeepgramGoogletelephony

A backup that you specify must meet these rules:

  • The backup provider must differ from the primary provider. Different ElevenLabs engines count as different providers, and so do different Deepgram models, such as nova-2 and nova-3.
  • A TTS backup must be Google, Amazon, or ElevenLabs.
  • An STT backup must be Google or Deepgram.
  • The voice and model must follow the same rules as a regular <Language> element.

When Conversation Relay fails over

when-conversation-relay-fails-over page anchor

What counts as a provider failure

what-counts-as-a-provider-failure page anchor
  • TTS: The provider returns a server error (HTTP 5xx) or ends the stream unexpectedly. ElevenLabs rate-limiting (HTTP 429) also counts. A 429 from Google or Amazon doesn't trigger failover.
  • STT: The provider fails to start, returns an error during the call, or drops its connection and can't reconnect.

What doesn't count as a provider failure

what-doesnt-count-as-a-provider-failure page anchor

Configuration errors, such as an invalid voice or a model that the language doesn't support, don't trigger failover. Conversation Relay reports these errors as it does without failover, and the call continues as described in What happens when there is no backup.

  • TTS: The backup voice speaks any remaining text in the order it was sent. Conversation Relay doesn't repeat a sentence that the caller was already hearing when the provider failed, and it skips the rest of that sentence.
  • STT: Speech recognition restarts on the backup provider immediately, within the same call. Conversation Relay might miss speech that occurs during the switch.

What happens for the rest of the call

what-happens-for-the-rest-of-the-call page anchor

Conversation Relay treats the failed provider as unavailable for the rest of the call and doesn't switch back automatically. Conversation Relay still attempts a configuration update from your application that names that provider.


What your application receives

what-your-application-receives page anchor

A successful failover doesn't send an error. Instead, Conversation Relay sends the config-updated event, the same event that a configuration update produces. To receive it, subscribe in your TwiML:

1
<Response>
2
<Connect>
3
<ConversationRelay url="wss://example.com/socket" events="config-updated">
4
<Failover />
5
</ConversationRelay>
6
</Connect>
7
</Response>

Conversation Relay then sends an info message that identifies the new provider and its voice or model. Conversation Relay doesn't send a separate failover message. The following example shows the message:

1
{
2
"type": "info",
3
"name": "configUpdated",
4
"value": {
5
"language": [
6
{
7
"code": "en-US",
8
"ttsProvider": "google",
9
"voice": "en-US-Neural2-J"
10
}
11
]
12
}
13
}

What happens when there is no backup

what-happens-when-there-is-no-backup page anchor

If the language has no <Failover> entry, or if the backup also fails, the call behaves as it does without failover:

  • TTS fails after the agent has spoken: Your application receives an error message (64111), Conversation Relay skips those tokens, and the call continues.
  • TTS fails before the agent has spoken: Your application receives an error message (64111 or 64112) and the call ends.
  • Speech recognition fails: Your application receives an error message (64114) and the call ends.

You set failover once, in TwiML, for the entire session. Keep these limits in mind:

  • Configuration updates can't change failover settings: A language that you add or switch to during a call has no failover unless your TwiML named that language in <Failover> or included an empty <Failover />.
  • Failover happens once for TTS and once for STT: If the backup provider also fails, the call ends. Conversation Relay doesn't fail over a second time, and it doesn't switch back to the original provider after that provider recovers. The exception is a TTS failure after the agent has spoken. In that case, Conversation Relay skips those tokens, sends your application an error, and the call continues.
  • Some languages and providers have no default backup: For example, Twilio has no default backup for multi, for TTS and STT in ca-ES, for STT in ml-IN, or for TTS in kn-IN, ml-IN, mr-IN, te-IN, and th-TH. If you request failover for one of these languages without specifying a backup, Conversation Relay rejects the call. A plain <Failover /> is also rejected if any session language has no default backup.
  • Welcome greeting delay: When <Failover> is present, the welcomeGreeting waits until speech recognition is ready, so it might start slightly later.

If your <Failover> configuration is invalid, Conversation Relay rejects the call with error 64101 at setup. Twilio sends the error to the action URL of your <Connect> element, and the error message identifies the rule that failed.

Error messageCause
Only one <Failover> TwiML element is allowedThe TwiML has more than one <Failover> element.
Invalid code or Duplicate codeA code is missing, is short (en), or is repeated.
is not a session languageThe session doesn't declare the language.
is not supported forTwilio has no default backup for that language and provider, and you didn't specify one.
must differ from the primaryThe backup is the same provider as the primary.
cannot be a fallbackYou specified a TTS provider that can't be a backup.

Use cases for provider failover

use-cases-for-provider-failover page anchor

Keep production AI agents available

keep-production-ai-agents-available page anchor

Use failover to prevent a provider outage from ending live calls. Specify a backup for each language that is critical to your application, and subscribe to config-updated so your application can log when a call switches providers.

Specify a custom backup voice and model

specify-a-custom-backup-voice-and-model page anchor

Specify the backup for a language when the default doesn't fit your brand or use case. For example, choose a Google voice that sounds similar to your primary ElevenLabs voice, or choose a speech model that recognizes your domain-specific terms.


After you complete this guide, your Conversation Relay calls switch to a backup STT or TTS provider when the provider in use fails. To confirm that failover works, verify that your application receives a configUpdated info message that identifies the backup provider.


To build on this guide, explore the following pages: