Provider failover for Conversation Relay
Learn how to configure provider failover so a Conversation Relay call continues when its text-to-speech (TTS) or speech-to-text (STT) provider fails. When the provider that a call uses fails, Conversation Relay switches to a backup provider. The caller stays on the line, and your application keeps the same WebSocket connection.
For details about the <ConversationRelay> attributes and WebSocket messages that this guide uses, see Related reference documentation.
Provider failover works as follows:
- TTS and STT fail over independently: If only the TTS provider fails, STT continues on its current provider. If only the STT provider fails, TTS continues on its current provider.
- Failover works at setup and during a call: Conversation Relay handles a provider that fails to connect when the call starts in the same way as a provider that fails partway through the call.
- Specify your backups: Name the backup provider, voice, and model for each language so the behavior is predictable. If you omit the TTS or STT backup for a language, Conversation Relay uses Twilio's default backup for that language and primary provider.
- Pending text is still spoken: The backup voice speaks any text from your agent that hasn't been spoken yet, in the order the agent sent it.
- You can monitor failover: Subscribe to the
config-updatedevent to see which provider the call uses.
To use failover, add <Failover> to your <ConversationRelay> TwiML. After that, failover is automatic. Your application doesn't need to detect or react to the outage.
Add one <Failover> element inside <ConversationRelay>, next to the <Language> elements. The following sections show the ways to write it.
An empty <Failover/> element applies to every language in the session. That includes the language of <ConversationRelay> and each <Language> element. Conversation Relay uses Twilio's default backups.
1<Response>2<Connect>3<ConversationRelay url="wss://example.com/socket" language="en-US" events="config-updated">4<Language code="es-ES" ttsProvider="elevenlabs" transcriptionProvider="deepgram" />5<Failover />6</ConversationRelay>7</Connect>8</Response>
If Twilio has no default backup for any of these languages, Conversation Relay rejects the call with error 64101. To avoid this error, list only the languages that you want to protect.
Add a nested <Language> element for each language that you want to protect. Languages that you omit have no failover.
1<Response>2<Connect>3<ConversationRelay url="wss://example.com/socket" language="en-US" events="config-updated">4<Failover>5<Language code="en-US" />6</Failover>7</ConversationRelay>8</Connect>9</Response>
This example uses Twilio's default backups for both TTS and STT for en-US only. Other languages in the session have no failover.
Specify the backup TTS provider, the backup STT provider, or both. You can also specify the voice or model. Conversation Relay uses Twilio's default backup for anything that you omit. Twilio recommends specifying backups explicitly.
1<Response>2<Connect>3<ConversationRelay url="wss://example.com/socket" language="en-US" events="config-updated">4<Language code="es-ES" ttsProvider="elevenlabs" transcriptionProvider="deepgram" />5<Failover>6<Language code="en-US" ttsProvider="google" voice="en-US-Neural2-J" transcriptionProvider="google" speechModel="telephony" />7<Language code="es-ES" transcriptionProvider="google" speechModel="telephony" />8</Failover>9</ConversationRelay>10</Connect>11</Response>
The en-US entry specifies both TTS and STT. The es-ES entry specifies only STT, so its TTS uses Twilio's default backup.
A <Language> element inside <Failover> supports the code, ttsProvider, voice, transcriptionProvider, and speechModel attributes. Conversation Relay ignores other attributes, such as hints. For the full attribute descriptions, see <Failover> element.
Keep these rules in mind:
- The
codemust be a full language code, such asen-US(noten), and must match a language in the session. - If you specify a provider without a voice or model, Conversation Relay uses that provider's default voice or model, as it does for a regular
<Language>element. To avoid unexpected defaults, specify the voice or model.
Conversation Relay uses a backup that you specify in <Failover> exactly as you write it. If you omit the TTS or STT backup, Conversation Relay uses Twilio's default backup. The default depends on the language and the primary provider, and it always differs from the provider that failed.
For example, the default backups for en-US are:
| Type | Primary provider | Backup provider | Backup voice or model |
|---|---|---|---|
| TTS | ElevenLabs or Amazon | en-US-Neural2-J | |
| TTS | ElevenLabs | UgBBYS2sOqTuMpoF3BR0 (Mark) | |
| STT | Deepgram | nova-3-general | |
| STT | Deepgram | telephony |
A backup that you specify must meet these rules:
- The backup provider must differ from the primary provider. Different ElevenLabs engines count as different providers, and so do different Deepgram models, such as
nova-2andnova-3. - A TTS backup must be Google, Amazon, or ElevenLabs.
- An STT backup must be Google or Deepgram.
- The voice and model must follow the same rules as a regular
<Language>element.
- TTS: The provider returns a server error (HTTP 5xx) or ends the stream unexpectedly. ElevenLabs rate-limiting (HTTP 429) also counts. A 429 from Google or Amazon doesn't trigger failover.
- STT: The provider fails to start, returns an error during the call, or drops its connection and can't reconnect.
Configuration errors, such as an invalid voice or a model that the language doesn't support, don't trigger failover. Conversation Relay reports these errors as it does without failover, and the call continues as described in What happens when there is no backup.
- TTS: The backup voice speaks any remaining text in the order it was sent. Conversation Relay doesn't repeat a sentence that the caller was already hearing when the provider failed, and it skips the rest of that sentence.
- STT: Speech recognition restarts on the backup provider immediately, within the same call. Conversation Relay might miss speech that occurs during the switch.
Conversation Relay treats the failed provider as unavailable for the rest of the call and doesn't switch back automatically. Conversation Relay still attempts a configuration update from your application that names that provider.
A successful failover doesn't send an error. Instead, Conversation Relay sends the config-updated event, the same event that a configuration update produces. To receive it, subscribe in your TwiML:
1<Response>2<Connect>3<ConversationRelay url="wss://example.com/socket" events="config-updated">4<Failover />5</ConversationRelay>6</Connect>7</Response>
Conversation Relay then sends an info message that identifies the new provider and its voice or model. Conversation Relay doesn't send a separate failover message. The following example shows the message:
1{2"type": "info",3"name": "configUpdated",4"value": {5"language": [6{7"code": "en-US",8"ttsProvider": "google",9"voice": "en-US-Neural2-J"10}11]12}13}
If the language has no <Failover> entry, or if the backup also fails, the call behaves as it does without failover:
- TTS fails after the agent has spoken: Your application receives an
errormessage (64111), Conversation Relay skips those tokens, and the call continues. - TTS fails before the agent has spoken: Your application receives an
errormessage (64111 or 64112) and the call ends. - Speech recognition fails: Your application receives an
errormessage (64114) and the call ends.
You set failover once, in TwiML, for the entire session. Keep these limits in mind:
- Configuration updates can't change failover settings: A language that you add or switch to during a call has no failover unless your TwiML named that language in
<Failover>or included an empty<Failover />. - Failover happens once for TTS and once for STT: If the backup provider also fails, the call ends. Conversation Relay doesn't fail over a second time, and it doesn't switch back to the original provider after that provider recovers. The exception is a TTS failure after the agent has spoken. In that case, Conversation Relay skips those tokens, sends your application an error, and the call continues.
- Some languages and providers have no default backup: For example, Twilio has no default backup for
multi, for TTS and STT inca-ES, for STT inml-IN, or for TTS inkn-IN,ml-IN,mr-IN,te-IN, andth-TH. If you request failover for one of these languages without specifying a backup, Conversation Relay rejects the call. A plain<Failover />is also rejected if any session language has no default backup. - Welcome greeting delay: When
<Failover>is present, thewelcomeGreetingwaits until speech recognition is ready, so it might start slightly later.
If your <Failover> configuration is invalid, Conversation Relay rejects the call with error 64101 at setup. Twilio sends the error to the action URL of your <Connect> element, and the error message identifies the rule that failed.
| Error message | Cause |
|---|---|
Only one <Failover> TwiML element is allowed | The TwiML has more than one <Failover> element. |
Invalid code or Duplicate code | A code is missing, is short (en), or is repeated. |
is not a session language | The session doesn't declare the language. |
is not supported for | Twilio has no default backup for that language and provider, and you didn't specify one. |
must differ from the primary | The backup is the same provider as the primary. |
cannot be a fallback | You specified a TTS provider that can't be a backup. |
Use failover to prevent a provider outage from ending live calls. Specify a backup for each language that is critical to your application, and subscribe to config-updated so your application can log when a call switches providers.
Specify the backup for a language when the default doesn't fit your brand or use case. For example, choose a Google voice that sounds similar to your primary ElevenLabs voice, or choose a speech model that recognizes your domain-specific terms.
After you complete this guide, your Conversation Relay calls switch to a backup STT or TTS provider when the provider in use fails. To confirm that failover works, verify that your application receives a configUpdated info message that identifies the backup provider.
To build on this guide, explore the following pages:
- Update configuration during a call: Change language, voice, and session settings while a call is in progress.
- Pick a voice: Choose the TTS provider and voice to use for each language.
- Conversation Relay best practices: Learn how to handle connection drops, language switching, and false interruptions.