Integrate Claude, Anthropic’s AI Assistant, with Twilio Voice Using ConversationRelay

Time to read: 4 minutes

May 13, 2025
Written by
Paul Kamp
Twilion
Reviewed by
Emily Shenfield
Twilion
Kelley Robinson
Twilion
Amanda Lange
Twilion
Brandon Hawkins
Twilion

Integrate Claude, Anthropic’s AI Assistant, with Twilio Voice Using ConversationRelay

Twilio’s ConversationRelay allows you to build real-time, human-like voice applications with AI Large Language Models (LLMs) that you can call using Twilio Voice. ConversationRelay connects to a WebSocket you control, allowing you to integrate the LLM provider of your choice over a fast two-way connection.

This tutorial demonstrates a basic integration of Claude, Anthropic's AI model, with Twilio Voice using ConversationRelay. By the end of this guide, you'll have a functioning Claude voice assistant powered by a Node.js server that lets you call a Twilio phone number and engage in conversation with an Anthropic model!

Prerequisites

To deploy this tutorial, you'll need:

  • Node.js 23+ (I used 23.9.0 to build this tutorial)
  • A Twilio account (you can sign up for free, here) and a phone number with voice capabilities
  • An Anthropic account and API Key
  • Ngrok or another tunneling solution to expose your local server to the internet for testing
  • A phone to place your outgoing call to Twilio
  • Your code editor or IDE of choice

Great! Let’s get started.

Set up your project

These next steps will cover project setup and the dependencies you’ll need to install. Then, you'll create the server code (and I’ll explain the more interesting parts).

Create the project directory

First, set up a new project folder:

mkdir conversation-relay-anthropic-node
cd conversation-relay-anthropic-node

Initialize the Node.js Project

Next, initialize a new Node.js project and install the necessary packages: 

npm init -y
npm pkg set type="module"
npm install fastify @fastify/websocket @fastify/formbody @anthropic-ai/sdk dotenv

You'll be using Fastify to create a server that handles both the WebSocket needed for ConversationRelay and the endpoint for Twilio instructions required by Twilio Voice. You’ll use Anthropic’s Typescript library to manage the conversation with Claude.

If you don’t want to walk through all the build steps, you can find the complete code in our repository here.

Configure Environment Variables

Create an .env file to store your Anthropic API key. Add the following line:

ANTHROPIC_API_KEY="YOUR_ANTHROPIC_API_KEY"

If you are working with git, add .env to a .gitignore file to prevent exposing sensitive information.

Write the code

Awesome, you’re really cooking now. Let’s move on to the server code.

Create a file named server.js and add the following imports and constants: 

import fastifyWs from "@fastify/websocket";
import fastifyFormBody from '@fastify/formbody';
import Anthropic from "@anthropic-ai/sdk";
import dotenv from "dotenv";
dotenv.config();
const PORT = process.env.PORT || 8080;
const DOMAIN = process.env.NGROK_URL;
const WS_URL = `wss://${DOMAIN}/ws`;
const WELCOME_GREETING = "Hi! I am an A I voice assistant powered by Twilio and Anthropic. Ask me anything!";
const SYSTEM_PROMPT = "You are a helpful assistant. This conversation is being translated to voice, so answer carefully. When you respond, please spell out all numbers, for example twenty not 20. Do not include emojis in your responses. Do not include bullet points, asterisks, or special symbols.";
const sessions = new Map();

Here, you’ll notice we define our WELCOME_GREETING. This is the first thing someone will hear when they dial into your assistant.

You’ll also notice a SYSTEM_PROMPT, which controls the behavior of the voice assistant. There, we let the LLM know some of our preferences for the conversation. This prompt asks the LLM to avoid special characters and asks it to write out numbers to make it easier for the Text-to-Speech step to work.

When you go to production, you'll want to iterate and test your prompt quite a bit. Find our Prompt Engineering best practices here to see what we've learned about working with LLMs and voice.

Implement the AI response function

In this next step, you will define how your app connects to the Anthropic API and handle responses. Paste this code below the above: 

const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });
async function aiResponse(messages) {
  let completion = await anthropic.messages.create({model: "claude-3-7-sonnet-20250219", max_tokens: 1024, messages: messages, system: SYSTEM_PROMPT });
  return completion.content[0].text;
}

Here, you add what we need to connect to Anthropic. process.env.ANTHROPIC_API_KEY will get our API key from the .env file to authenticate our usage of Anthropic, then the aiResponse function is what is used to control the behavior of each LLM response.

For this demo, you will use ​​claude-3-7-sonnet-20250219 to test the build. You can find Anthropic’s supported Claude models in their documentation.

Anthropic's Docs feature a model comparison table to help you decide which model best suits your needs. Sonnet balances cost, latency, and intelligence for this test application, and we also successfully tested with Haiku.

Configure Fastify, and set up routes

Now that you've defined your interactions with Claude, it's time to move on to Twilio. You're going to set up two endpoints in this step:

  • /twiml - to control Twilio’s behavior when we make an incoming call to our purchased number. This uses Twilio Markup Language , or TwiML, to instruct Twilio further.
  • /ws - for a WebSocket connection that ConversationRelay will use to communicate with your application. Here you will receive messages from Twilio, but you will also need to pass messages from your LLM back to Twilio to trigger the Text-to-Speech step (the relay, if you will).

Below the code from the above step, paste the following: 

fastify.register(fastifyFormBody);
fastify.register(fastifyWs);
fastify.all("/twiml", async (request, reply) => {
  reply.type("text/xml").send(`<?xml version="1.0" encoding="UTF-8"?><Response><Connect><ConversationRelay url="${WS_URL}" welcomeGreeting="${WELCOME_GREETING}" /></Connect></Response>`);
});
fastify.register(async function (fastify) {
  fastify.get("/ws", { websocket: true }, (ws, req) => {
    ws.on("message", async (data) => {
      const message = JSON.parse(data);
      switch (message.type) {
        case "setup":
          const callSid = message.callSid;
          console.log("Setup for call:", callSid);
          ws.callSid = callSid;
          sessions.set(callSid, []);
          break;
        case "prompt":
          console.log("Processing prompt:", message.voicePrompt);
          const conversation = sessions.get(ws.callSid);
          conversation.push({ role: "user", content: message.voicePrompt });
          const responseText = await aiResponse(conversation);
          conversation.push({ role: "assistant", content: responseText });
          ws.send(
            JSON.stringify({
              type: "text",
              token: responseText,
              last: true,
            })
          );
          console.log("Sent response:", responseText);
          break;
        case "interrupt":
          console.log("Handling interruption.");
          break;
        default:
          console.warn("Unknown message type received:", message.type);
          break;
      }
    });
    ws.on("close", () => {
      console.log("WebSocket connection closed");
      sessions.delete(ws.callSid);
    });
  });
});
try {
  fastify.listen({ port: PORT });
  console.log(`Server running at http://localhost:${PORT} and wss://${DOMAIN}/ws`);
} catch (err) {
  fastify.log.error(err);
  process.exit(1);
}

And that’s all the code you need for a basic integration – now, let’s get everything running and connect it to Twilio!

Start ngrok and the server

Open a terminal (or a new terminal tab) and start ngrok with ngrok http 8080.

There, copy the ngrok URL without the scheme (that is, leave out the “https://” or “http://”) and update your .env file by adding a new line: 

NGROK_URL="your-ngrok-subdomain.ngrok.app"

This screenshot shows the URL I chose for my environment variable:

Terminal window displaying Ngrok session details with an online status and an orange arrow pointing to a URL.

Finally, start your server. In the original terminal (not the one running ngrok), run: 

node server

Configure Twilio

In the Twilio Console, configure your phone number with the Voice Configuration Webhook for A call comes in set to https://[your-ngrok-subdomain].ngrok.app/twiml. 

Here’s an example from my Console:

Screenshot of voice configuration settings in an online portal with fields for routing and URL configuration.

Save your changes, you’re now ready to test.

Test your integration

Are you giddy with anticipation? It’s time!

Call your Twilio number. The AI should greet you with the WELCOME_MESSAGE and be ready for a conversation – be sure to ask it for owl jokes!

Next steps with Twilio and Claude

I hope you enjoyed this basic integration of Twilio Voice with Anthropic’s Claude using ConversationRelay – dial-a-voice-LLM in under 100 lines of code! 

You’ll probably want to add more advanced features next. Our follow-up article shows you how to stream tokens from Anthropic to ConversationRelay to reduce latency, as well as how to better handle interruptions by tracking your conversation locally. And after that, I’ll show you the basics of how to call tools or functions with Claude using ConversationRelay (coming soon).

But until then, have fun with the scaffolding… check out Twilio’s ConversationRelay docs, Anthropic’s API documentation, and Anthropic’s TypeScript SDK to learn more.

Paul Kamp is the Technical Editor-in-Chief of the Twilio Blog. He was excited to talk to the LLM about recipes… but a bit disappointed he only found time to cook a few suggestions (when will agents make food?). Reach Paul at pkamp [at] twilio.com.

Related Posts

Related Resources

A newspaper article

Twilio Docs

From APIs to SDKs to sample apps

API reference documentation, SDKs, helper libraries, quickstarts, and tutorials for your language and platform.
An Open book

Resource Center

The latest ebooks, industry reports, and webinars

Learn from customer engagement experts to improve your own communication.
User group reactions

Ahoy

Twilio's developer community hub

Best practices, code samples, and inspiration to build communications and digital engagement experiences.