How to Connect Your Twilio Agent to External APIs in Node.js

Time to read:

September 14, 2026

Written by

Twilion

How to Connect Your Twilio Agent to External APIs in Node.js

The world is starting to increasingly rely on voice-enabled AI agents to get work done. But an agent can only do so much. Voice AI agents by themselves are able to hold conversations, but what happens if the agent needs to do something like retrieve customer data, look at an inventory, or book an appointment for a user?

In order to make your AI agent truly helpful, you need for that AI agent to have access to real time information. External APIs can provide that information. When your agent works together with an API, your agent is empowered to get information your users really need, and take actions on the user's behalf like viewing inventory, calendars, menus, and more.

In this tutorial, you will use Node.js to build a voice agent using Twilio Conversation Relay. Your agent will use tool calling from an LLM-driven conversation to dynamically fetch live data from an external REST API. This tutorial uses a simple API with no additional authentication requirements to showcase the potential of the AI tool. When you have completed the tutorial, you should understand the pipeline to interact with an external API, and how you could employ this functionality in your own builds.

This tutorial is geared towards Node.js developers, but you can find other languages below:

C# .NET
Python
PHP

Prerequisites

To complete this tutorial you will need the following:

A free Twilio account with a voice-capable phone number
Node.js version 18 or higher installed on your machine
An OpenAI API key
ngrok to expose local webhooks to Twilio
An IDE or text editor such as Visual Studio Code

Building the application

Step 1 - Set up the Node.js project

Your first step is creating a new folder and a Node.js project. Go into your terminal and type the following:

mkdir twilio-agent-api
cd twilio-agent-api
npm init -y

The npm init -y command creates a package.json file with default values, which will keep track of your dependencies.

Step 2 - Install dependencies

Install the npm packages you will need for your project by typing the following into your terminal:

npm install express express-ws dotenv openai

These packages are necessary for your project setup: The express package is a minimal web framework used to handle HTTP requests. The express-ws package adds WebSocket support to Express so your app can communicate with Twilio Conversation Relay. The dotenv package allows you to import your environment variables into your solution using a .env file. The openai package will be used to connect your solution to OpenAI.

Step 3 - Configure environment variables

This tutorial is simple enough not to require much information from your Twilio account. But you will need somewhere to safely store your OpenAI API key. Create a file called .env in your project folder. Add to that file the following text:

OPENAI_API_KEY=XXXXXXXXXXXX

Your OpenAI API Key is generated from OpenAI's dashboard. Replace XXXXXXXXXXXX with your actual key. You shouldn't need any other keys in this file. However, if you decide later to call an API that has additional authentication, that key can be stored here as well.

Step 4 - Build the base application

You will use the very simple API, Cat Facts, in this demo. Your application is going to make a simple API call to request a "Cat Fact" from your agent. This API requires no additional authentication and has simple output, which makes it very useful for a demonstration.

Create a new file called index.js in your project folder. Open this file in your IDE of choice, and add the following code:

require('dotenv').config();
const express = require('express');
const expressWs = require('express-ws');
const { handleConversationRelay } = require('./conversationRelayHandler');
const app = express();
expressWs(app);
const PORT = process.env.PORT || 5000;
app.post('/voice', (req, res) => {
  const host = req.headers.host;
  const twiml = `<?xml version="1.0" encoding="UTF-8"?>
<Response>
    <Connect>
        <ConversationRelay url="wss://${host}/ws" welcomeGreeting="Hello! Ask me for a cat fact." />
    </Connect>
</Response>`;
  res.type('application/xml').send(twiml);
});
app.ws('/ws', async (ws, req) => {
  await handleConversationRelay(ws);
});
app.listen(PORT, () => {
  console.log(`Server is running on port ${PORT}`);
});

This code is making a connection to a websocket to enable your agent. You are using Conversation Relay to build the connection between OpenAI and your voice-capable Twilio number, creating a voice agent that can hold a natural sounding conversation. Notice that you have added a simple greeting for your agent using TwiML. This greeting line can be adjusted as needed to give the user an initial prompt for interaction.

Step 5 - Handle Conversation Relay

You will need some additional code to connect your websocket to Conversation Relay. Do this by creating a new file, called conversationRelayHandler.js, in your project folder. Add the following code:

const { streamResponse } = require('./openAiService');
async function handleConversationRelay(ws) {
  let callSid = null;
  const messages = [];
  ws.on('message', async (data) => {
    let msg;
    try {
      msg = JSON.parse(data.toString());
    } catch (err) {
      console.error('Failed to parse message:', err);
      return;
    }
    switch (msg.type) {
      case 'setup':
        callSid = msg.callSid;
        console.log(`[${callSid}] Call connected`);
        break;
      case 'prompt':
        if (!msg.last) break;
        const userText = msg.voicePrompt || '';
        console.log(`[${callSid}] Caller: ${userText}`);
        messages.push({ role: 'user', content: userText });
        const responseText = await streamResponse(ws, callSid, messages);
        messages.push({ role: 'assistant', content: responseText });
        break;
      case 'interrupt':
        const spoken = msg.utteranceUntilInterrupt || '';
        console.log(`[${callSid}] Interrupted after: '${spoken}'`);
        if (messages.length > 0 && messages[messages.length - 1].role === 'assistant') {
          messages.pop();
        }
        break;
      case 'error':
        console.log(`[${callSid}] Conversation Relay error: ${msg.description || ''}`);
        break;
    }
  });
  ws.on('close', () => {
    console.log(`[${callSid}] Call ended`);
  });
  ws.on('error', (err) => {
    console.log(`[${callSid}] WebSocket error: ${err.message}`);
  });
}
function sendJson(ws, payload) {
  ws.send(JSON.stringify(payload));
}
module.exports = { handleConversationRelay, sendJson };

This code is communicating with your websocket, breaking your voice inquiries down into conversation messages to be processed by the AI. Changing your voice responses to text, it then streams that text in real time to the AI in order to get fast responses.

This is one important component, but you still need to make the connection to OpenAI. You will do that in the next step.

Step 6 - Connect to OpenAI

In this step, you will configure tool function schemas using the OpenAI Node.js SDK. For this, create a file called openAiService.js.

Write the system prompt instructing the agent when to execute external API calls based on user voice prompts. You'll see the prompt inside the SYSTEM_PROMPT constant in the code below. You can adjust this to your needs. In this prompt, you make sure that the AI realizes it's being used for voice interaction, by reminding it not to use any bullet points or emojis when it communicates.

const OpenAI = require('openai');
const { sendJson } = require('./conversationRelayHandler');
const MODEL = 'gpt-4o-mini';
const SYSTEM_PROMPT = `You are a cat fact generator. If your user asks you for a cat fact you will respond with a cat fact.
Speak naturally as if talking on the phone. Use plain sentences only. Do not use lists or bullet points. Do not use any emojis.
When you are asked for a cat fact you must call the get_cat_fact tool to retrieve one from the API.
Do not just make up facts. If you do not know the answer, respond with "I don't know."`;
const catFactTool = {
  type: 'function',
  function: {
    name: 'get_cat_fact',
    description: 'Retrieves a random cat fact from the catfact.ninja API.',
    parameters: {
      type: 'object',
      properties: {},
      required: [],
    },
  },
};
const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
async function streamResponse(ws, callSid, messages) {
  const openAiMessages = [
    { role: 'system', content: SYSTEM_PROMPT },
    ...messages,
  ];
  let fullResponse = '';
  try {
    const stream = await client.chat.completions.create({
      model: MODEL,
      messages: openAiMessages,
      tools: [catFactTool],
      max_tokens: 300,
      stream: true,
    });
    const toolCallsAcc = {};
    let finishReason = null;
    for await (const chunk of stream) {
      const choice = chunk.choices[0];
      if (choice.finish_reason) finishReason = choice.finish_reason;
      const delta = choice.delta;
      if (delta.content) {
        fullResponse += delta.content;
        sendJson(ws, { type: 'text', token: delta.content, last: false });
      }
      if (delta.tool_calls) {
        for (const tc of delta.tool_calls) {
          if (!toolCallsAcc[tc.index]) {
            toolCallsAcc[tc.index] = { id: '', name: '', arguments: '' };
          }
          const acc = toolCallsAcc[tc.index];
          if (tc.id) acc.id = tc.id;
          if (tc.function?.name) acc.name = tc.function.name;
          if (tc.function?.arguments) acc.arguments += tc.function.arguments;
        }
      }
    }
    if (finishReason === 'tool_calls' && Object.keys(toolCallsAcc).length > 0) {
      const toolCalls = Object.values(toolCallsAcc);
      openAiMessages.push({
        role: 'assistant',
        tool_calls: toolCalls.map((tc) => ({
          id: tc.id,
          type: 'function',
          function: {
            name: tc.name,
            arguments: tc.arguments || '{}',
          },
        })),
      });
      for (const tc of toolCalls) {
        const result = await executeTool(tc.name);
        console.log(`[${callSid}] Tool ${tc.name}(${tc.arguments}) -> ${result}`);
        openAiMessages.push({
          role: 'tool',
          tool_call_id: tc.id,
          content: result,
        });
      }
      fullResponse = '';
      const stream2 = await client.chat.completions.create({
        model: MODEL,
        messages: openAiMessages,
        max_tokens: 400,
        stream: true,
      });
      for await (const chunk of stream2) {
        const content = chunk.choices[0]?.delta?.content;
        if (content) {
          fullResponse += content;
          sendJson(ws, { type: 'text', token: content, last: false });
        }
      }
    }
  } finally {
    sendJson(ws, { type: 'text', token: '', last: true });
  }
  console.log(`[${callSid}] Assistant: ${fullResponse}`);
  return fullResponse;
}
async function executeTool(name) {
  if (name !== 'get_cat_fact') return 'Unknown tool.';
  try {
    const response = await fetch('https://catfact.ninja/fact');
    const data = await response.json();
    return data.fact || 'No fact returned.';
  } catch (err) {
    return `Error retrieving cat fact: ${err.message}`;
  }
}
module.exports = { streamResponse };

The executeTool function is what actually calls your external API. It reaches out to the API located at https://catfact.ninja and parses the JSON response from the API. If it can't find a fact, say, if the connection to the API is interrupted, it returns an error.

Testing your application

Now it is time to test your application and chat with your AI.

First, run your application using this command in the terminal:

node index.js

You should see output indicating your server is running on port 5000.

Once your webhook is running, you will need to expose it to the internet by using ngrok or another tunneling service. Open a second terminal window and type the following:

ngrok http 5000

Replace 5000 with whatever port your application is running on if you have a different port.

Now ngrok will provide you with a URL for utilizing in your Twilio Console. Go into your Twilio Console and find the Twilio phone number that you prepared. Under the option A Call Comes In, choose Webhook, and fill in your ngrok URL followed by /voice, as shown in the graphic below:

configuration for webhooks in the Twilio console

Be sure also that your HTTP block is set to POST.

Now save this configuration, and call your Twilio Phone Number.

You should hear a message with the AI greeting that you provided in index.js.

Try asking your AI about a cat fact and you will get a cat fact from the cat fact API!

Troubleshooting

If you are having some difficulty with your call, there are some common problems you might want to check. First of all, make sure your ngrok URL is correct in the console and matches the one that's in your terminal, with /voice appended to the end.

If you are still having issues, check your environment variables. You will need to make sure your API keys are correct for any key that you happen to be using, including your key for OpenAI. The sample API requires no additional keys, but if you decide to expand the application, you will also need to authenticate any external APIs that you call. Check the rules for your individual APIs.

If you see an error about fetch being undefined, make sure you are running Node.js version 18 or higher, as fetch is built in starting with that version.

Conclusion

Connecting LLM function tools to external HTTP endpoints empowers Twilio voice agents with real-time data. With the use of external APIs, you can create an agent that doesn't just respond to questions, but truly does the work your customers need.

Are you looking for some further project ideas or further reading? Twilio also has a series on getting started creating your AI Phone Agent with Conversation Relay. Or check out how to do function and tool calling in Node.js or your language of choice!

Twilio can't wait to see what you build!

Dhruv Patel is a Developer on Twilio's Developer Voices team. You can find Dhruv working in a coffee shop with a glass of cold brew or he can be reached at dhrpatel [at] twilio.com.

Related Resources

Twilio Docs

From APIs to SDKs to sample apps

API reference documentation, SDKs, helper libraries, quickstarts, and tutorials for your language and platform.

Resource Center

The latest ebooks, industry reports, and webinars

Learn from customer engagement experts to improve your own communication.

Ahoy

Twilio's developer community hub

Best practices, code samples, and inspiration to build communications and digital engagement experiences.

How to Connect Your Twilio Agent to External APIs in Node.js

Related Posts

Related Resources