How to Orchestrate Multi-Call Conversations with an LLM and Twilio Conversation Memory with PHP
Time to read:
Have you ever been on the phone with an AI voice agent and gotten frustrated with its lack of memory? Maybe your agent hung up on you, or it got disconnected, forcing you to start a conversation all over again. This kind of interruption can waste time for you and your users, and cause a lot of frustration.
Twilio Conversation Memory is your solution. Conversation Memory allows context to be persisted between calls. This means that if you call the Twilio agent back, it won't lose the context of what you were talking about when you hung up, and can pick up right where you left off. This can save you a lot of frustration and help you get things done better and faster when you're talking to an agent.
In this tutorial, you will make a PHP service using Open Swoole that retains caller context, preferences and action history across multiple separate inbound calls.
Prerequisites
To complete this tutorial you will need:
- A free Twilio account with a voice-capable phone number
- Composer installed globally
- PHP 8.5 or later
- An OpenAI API key
- ngrok to expose local webhooks to Twilio
- An IDE or text editor such as Neovim, PhpStorm, or Visual Studio Code
Building the app
Step 1 - Set up the PHP project
To get started, create a new PHP project:
Step 2 - Install required dependencies
Install the Twilio, OpenAI, PHP Dotenv, Monolog, and Guzzle packages.
The Twilio package will allow your application to interface with Twilio's services. The PHP Dotenv package allows you to import your environment variables into your solution using a .env file. You will add those variables in the next step. The OpenAI package will be used to connect your solution to OpenAI. Monolog will be used for simplified logging.
Then, create a custom PSR-4 namespace named "App" by adding the following configuration to composer.json:
Step 3 - Create a Twilio Memory Store
For this tutorial, you will need a Conversation Memory Store. Go into your Twilio console and look for Memory Stores. You can use the console search, or look for Orchestration > Conversation Memory > Memory Stores.
Memory Stores use machine learning, and you may have to agree to a warning before proceeding. Keep in mind that Conversation Memory is not intended for use with sensitive information. Conversation products are only available on the new Twilio Console, so make sure your account has been migrated. For more information about Conversation Memory, you may want to read the documentation, including the Getting Started Guide.
Once you have found the correct tab, click on Create New Store.
Now follow the steps to set up your memory store.
The console gives you a setup checklist to get you started. Click on Connect Conversation Orchestrator, and give it a friendly name. You write a short description, then can move on to Messaging and Chat Traffic. For the remainder of the items in this checklist, you can select the default values for now.
You don't have any customer profiles yet, so you can skip the rest of the checklist. However, you will need your memory store ID, which is at the top left of the memory store screen. There should be a convenient button to copy-paste that ID. Keep that ID for the next step.
Step 4 - Configure environment variables
Now that you have a memory store created, you will need to be able to access that from your application. For this, you will need to get your Memory Store ID and paste that into your secrets file. Create a .env file in the root directory of your project. Add the following values, replacing the placeholders.
Get your API key from your Twilio console, created under Settings > Account Settings > API Keys & Auth Tokens. Creating a Main API key is the simplest method for this tutorial. You get your memory store key from the previous step and paste it in here. Your OpenAI API Key is generated from OpenAI's dashboard.
Save the file, and move on to the next step, creating your services.
Step 5 - Set up the OpenAI service
This demonstration uses the fiction of an auto repair shop as the agent that you are calling. However, Conversation Memory would be useful in lots of different scenarios, such as tech support, travel, and more. Feel free to adjust the audio prompts as you see fit for your own personal projects.
Create a new file in src/Service named OpenAiService.php class to handle interaction with gpt-4o-mini.
Paste the following into your new class:
This code handles your initial connection to OpenAI. Its function is to parse the information from a caller and stream it to the OpenAI API. Notice the system prompt here, which explains the functionality of the agent. It contains some useful instructions for the agent, such as to avoid bullet points and emojis when speaking on the phone.
Next, create the webhook for Twilio's connection.
Step 6 - Build the Twilio webhook and Conversation Memory pipeline
Create another new file, this time in src/WebSocket called ConversationRelayHandler.php. Paste in this code:
ConversationRelayHandler is the bridge between Twilio's WebSocket and the rest of the app. When Twilio opens the socket after a <ConversationRelay> TwiML directive, the class loops reading frames one at a time, assembling multi-part messages into a memory context. A prompt frame carries the caller's transcribed speech. It appends the text to an in-memory conversation history and hands the whole history plus the memory context off to OpenAiService::streamResponse, which streams tokens back out through the same socket.
The code also contains interruption handling for your agent. If the agent is interrupted during a conversation, it pops the last message off of the history so the agent realizes the full message was not sent and was incomplete. This will allow the customer to continue talking and handle the interruption in a more human way, without your user missing context.
Step 7 - Process multi-turn historical context
Now you will create one more class to handle the context and memory processing. You'll call this file ConversationMemoryService.php and create it in src/Service. Paste the following into the file:
This part of the code is what will handle your multi-turn conversation.
ConversationMemoryService is a thin wrapper around Twilio's Conversation Memory REST API that turns a caller's phone number into a prompt-ready string of prior-call context. It reads from environment variables and configures your HTTP client.
The public entry point getContext() runs a two-step lookup: first lookupProfileIdId() normalizes the number, matches it against a canonical profile, and returns a profile ID (or null if no previous caller with that number was found). If a profile was found, recall() posts, asking for up to 20 observations and 5 summaries. The response is logged raw for inspection and then passed to formatRecall(), which parses the response.
The application tries several plausible field names (text, content, observation, summary, value) via ExtractText and stitches whatever it finds into a bulleted string. That string is what eventually gets prepended to the OpenAI system prompt as "Prior context on this customer," making the caller's history part of the model's instructions before they've even spoken.
Step 8 - Finalize your application
To complete your project you will need to create two further files. Firstly, create a file named WebSocketServer.php in src/WebSocket, and paste the code below into the file:
This sets up the websockets to call your ConversationRelayHandler, and includes the initial greeting for your user. Feel free to change the greeting according to your needs.
Now, create a file named index.php in the project's top-level directory, then paste the code below into the file.
This initialises the WebSocketServer, starting it listening on port 9501 on the local machine (or the port that you set in .env, if you changed the default value.
Testing, troubleshooting, or product demonstration
It is now time to test your voice application. Save your files and run the project with:
Once your webhook is running, you will need to expose it to the internet by using ngrok or another tunneling service.
Replace 9501 with whatever port your application is running on if you have a different port shown.
Now ngrok will provide you with a url for utilizing in your Twilio console. Go into your Twilio console and find the Twilio phone number that you prepared. Under the option A Call Comes In, choose Webhook, and fill in your ngrok URL followed by /voice, as shown in the graphic below:
To put your AI to the test, you'll have to make two phone calls and check the conversation memory.
- Make call #1: Talk to the agent about a car repair issue, including some details such as make and model.
- Hang up and make call #2 from the same phone number.
- Verify the agent greets you and remembers the details of your first call without any prompting.
Conclusion
Today you have learned how Twilio Conversation Memory simplifies maintaining state across separate voice calls in PHP. This should provide value to any phone AI agent, storing information that keeps conversations feeling more convenient and human.
Do you want to do more with Twilio Conversations? Explore the possibilities by checking out the conversations documentation, where you can find blueprints for Conversational Agents, AI-to-Human handoff, and more.
Matthew Setter is a PHP, Go, and Rust Editor in the Twilio Voices team. He’s also the author of Mezzio Essentials and Deploy with Docker Compose. You can find him at msetter@twilio.com. He's also on LinkedIn and GitHub.
Related Posts
Related Resources
Twilio Docs
From APIs to SDKs to sample apps
API reference documentation, SDKs, helper libraries, quickstarts, and tutorials for your language and platform.
Resource Center
The latest ebooks, industry reports, and webinars
Learn from customer engagement experts to improve your own communication.
Ahoy
Twilio's developer community hub
Best practices, code samples, and inspiration to build communications and digital engagement experiences.