How to Augment Voice Calls with Twilio Intelligence

September 29, 2026
Written by

How to Augment Voice Calls with Twilio Intelligence

You know how to build a voice AI agent from scratch, one that supports speech recognition, text-to-speech, turn detection, and real-time audio streaming — all at low latency. But, what if you could also analyze customer conversations in real time and collect useful data insights for future calls?

With Twilio Conversation Intelligence and Conversation Orchestrator, backed by Conversation Memory, you can!

Specifically, in this tutorial you're going to learn how to use these three technologies together to retrieve a short summary of each customer call, and an analysis of the caller's sentiment. What's more, you'll also see whether your agent followed the guidelines you set for it. All of this information will then be persisted to a SQLite database, so that you can make use of it later.

Programming language support

This tutorial is geared toward PHP developers. If you would like to build this project in a different programming language, see the following options:

Architecture

As this tutorial adds three new technologies to the previous application, here's a quick overview of how the new functionality works.

You will add a new route which receives a POST (webhook) request from Twilio after customer calls end. The request body will be a JSON string that contains, among other things, a short summary of the call, an assessment of the caller's sentiment, and how the agent adhered to a series of criteria. That information will be extracted from the request and then persisted to the application's SQLite database.

You're not going to do more with the received information. But, there are links at the end of the tutorial showing how you could continue building on the changes made in this tutorial, should you want to.

Prerequisites

To follow along with the tutorial, you will need the following:

Build the app

Step 1: Set up Conversation Orchestrator and Conversation Memory

Before you can set up Conversation Intelligence, which does most of the work, you need to create a Memory Store and Conversation Configuration.

To do that, sign in to the Twilio Console, and go to Products & Services > Conversation Orchestrator > Conversation configurations. There, click Create a Conversation configuration. On the Name Configuration step, enter a name and description, then click Next.

Twilio setup screen showing the Name Configuration step with fields for Conversation name and description.

On the Messaging and chat traffic step, click Next. On the Voice traffic step, scroll down and enable the Set up automatic capture checkbox. From the Voice phone numbers list, select your Twilio phone number, then click Next.

Now, on the Configure lifecycle step click Next. After that, on the Enable Conversation Memory step, create a memory store, by clicking Create new memory store, entering a name In the Memory store name field, and clicking Save.

Interface screen for creating a new memory store with fields for name and description, and buttons for Save and Cancel.

From the Memory store list, select the memory store you just created, leave Turn on observations and summaries enabled, and click Next.

Finally, On the Summary step, review your settings and click Create Conversation configuration. Copy the conversation configuration ID for use later.

Step 2: Set up Conversation Intelligence

Before you can complete this step, you need to make the application publicly accessible on the internet as you'll need the ngrok URL later in this section. Run the command, below, to create a connection to the app on port 8080.

ngrok http 8080

To do that, go to Products & Services > Conversation Intelligence > Intelligence configurations. There, click Create Intelligence configuration. Add a name, description, and, in the Attach Conversation configurations section, select the name of the Conversation configuration that you created in the previous step, and click Submit.

With that done, in Intelligence configurations, click Create rule next to the Intelligence configuration which you just created. Then, in the Add language operators section, enable Sentiment, Summary, and Script-Adherence, and click Next.

Now, in the Script-Adherence section, at the bottom of the Set Parameters step, add the following text into the script field and click Next.

Category: introduction
- introduction: The agent should identify themselves by first name. Required Phrase: Thanks for calling Owl Air! I'm Hoot.

Category: assistance
- offer_assistance: I can help with flight status, baggage policy, loyalty points, or booking changes. Which of those can I help you with?
Screen showing settings for script adherence rules in a customer service platform, with options to add language operators.

Now, on the Trigger and action step, choose At conversation end in the Trigger section. Then, in the Action section, paste your ngrok URL plus "/intelligence-results" in the Webhook action field. Click Next.

Screenshot of rule creation interface with options to add operators, set parameters, and enable conversation memory.

In the Add context step, scroll down to the Conversation Memory section and enable Enable Conversation Memory for this rule and click Next. In the Summary step, click Create rule.

Step 3: Update the existing project structure

Now, it's time to start augmenting the PHP code. But, before you can do that, you have to add a few new directories by running the following command.

mkdir -p data/database src/App/src/Types/OperatorResults/ScriptAdherence/Categories
If you're using Microsoft Windows, the -p option is not required.

The data/database directory will store the application's SQLite database and a SQL file defining the database's schema. The src/App/src/Types/OperatorResults directory will contain a series of plain old PHP classes which will store and model the information extracted from the webhook received from Twilio; you'll create those classes shortly.

Step 4: Set up the application's database

Create a new file named dump.sql in the data/database directory and paste the following SQL into that file.

-- Enable SQLite's WAL mode
PRAGMA journal_mode=WAL;

-- Enable foreign key support
PRAGMA foreign_keys = ON;

CREATE TABLE IF NOT EXISTS intelligence_results
(
    'conversation_id' TEXT NOT NULL PRIMARY KEY,
    'call_started' TEXT NOT NULL,
    'call_ended' TEXT NOT NULL
);

CREATE TABLE IF NOT EXISTS operators
(
    'operator_type' TEXT NOT NULL,
    'operator_value' TEXT NOT NULL,
    'conversation_id' TEXT NOT NULL,
     FOREIGN KEY(conversation_id) REFERENCES intelligence_results(conversation_id)
);

CREATE TABLE IF NOT EXISTS operator_script_adherence_categories
(
    'operator_id' INTEGER NOT NULL,
    'category_name' TEXT NOT NULL,
    'category_met' INTEGER NOT NULL DEFAULT false,
     FOREIGN KEY(operator_id) REFERENCES operators(rowid)
);

The instructions:

  • Enable SQLite's WAL (Write-Ahead Logging) mode (which, among other benefits, significantly improves performance)
  • Enable foreign key support
  • Define three tables:
    • intelligence_results: stores the core information about the conversation
    • operators: stores the information extracted by Conversation Intelligence, such as the call summary and intent
    • operator_script_adherence_categories: stores the script adherence information, linking it to the relevant record in operators

It's not the most sophisticated schema, but it can store the information in a maintainable way.

Now, use SQLite's Command-Line Shell (or your preferred database management tool) to provision the database with the following command.

sqlite3 data/database/database.sqlite3 < data/database/dump.sql

Step 5: Create the Conversation Intelligence route

Add the required packages

The application needs one extra package, PHP-DB's SQLite adapter, to simplify interacting with the application's SQLite database. To install it, in composer.json set minimum-stability to dev (if it's not already set to that). Then, install the package with the following command:

composer require php-db/phpdb-sqlite -W --ignore-platform-reqs

Accept the default answers to the two prompts which you'll see during installation of the package.

Create a route and handler for processing the webhook

Next, using Mezzio's command-line tooling to create the handler class and an accompanying factory class to handle instantiating it, by running the command below:

composer mezzio mezzio:handler:create -- \
    --no-register \
    "App\Handler\IntelligenceResultsHandler"

Both classes are located in src/App/src/Handler, with the handler class being named IntelligenceResultsHandler.php and the factory class named IntelligenceResultsHandlerFactory.php. Open IntelligenceResultsHandler.php and replace the existing code with the code below.

<?php

declare(strict_types=1);

namespace App\Handler;

use App\Service\DatabaseService;
use App\Types\OperatorResults;
use Laminas\Diactoros\Response\JsonResponse;
use Psr\Http\Message\ResponseInterface;
use Psr\Http\Message\ServerRequestInterface;
use Psr\Http\Server\RequestHandlerInterface;
use Psr\Log\LoggerInterface;
use Twilio\Rest\Client as TwilioClient;
use Twilio\Exceptions\TwilioException;

use function json_decode;

use const JSON_OBJECT_AS_ARRAY;

class IntelligenceResultsHandler implements RequestHandlerInterface
{
    public function __construct(
        private readonly LoggerInterface $logger,
        private readonly TwilioClient $twilioClient,
        private readonly DatabaseService $dbService
    ) {
    }

    public function handle(ServerRequestInterface $request): ResponseInterface
    {
        $data = (array) json_decode(
            $request->getBody()->getContents(),
            associative: true,
            flags: JSON_OBJECT_AS_ARRAY,
        );
        $conversationId  = $data['conversationId'] ?? '';
        $operatorResults = $data['operatorResults'] ?? [];
        $operators = [];
        foreach ($operatorResults as $operatorData) {
            $operatorType = match ($operatorData['operator']['displayName'] ?? 'Unknown') {
                'Summary' => OperatorResults\Summary::class,
                'Sentiment' => OperatorResults\Sentiment::class,
                'Script-Adherence' => OperatorResults\ScriptAdherence::class,
            };
            $operators[] = $this->importOperator($operatorType, $conversationId, $operatorData);
        }

        try {
            $conversation = $this->twilioClient
                ->conversations
                ->v2
                ->conversations($conversationId)
                ->fetch();
            $this->dbService->recordCall(
                $conversationId,
                $conversation->createdAt,
                $conversation->updatedAt,
                $operators
            );
        } catch(TwilioException $e) {
            $this->logger
                ->debug(
                    'Conversation retrieval failed',
                    [
                        'conversation id' => $conversationId,
                        'error' => $e->getMessage(),
                    ],
                );
        }

        return new JsonResponse('Log Data Received');
    }

    private function importOperator(
        string $operator,
        string $conversationId,
        array $data = []
    ): OperatorResults\TypeOperatorInterface|null
    {
        if ($operator === OperatorResults\Sentiment::class) {
            return new OperatorResults\Sentiment(
                $data['result']['label'] ?? '',
                $conversationId,
            );
        }

        if ($operator === OperatorResults\ScriptAdherence::class) {
            $categories = [];
            foreach ($data['result']['categories'] ?? [] as $category) {
                $categories[] = new OperatorResults\ScriptAdherence\Categories\Category(
                    $category['category_key'] ?? '',
                    match ($category['criteria']['criteria_met'] ?? '') {
                        'Succeeded' => true,
                        'Failed' => false,
                        default => false,
                    },
                );
            }

            return new OperatorResults\ScriptAdherence(
                $data['parameters']['script'] ?? '',
                $conversationId,
                $categories,
            );
        }

        if ($operator === OperatorResults\Summary::class) {
            return new OperatorResults\Summary(
                $data['result']['text'] ?? '',
                $conversationId,
            );
        }

        return null;
    }
}

The handle() function is the central focus of the class. It's called when the "/intelligence-results" route is requested. It starts off by deserialising the JSON request body into a PHP array, before progressively extracting the essential information from the deserialised request data. This is the conversation ID (the conversation's unique identifier), and the details that Conversation Intelligence determined about the call, contained in the operatorResults element, using the importOperator() function.

The importOperator() function instantiates a Sentiment object from the result.label element, a Summary object from the result.text element, and a ScriptAdherence object from the result.categories element.

With the relevant information collected, the function makes a call to the Conversations (v2) API to get the start and end time of the call (details which aren't available in the received webhook data). Then, using the DatabaseService's recordCall() function, the collated information is persisted to the SQLite database.

Now, open IntelligenceResultsHandlerFactory.php and replace the existing code with the code below.

<?php

declare(strict_types=1);

namespace App\Handler;

use App\Service\DatabaseService;
use Psr\Container\ContainerInterface;
use Psr\Log\LoggerInterface;
use Twilio\Rest\Client as TwilioClient;

class IntelligenceResultsHandlerFactory
{
    public function __invoke(ContainerInterface $container): IntelligenceResultsHandler
    {
        return new IntelligenceResultsHandler(
            $container->get(LoggerInterface::class),
            $container->get(TwilioClient::class),
            $container->get(DatabaseService::class),
        );
    }
}

You can see that the __invoke() function, called when the IntelligenceResultsHandler service is retrieved from the app's DI container, instantiates (and returns) a new IntelligenceResultsHandler object with a LoggerInterface, TwilioClient, and DatabaseService object.

Now, register the IntelligenceResultsHandler as a service with the DI container by adding the following line to the factories element of the array returned from the getDependencies() function in src/App/src/ConfigProvider.php.

Handler\IntelligenceResultsHandler::class => Handler\IntelligenceResultsHandlerFactory::class,

When requested, the IntelligenceResults service will be instantiated and returned by Handler\IntelligenceResultsHandlerFactory.

Create the plain-old PHP objects

Now, it's time to create the classes to store the Conversation Intelligence data. Start off by creating a file named Summary.php in src/App/src/App/Types/OperatorResults, and paste the code, below, into the file.

<?php

declare(strict_types=1);

namespace App\Types\OperatorResults;

final readonly class Summary implements TypeOperatorInterface
{
    public function __construct(
        public string $summary,
        public string $conversationId,
    )
    {
    }

    public function getValue(): string
    {
        return $this->summary;
    }
}

Then, create a file named Sentiment.php in src/App/src/App/Types/OperatorResults, and paste the code, below, into the file.

<?php

declare(strict_types=1);

namespace App\Types\OperatorResults;

final readonly class Sentiment implements TypeOperatorInterface
{
    public function __construct(
        public string $sentiment,
        public string $conversationId,
    )
    {
    }

    public function getValue(): string
    {
        return $this->sentiment;
    }
}

Create another file, this time named ScriptAdherence.php in src/App/src/App/Types/OperatorResults, and paste the code, below, into the file.

<?php

declare(strict_types=1);

namespace App\Types\OperatorResults;

use App\Types\OperatorResults\Categories\Category;

final readonly class ScriptAdherence implements TypeOperatorInterface
{
    public function __construct(
        public string $summary,
        public string $conversationId,
        /** @var list<Category> */
        public array $categories = []
    )
    {
    }

    public function getValue(): string
    {
        return $this->summary;
    }
}

Create a file named Category.php in src/App/src/App/Types/OperatorResults/Categories, and paste the code, below, into the file.

<?php

declare(strict_types=1);

namespace App\Types\OperatorResults\ScriptAdherence\Categories;

final readonly class Category
{
    public function __construct(
        public string $name,
        public bool $met,
    )
    {
    }
}

Finally, create the interface which the first three classes implement, by creating a file named TypeOperatorInterface.php in src/App/src/App/Types/OperatorResults, and paste the code, below, into the file.

<?php

declare(strict_types=1);

namespace App\Types\OperatorResults;

interface TypeOperatorInterface
{
    public function getValue(): string;
}

The reason that the first three classes implement TypeOperatorInterface is to make it simpler to work with them in DatabaseService, which we'll create shortly.

Update the application's routing table

To do that, open config/routes.php, and update the static function returned from the file to match the version below.

return static function (Application $app, MiddlewareFactory $factory, ContainerInterface $container): void {
    $app->post('/intelligence-results', 
        [
            BodyParamsMiddleware::class,
            Handler\IntelligenceResultsHandler::class
        ], 
        'intelligence-results'
    );
    $app->post('/twiml', Handler\TwimlHandler::class, 'twiml');
    $app->get('/api/ping', Handler\PingHandler::class, 'api.ping');
};

The new route accepts only POST requests to the "/intelligence-results" endpoint, passing requests through BodyParamsMiddleware (which adds support to Mezzio for processing JSON request bodies) and then to IntelligenceResultsHandler.

Now, update the file's use statements to match the list below.

use App\Handler;
use Mezzio\Application;
use Mezzio\Helper\BodyParams\BodyParamsMiddleware;
use Mezzio\MiddlewareFactory;
use Psr\Container\ContainerInterface;

Create the database service

It's time to create the database service which IntelligenceResultsHandler uses to simplify persisting the retrieved webhook data into the application's database. In src/App/src/Service, create a file named DatabaseService.php, and paste the code below into the file.

<?php

declare(strict_types=1);

namespace App\Service;

use App\Types\OperatorResults\{
    ScriptAdherence,
    Sentiment,
    Summary,
    TypeOperatorInterface
};
use DateTime;
use PhpDb\Adapter\AdapterInterface;
use PhpDb\TableGateway\TableGateway;

final class DatabaseService
{
    public function __construct(private readonly AdapterInterface $adapter) {}

    /**
     * @param list<TypeOperatorInterface> $operators
     */
    public function recordCall(
        string $conversationId,
        DateTime $startedAt,
        DateTime $endedAt,
        array $operators
    ): void {
        $intelligenceResultsTableGateway = new TableGateway('intelligence_results', $this->adapter);
        $result = $intelligenceResultsTableGateway->insert([
            'conversation_id' => $conversationId,
            'call_started'    => $startedAt->format('Y-m-d H:i:s'),
            'call_ended'      => $endedAt->format('Y-m-d H:i:s'),
        ]);
        $operatorsTableGateway = new TableGateway('operators', $this->adapter);
        foreach ($operators as $operator) {
            $operatorType = match ($operator::class) {
                ScriptAdherence::class => 'script-adherence',
                Summary::class => 'summary',
                Sentiment::class => 'sentiment',
            };

            $operatorsTableGateway->insert([
                'conversation_id' => $conversationId,
                'operator_value'  => $operator->getValue(),
                'operator_type'   => $operatorType,
            ]);
            if (
                $operator instanceof ScriptAdherence
                    && $operator->categories !== []
            ) {
                $operatorId             = $operatorsTableGateway->getLastInsertValue();
                $categoriesTableGateway = new TableGateway('operator_script_adherence_categories', $this->adapter);

                /** @var list<Category> $categories */
                $categories = $operator->categories;
                foreach ($categories as $category) {
                    $categoriesTableGateway->insert([
                        'operator_id'   => $operatorId,
                        'category_name' => $category->name,
                        'category_met'  => $category->met,
                    ]);
                }
            }
        }
    }
}

The class' recordCall() function uses PHP-DB/SQLite's TableGateway class to simplify interacting with the tables in the database. If you're not familiar with the "TableGateway" or "Table Data Gateway" pattern, quoting Martin Fowler:

Table Data Gateway is an object that acts as a gateway to a database table. One instance handles all the rows in the table.

In short, you don't have to know, almost, anything about SQL to manage records in a database table, as the class has functions to handle that for you. Instead, you can work with tables in an object-oriented way.

Next, you need to create a factory class to instantiate DatabaseService, when requested from the DI container. To do that, create another file in src/App/src/Service, this time named DatabaseServiceFactory.php, and paste the code below into the file.

<?php

declare(strict_types=1);

namespace App\Service;

use PhpDb\Adapter\AdapterInterface;
use Psr\Container\ContainerInterface;

final class DatabaseServiceFactory
{
    public function __invoke(ContainerInterface $container): DatabaseService
    {
        return new DatabaseService($container->get(AdapterInterface::class));
    }
}

When the __invoke() function is called, it will instantiate a new DatabaseService object with the AdapterInterface service, retrieved from the application's DI container, and return it.

The AdapterInterface service is the central object in PHP-DB. It is responsible for adapting any code written in or for PHP-DB to the targeted PHP extensions and vendor databases, i.e., SQLite.

Finally, for this section, you need to provide the configuration for PHP-DB to know which database to use and where to find it. To do that, in config/autoload, create a file named database.global.php, and paste the code below into the file.

<?php

declare(strict_types=1);

use PhpDb\Adapter\AdapterInterface;
use PhpDb\Adapter\Driver\PdoDriverInterface;

return [
    AdapterInterface::class => [
        'driver'     => PdoDriverInterface::class,
        'connection' => [
            'dsn' => sprintf('sqlite:%s', __DIR__ . "/../../data/database/database.sqlite3"),
        ],
    ],
];

This tells PHP-DB to use the PdoDriverInterface from PHP-DB/SQLite for interacting with the database, and the DSN (Data Source Name) containing the path to the SQLite database file.

Now, register the DatabaseService as a service with the DI container by adding the following line to the factories element of the array returned from the getDependencies() function in src/App/src/ConfigProvider.php.

DatabaseService::class => DatabaseServiceFactory::class,

Then, add the following use statement to the top of the file.

use App\Service\{DatabaseService,DatabaseServiceFactory}

Register a Twilio Rest Client with the DI container

Now, you need to register a Twilio Rest Client, as the application will need it to query the Conversations (v2) API to retrieve a conversation's start and end time, in the IntelligentResultsHandler. To do that, in src/App/src/Factory, create a file named TwilioClientFactory.php, and in that file paste the code below.

<?php

declare(strict_types=1);

namespace App\Factory;

use Psr\Container\ContainerInterface;
use Twilio\Rest\Client as TwilioClient;

class TwilioClientFactory
{
    public function __invoke(ContainerInterface $container): TwilioClient
    {
        return new TwilioClient(
            $_ENV['TWILIO_ACCOUNT_SID'] ?? getenv('TWILIO_ACCOUNT_SID'),
            $_ENV['TWILIO_AUTH_TOKEN'] ?? getenv('TWILIO_AUTH_TOKEN'),
        );
    }
}

Next, in src/App/src/ConfigProvider.php, add the following line to the factories element of the array returned from the getDependencies() function.

TwilioClient::class => TwilioClientFactory::class,

Then, add the following use statement to the top of the file.

use Twilio\Rest\Client as TwilioClient;

Step 6: Start the application

With the code now complete, start the application by running the command below in your terminal, after replacing the four environment variable placeholders with their respective values.

TWILIO_ACCOUNT_SID=<Your Twilio Account SID> \
TWILIO_AUTH_TOKEN=<Your Twilio Auth Token> \
DOMAIN=<Your ngrok URI> \
OPENAI_API_KEY=<Your OpenAI API key> \
vendor/bin/laminas mezzio:swoole:start
If you're running Microsoft Windows, replace the backslashes in the command above with carets (^), or remove the backslashes and put the command all on one line.

You will see output similar to the example below, after the application starts.

WebSocketServerFactory built: Swoole\WebSocket\Server
[2026-09-18T01:10:41.895336+00:00] swoole-http-server.NOTICE: Worker started in /opt/php/owl-air-phone-agent with ID 0 {"cwd":"/opt/php/owl-air-phone-agent","pid":0} []
[2026-09-18T01:10:41.895497+00:00] swoole-http-server.NOTICE: Worker started in /opt/php/owl-air-phone-agent with ID 1 {"cwd":"/opt/php/owl-air-phone-agent","pid":1} []
[2026-09-18T01:10:41.895261+00:00] swoole-http-server.NOTICE: Worker started in /opt/php/owl-air-phone-agent with ID 3 {"cwd":"/opt/php/owl-air-phone-agent","pid":3} []
[2026-09-18T01:10:41.895291+00:00] swoole-http-server.NOTICE: Worker started in /opt/php/owl-air-phone-agent with ID 2 {"cwd":"/opt/php/owl-air-phone-agent","pid":2} []
[2026-09-18T01:10:41.902221+00:00] swoole-http-server.NOTICE: Swoole is running at 0.0.0.0:8080, in /opt/php/owl-air-phone-agent {"host":"0.0.0.0","port":8080,"cwd":"/opt/php/owl-air-phone-agent"} []

Test that the app works as expected

With the application running, call your Twilio phone number. You should hear Hoot's greeting within a second or two of the call connecting:

"Thanks for calling Owl Air! I'm Hoot. I can help with flight status, baggage policy, loyalty points, or booking changes. Which of those can I help you with?"

Then, like when you tested the first version of the app, try a few test questions to verify the full flow is working:

  • "What's the baggage policy?": Hoot should describe carry-on and checked bag rules in natural spoken language.
  • "How do loyalty points work?": Hoot should explain the earn and redemption rates.
  • "Can I change my flight?": Hoot should give the change fee policy, with amounts spelled out in words.
If you hear an error message on the call or see an error message in your terminal logs, check the error code against the Conversation Relay error code reference in the Twilio documentation.

After the call finishes, using your database tool of choice, have a look at the records in the application's database. There, you should see a summary of the conversation, along with the related operator information.

Conclusion

You've now learned how to use Conversation Intelligence, Conversation Orchestrator, and Conversation Memory, as well as the Conversations API (V2), to retrieve a short summary of each call and an analysis of the caller's sentiment, and persist the information to a SQLite database, so that you can make use of it later. What's more, you also know whether your agent followed the guidelines you set for it.

But don't stop there! Now that the application can store conversation information, why not add a route for viewing a summary of all stored conversations, and one for viewing individual conversation details?

Then, I strongly encourage you to learn more about Conversation Intelligence, Conversation Orchestrator, and Conversation Memory, as well as the Conversations API (V2).

Matthew Setter is a PHP, Go, and Rust Editor in the Twilio Voices team. He’s also the author of Mezzio Essentials and Deploy with Docker Compose. You can find him at msetter@twilio.com. He's also on LinkedIn and GitHub.