Contact sales Start for free
Contact sales Start for free

How to Augment Voice Calls with Twilio Intelligence in C#

Time to read:

September 29, 2026
Written by
Amanda Lange
Twilion
Matthew Setter
Twilion

How to Augment Voice Calls with Twilio Intelligence in C#

You know how to build a voice AI agent from scratch, one that supports speech recognition, text-to-speech, turn detection, and real-time audio streaming — all at low latency. But, what if you could also analyze customer conversations in real time and collect useful data insights for future calls?

With Twilio Conversation Intelligence and Conversation Orchestrator, backed by Conversation Memory, you can!

Specifically, in this tutorial you're going to learn how to use these three technologies together to retrieve a short summary of each customer call, and an analysis of the caller's sentiment. What's more, you'll also see whether your agent followed the guidelines you set for it. All of this information will then be persisted to a SQLite database, so that you can make use of it later.

Architecture

As this tutorial adds three new technologies to the previous application, here's a quick overview of how the new functionality works.

You will add a new route which receives a POST (webhook) request from Twilio after customer calls end. The request body will be a JSON string that contains, among other things, a short summary of the call, an assessment of the caller's sentiment, and how the agent adhered to a series of criteria. That information will be extracted from the request and then persisted to the application's SQLite database.

You're not going to do more with the received information. But, there are links at the end of the tutorial showing how you could continue building on the changes made in this tutorial, should you want to.

Prerequisites

To follow along with the tutorial, you will need the following:

Build the app

Step 1: Set up Conversation Orchestrator and Conversation Memory

Before you can set up Conversation Intelligence, which does most of the work, you need to create a Memory Store and Conversation Configuration.

To do that, sign in to the Twilio Console, and go to Products & Services > Conversation Orchestrator > Conversation configurations. There, click Create a Conversation configuration. On the Name Configuration step, enter a name and description, then click Next.

Twilio Console screen showing Name Configuration for setting up Conversations and Profiles

On the Messaging and chat traffic step, click Next. On the Voice traffic step, scroll down and enable the Set up automatic capture checkbox. From the Voice phone numbers list, select your Twilio phone number, then click Next.

Now, on the Configure lifecycle step click Next. After that, on the Enable Conversation Memory step, create a memory store, by clicking Create new memory store, entering a name In the Memory store name field, and clicking Save.

Interface for creating a new memory store with fields for name and description and options on the left sidebar.

From the Memory store list, select the memory store you just created, leave Turn on observations and summaries enabled, and click Next.

Finally, On the Summary step, review your settings and click Create Conversation configuration. Copy the conversation configuration ID for use later.

Step 2: Set up Conversation Intelligence

Before you can complete this step, you need to make the application publicly accessible on the internet as you'll need the ngrok URL later in this section. Run the command, below, to create a connection to the app on port 5000. 

ngrok http 5000

Now it is time to configure your Conversation Intelligence.

To do that, go to Products & Services > Conversation Intelligence > Intelligence configurations. There, click Create Intelligence configuration. Add a name, description, and, in the Attach Conversation configurations section, select the name of the Conversation configuration that you created in the previous step, and click Submit.

With that done, in Intelligence configurations, click Create rule next to the Intelligence configuration which you just created. Then, in the Add language operators section, enable Sentiment, Summary, and Script-Adherence, and click Next.

Now, in the Script-Adherence section, at the bottom of the Set Parameters step, add the following text into the script field and click Next.

Category: introduction
- introduction: The agent should identify themselves by first name. Required Phrase: Thanks for calling Owl Air! I'm Hoot.
Category: assistance
- offer_assistance: I can help with flight status, baggage policy, loyalty points, or booking changes. Which of those can I help you with?
User interface showing script adherence rule setup form with fields for script, version, and other parameters.

Now, on the Trigger and action step, choose At conversation end in the Trigger section. Then, in the Action section, paste your ngrok URL plus "/intelligence-results" in the Webhook action field. Click Next.

Screenshot of rule creation interface with options for language operators, setting parameters, and adding context.

In the Add context step, scroll down to the Conversation Memory section and enable Enable Conversation Memory for this rule and click Next. In the Summary step, click Create rule.

Step 3: Update the existing project structure

Add some new directories to your project by running the following command: 

mkdir -p data/database Models Services Controllers
If you're using Microsoft Windows, the -p option is not required.

This data/database directory will store the application's SQLite database and a SQL file defining the database's schema. The other folders, which may or may not already exist in your base project, will help structure the .NET application.

Step 4: Set up the application's database

Create a new file named dump.sql in the data/database directory and paste the following SQL into that file. 

-- Enable SQLite's WAL mode
PRAGMA journal_mode = WAL;
-- Enable foreign key support
PRAGMA foreign_keys = ON;
CREATE TABLE IF NOT EXISTS intelligence_results
(
   "conversation_id" TEXT NOT NULL PRIMARY KEY,
   "call_started"    TEXT NOT NULL,
   "call_ended"      TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS operators
(
   "operator_id"     INTEGER PRIMARY KEY AUTOINCREMENT,
   "operator_type"   TEXT NOT NULL,
   "operator_value"  TEXT NOT NULL,
   "conversation_id" TEXT NOT NULL,
   FOREIGN KEY (conversation_id) REFERENCES intelligence_results (conversation_id)
);
CREATE TABLE IF NOT EXISTS operator_script_adherence_categories
(
   "operator_id"   INTEGER NOT NULL,
   "category_name" TEXT NOT NULL,
   "category_met"  INTEGER NOT NULL DEFAULT 0,
   FOREIGN KEY (operator_id) REFERENCES operators (operator_id)
);

The instructions:

  • Enable SQLite's WAL (Write-Ahead Logging) mode (which, among other benefits, significantly improves performance)
  • Enable foreign key support
  • Define three tables:
  • intelligence_results: stores the core information about the conversation
  • operators: stores the information extracted by Conversation Intelligence, such as the call summary and intent
  • operator_script_adherence_categories: stores the script adherence information, linking it to the relevant record in operators

It's not the most sophisticated schema, but it can store the information in a maintainable way.

Now, use SQLite's Command-Line Shell (or your preferred database management tool) to provision the database with the following command. 

sqlite3 data/database/database.sqlite3 < data/database/dump.sql

Step 5: Create the Conversation Intelligence route

Add the required packages

The application needs a few NuGet packages to run: Dapper, a lightweight object mapper that simplifies working with the SQLite database, and Microsoft.Data.Sqlite, the official SQLite driver for .NET. You will also need to be sure you are running the latest version of the Twilio package. Install these dependencies by running the following commands from the project root:

dotnet add package Dapper
dotnet add package Microsoft.Data.Sqlite
dotnet add package Twilio

Create the operator result records

Now, create the small immutable data types that will hold the pieces of information extracted from the webhook. Each of the three operator types — Summary, Sentiment, and ScriptAdherence — implements a common ITypeOperator interface, so that the database repository can treat them uniformly. ScriptAdherence additionally holds a list of Category records, one per script-adherence category that Conversation Intelligence evaluates. In a final build these would all be different files for extensibility. But for the purpose of a tutorial, because these are small, related, immutable data types, you can keep all five in a single file. In the Models folder of your existing project, create a file named OperatorResults.cs, and paste the code below into it. 

namespace OwlAir;
public interface ITypeOperator
{
   string ConversationId { get; }
   string GetValue();
}
public sealed record Summary(string SummaryText, string ConversationId) : ITypeOperator
{
   public string GetValue() => SummaryText;
}
public sealed record Sentiment(string SentimentLabel, string ConversationId) : ITypeOperator
{
   public string GetValue() => SentimentLabel;
}
public sealed record ScriptAdherence(
   string Script,
   string ConversationId,
   IReadOnlyList<Category> Categories) : ITypeOperator
{
   public string GetValue() => Script;
}
public sealed record Category(string Name, bool Met);

Create the database service

It's time to create the database service that the webhook controller will use to persist the retrieved webhook data into the application's SQLite database.

In the Services folder, create a file named DatabaseService.cs, and paste the code below into the file. 

using Dapper;
using Microsoft.Data.Sqlite;
namespace OwlAir;

public class DatabaseService
{
  private readonly string _connectionString;
  public DatabaseService()
  {
      _connectionString = Environment.GetEnvironmentVariable("SQLITE_CONNECTION_STRING")
          ?? "Data Source=data/database/database.sqlite3";
  }
  public void RecordCall(
      string conversationId,
      DateTime startedAt,
      DateTime endedAt,
      IEnumerable<ITypeOperator> operators)
  {
      using var connection = new SqliteConnection(_connectionString);
      connection.Open();
      using var transaction = connection.BeginTransaction();
      connection.Execute(
          @"INSERT INTO intelligence_results (conversation_id, call_started, call_ended)
            VALUES (@ConversationId, @CallStarted, @CallEnded);",
          new
          {
              ConversationId = conversationId,
              CallStarted = startedAt.ToString("yyyy-MM-dd HH:mm:ss"),
              CallEnded = endedAt.ToString("yyyy-MM-dd HH:mm:ss"),
          },
          transaction);
      foreach (var op in operators)
      {
          var operatorType = op switch
          {
              Summary => "summary",
              Sentiment => "sentiment",
              ScriptAdherence => "script-adherence",
              _ => throw new InvalidOperationException($"Unknown operator type: {op.GetType().Name}"),
          };
          var operatorId = connection.ExecuteScalar<long>(
              @"INSERT INTO operators (conversation_id, operator_value, operator_type)
                VALUES (@ConversationId, @OperatorValue, @OperatorType);
                SELECT last_insert_rowid();",
              new
              {
                  ConversationId = conversationId,
                  OperatorValue = op.GetValue(),
                  OperatorType = operatorType,
              },
              transaction);
          if (op is ScriptAdherence adherence && adherence.Categories.Count > 0)
          {
              connection.Execute(
                  @"INSERT INTO operator_script_adherence_categories
                        (operator_id, category_name, category_met)
                    VALUES (@OperatorId, @CategoryName, @CategoryMet);",
                  adherence.Categories.Select(c => new
                  {
                      OperatorId = operatorId,
                      CategoryName = c.Name,
                      CategoryMet = c.Met ? 1 : 0,
                  }),
                  transaction);
          }
      }
      transaction.Commit();
  }
}

This code opens a connection, starts a transaction, and inserts into intelligence_results. For each operator, the switch expression maps the runtime type ( Summary, Sentiment, or ScriptAdherence) to the short string stored in the database ("summary", "sentiment", or "script-adherence"). Dapper's ExecuteScalar<long> runs the insert and returns the value of last_insert_rowid(), which is SQLite's built-in function for getting the ID of the most recently inserted row.

Note that this sample project is building off of the demo project OwlAir. If you're working from a different Conversation Relay sample, update the namespace accordingly in your own code.

Create the webhook controller

Now, create the controller that will handle incoming Conversation Intelligence webhooks. In the Controllers folder, create a file named IntelligenceResultsController.cs, and paste the code below into the file. 

using System.Text.Json;
using Microsoft.AspNetCore.Mvc;
using Twilio.Exceptions;
using Twilio.Rest.Conversations.V2;
namespace OwlAir;

[ApiController]
[Route("intelligence-results")]
public class IntelligenceResultsController : ControllerBase
{
   private readonly ILogger<IntelligenceResultsController> _logger;
   private readonly DatabaseService _databaseService;
   public IntelligenceResultsController(
       ILogger<IntelligenceResultsController> logger,
       DatabaseService databaseService)
   {
       _logger = logger;
       _databaseService = databaseService;
   }
   [HttpPost]
   public async Task<IActionResult> Post([FromBody] JsonElement body)
   {
       _logger.LogInformation("Intelligence webhook raw body: {Body}", body.GetRawText());
       var conversationId = TryString(body, "conversationId");
       var operators = new List<ITypeOperator>();
       if (body.ValueKind == JsonValueKind.Object
           && body.TryGetProperty("operatorResults", out var resultsProp)
           && resultsProp.ValueKind == JsonValueKind.Array)
       {
           foreach (var operatorData in resultsProp.EnumerateArray())
           {
               var op = ImportOperator(conversationId, operatorData);
               if (op is not null)
               {
                   operators.Add(op);
               }
           }
       }
       _logger.LogInformation(
           "Intelligence webhook parsed for {ConversationId} with {OperatorCount} operators",
           conversationId,
           operators.Count);
       DateTime callStarted = DateTime.UtcNow;
       DateTime callEnded = DateTime.UtcNow;
       try
       {
           var conversation = await ConversationResource.FetchAsync(pathId: conversationId);
           callStarted = conversation.CreatedAt ?? callStarted;
           callEnded = conversation.UpdatedAt ?? callEnded;
       }
       catch (TwilioException ex)
       {
           _logger.LogWarning(
               ex,
               "Conversation retrieval failed for {ConversationId}. Using current time for call_started/call_ended.",
               conversationId);
       }
       try
       {
           _databaseService.RecordCall(conversationId, callStarted, callEnded, operators);
           _logger.LogInformation("Persisted intelligence results for {ConversationId}", conversationId);
       }
       catch (Exception ex)
       {
           _logger.LogError(ex, "Failed to persist intelligence results for {ConversationId}", conversationId);
       }
       return Ok("Log Data Received");
   }
   private static ITypeOperator? ImportOperator(string conversationId, JsonElement data)
   {
       if (data.ValueKind != JsonValueKind.Object) return null;
       var displayName = TryObject(data, "operator", out var operatorObj)
           ? TryString(operatorObj, "displayName")
           : string.Empty;
       TryObject(data, "result", out var result);
       return displayName switch
       {
           "Summary" => new Summary(TryString(result, "text"), conversationId),
           "Sentiment" => new Sentiment(TryString(result, "label"), conversationId),
           "Script-Adherence" => new ScriptAdherence(
               TryObject(data, "parameters", out var parameters)
                   ? TryString(parameters, "script")
                   : string.Empty,
               conversationId,
               ReadCategories(result)),
           _ => null,
       };
   }
   private static IReadOnlyList<Category> ReadCategories(JsonElement result)
   {
       if (result.ValueKind != JsonValueKind.Object
           || !result.TryGetProperty("categories", out var categoriesProp)
           || categoriesProp.ValueKind != JsonValueKind.Array)
       {
           return Array.Empty<Category>();
       }
       var categories = new List<Category>();
       foreach (var category in categoriesProp.EnumerateArray())
       {
           if (category.ValueKind != JsonValueKind.Object) continue;
           var name = TryString(category, "category_key");
           var met = TryObject(category, "criteria", out var criteria)
               && TryString(criteria, "criteria_met") == "Succeeded";
           categories.Add(new Category(name, met));
       }
       return categories;
   }
   private static string TryString(JsonElement element, string propertyName)
   {
       if (element.ValueKind != JsonValueKind.Object) return string.Empty;
       if (!element.TryGetProperty(propertyName, out var prop)) return string.Empty;
       return prop.ValueKind == JsonValueKind.String ? prop.GetString() ?? string.Empty : string.Empty;
   }
   private static bool TryObject(JsonElement element, string propertyName, out JsonElement value)
   {
       value = default;
       if (element.ValueKind != JsonValueKind.Object) return false;
       if (!element.TryGetProperty(propertyName, out var prop)) return false;
       if (prop.ValueKind != JsonValueKind.Object) return false;
       value = prop;
       return true;
   }
}

The Post method is the heart of the class. It's called when Twilio sends a POST request to /intelligence-results at the end of a call. It:

1. Reads the raw JSON request body into a JsonElement, so we can walk the tree by hand without having to define a matching DTO for every nested shape Twilio might send.

2. Extracts conversationId and iterates over operatorResults, calling ImportOperator for each one.

3. ImportOperator looks at each operator's displayName and constructs the appropriate typed record: a Summary, Sentiment, or ScriptAdherence. Unknown display names return null and are skipped.

4. Once operators are collected, it fetches the conversation from the Conversations v2 API to get the start (CreatedAt) and end (UpdatedAt) timestamps — the webhook payload doesn't include these.

5. Finally, it passes everything to DatabaseService.RecordCall for persistence.

Register services and initialize database

Open Program.cs and replace its contents with: 

using System.Net.WebSockets;
using dotenv.net;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Data.Sqlite;
using OwlAir;
using Twilio;

DotEnv.Load();

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddControllers();
builder.Services.AddSingleton<DatabaseService>();
builder.Services.AddSingleton<OpenAiService>();
var app = builder.Build();
TwilioClient.Init(
   Environment.GetEnvironmentVariable("TWILIO_ACCOUNT_SID"),
   Environment.GetEnvironmentVariable("TWILIO_AUTH_TOKEN"));
var connectionString = Environment.GetEnvironmentVariable("SQLITE_CONNECTION_STRING")
   ?? "Data Source=data/database/database.sqlite3";
using (var connection = new SqliteConnection(connectionString))
{
   connection.Open();
   var schemaPath = Path.Combine(AppContext.BaseDirectory, "data", "database", "dump.sql");
   var schemaSql = File.ReadAllText(schemaPath);
   using var command = connection.CreateCommand();
   command.CommandText = schemaSql;
   command.ExecuteNonQuery();
}
app.UseWebSockets();
app.MapPost("/twiml", ([FromServices] IConfiguration _) =>
{
   var domain = Environment.GetEnvironmentVariable("DOMAIN")
       ?? throw new InvalidOperationException("DOMAIN env var is not set.");
   var host = domain.Replace("https://", "").Replace("http://", "").TrimEnd('/');
   var twiml = $"""
       <?xml version="1.0" encoding="UTF-8"?>
       <Response>
         <Connect>
           <ConversationRelay url="wss://{host}/ws" welcomeGreeting="Thanks for calling Owl Air! I'm Hoot. I can help with flight status, baggage policy, loyalty points, or booking changes. Which of those can I help you with?" />
         </Connect>
       </Response>
       """;
   return Results.Content(twiml, "application/xml");
});
app.Map("/ws", async (HttpContext context, OpenAiService openAi) =>
{
   if (!context.WebSockets.IsWebSocketRequest)
   {
       context.Response.StatusCode = 400;
       return;
   }
   using var ws = await context.WebSockets.AcceptWebSocketAsync();
   await ConversationRelayHandler.HandleAsync(ws, openAi);
});
app.MapControllers();
app.Run();

Note that this code builds off of the existing OwlAir demo, so your namespace and introductory prompt may vary if you are using another starting demo project. You may also be using appsettings.json instead of a .env file so check your personal configuration.

The addition to your code helps connect with the database and write to your conversation intelligence:

1. DatabaseService is registered with the DI container. Any controller that declares a DatabaseService constructor parameter will be handed the same singleton instance.

2. TwilioClient.Init is called once at startup, with the account SID and auth token from configuration.

3. The SQL schema runs on startup. This creates the database.sqlite3 file and its tables the first time the app runs, so you don't need to invoke the SQLite command-line shell manually.

Add this block to ConversationRelayDemo.csproj so the schema file gets copied to the build output directory: 

<ItemGroup>
    <Content Include="data\database\dump.sql">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </Content>
  </ItemGroup>

Step 6: Start the application

With the code now complete, start the application by running the command below in your terminal, after replacing the four environment variable placeholders with their respective values.

Set your Twilio credentials and OpenAI API key as environment variables. In your .env file, it will look like this, with the placeholders replaced with correct values. 

TWILIO_ACCOUNT_SID=Twilio Account SID
TWILIO_AUTH_TOKEN=Twilio Auth Token
DOMAIN=Your ngrok URI 
OPENAI_API_KEY=Your OpenAI API key

The app will start on http://localhost:5000 by default. Expose it with ngrok by typing: 

ngrok http 5000

Add the resulting ngrok URI to your DOMAIN variable above.

Now, start (or restart) your server, by typing: 

dotnet run

You will see output similar to the example below, after the application starts. 

info: Microsoft.Hosting.Lifetime[14]
      Now listening on: http://localhost:5000
info: Microsoft.Hosting.Lifetime[0]
      Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
      Hosting environment: Production
info: Microsoft.Hosting.Lifetime[0]

To make sure your call connects, wire your Twilio call as you may have done previously, by adding the ngrok URL, followed by /twiml to your Twilio dashboard as the A Call Comes In webhook connection. Look for it in the dashboard under Numbers and Senders > Overview > Configuration Details > Voice.

Place a call to your Twilio number. After the call ends and Twilio Conversation Intelligence has finished processing, you should see a new row in intelligence_results and its associated rows in operators and operator_script_adherence_categories.

Test that the app works as expected

With the application running, call your Twilio phone number. You should hear Hoot's greeting within a second or two of the call connecting:

> "Thanks for calling Owl Air! I'm Hoot. I can help with flight status, baggage policy, loyalty points, or booking changes. Which of those can I help you with?"

Then, like when you tested the first version of the app, try a few test questions to verify the full flow is working:

  • "What's the baggage policy?": Hoot should describe carry-on and checked bag rules in natural spoken language.
  • "How do loyalty points work?": Hoot should explain the earn and redemption rates.
  • "Can I change my flight?": Hoot should give the change fee policy, with amounts spelled out in words.
If you hear an error message on the call or see an error message in your terminal logs, check the error code against the Conversation Relay error code reference in the Twilio documentation.

After the call finishes, using your database tool of choice, have a look at the records in the application's database. There, you should see a summary of the conversation, along with the related operator information. You can also look on your Twilio dashboard and see records under your Intelligence Configuration.

Conclusion

You've now learned how to use Conversation Intelligence, Conversation Orchestrator, and Conversation Memory, as well as the Conversations API (V2), to retrieve a short summary of each call and an analysis of the caller's sentiment, and persist the information to a SQLite database, so that you can make use of it later. What's more, you also know whether your agent followed the guidelines you set for it.

But don't stop there! Now that the application can store conversation information, why not add a route for viewing a summary of all stored conversations, and one for viewing individual conversation details?

Then, I strongly encourage you to learn more about Conversation Intelligence, Conversation Orchestrator, and Conversation Memory, as well as the Conversations API (V2).

Amanda Lange is a .NET Engineer of Technical Content. She is here to teach how to create great things using C# and .NET programming. She can be reached at amlange [ at] twilio.com.

Related Posts

Related Resources

A newspaper article

Twilio Docs

From APIs to SDKs to sample apps

API reference documentation, SDKs, helper libraries, quickstarts, and tutorials for your language and platform.
An Open book

Resource Center

The latest ebooks, industry reports, and webinars

Learn from customer engagement experts to improve your own communication.
User group reactions

Ahoy

Twilio's developer community hub

Best practices, code samples, and inspiration to build communications and digital engagement experiences.