Skip to contentSkip to navigationSkip to topbar
Page toolsOn this page
Looking for more inspiration?Visit the

Add TAC to your agent


This guide walks you through connecting your AI agent to Twilio's platform using the TAC SDK. You'll configure channels, handle messages, retrieve memory context, define custom tools, and search knowledge bases — all wired together in a single integration flow.

If you haven't tried TAC yet, start with the Quickstart to get a working application first. To add a single capability to an existing TAC application, see the how-to guides in the sidebar.


Prerequisites

prerequisites page anchor

Before you begin, make sure you have:

  • A Twilio account with API credentials (Account SID, Auth Token, API Key, and API Secret)
  • A Twilio phone number configured for SMS and/or Voice
  • A Conversation Configuration created through the Console or REST API
  • Python 3.10+ or Node.js 22.13.0+

PythonTypeScript
pip install twilio-agent-connect[server]

Configure environment variables

configure-environment-variables page anchor

Create a .env file in your project root with your Twilio credentials.

PythonTypeScript
1
TWILIO_ACCOUNT_SID=your_account_sid
2
TWILIO_AUTH_TOKEN=your_auth_token
3
TWILIO_API_KEY=your_api_key_sid
4
TWILIO_API_SECRET=your_api_key_secret
5
TWILIO_PHONE_NUMBER=+1234567890
6
TWILIO_CONVERSATION_CONFIGURATION_ID=your_configuration_id
7
TWILIO_VOICE_PUBLIC_DOMAIN=your-ngrok-domain.ngrok-free.app

For Voice, set TWILIO_VOICE_PUBLIC_DOMAIN to your ngrok domain without https://.


Create a TAC instance that loads configuration from your environment variables.

PythonTypeScript
1
from dotenv import load_dotenv
2
from tac import TAC, TACConfig
3
4
load_dotenv()
5
tac = TAC(config=TACConfig.from_env())

Configure channels for each communication method you want to support. Set memory_mode to "always" (Python) or memoryMode to "always" (TypeScript) if you want TAC to retrieve memory context with each incoming message. All channels share the same on_message_ready callback, so your agent handles every channel with a single function.

PythonTypeScript
1
from tac.channels.voice import VoiceChannel, VoiceChannelConfig
2
from tac.channels.sms import SMSChannel, SMSChannelConfig
3
from tac.channels.whatsapp import WhatsAppChannel, WhatsAppChannelConfig
4
from tac.channels.rcs import RCSChannel, RCSChannelConfig
5
6
voice_channel = VoiceChannel(tac, config=VoiceChannelConfig(memory_mode="always"))
7
sms_channel = SMSChannel(tac, config=SMSChannelConfig(memory_mode="always"))
8
whatsapp_channel = WhatsAppChannel(tac, config=WhatsAppChannelConfig(memory_mode="always"))
9
rcs_channel = RCSChannel(tac, config=RCSChannelConfig(memory_mode="always"))

For more details on channel configuration, see Channels.


Handle incoming messages

handle-incoming-messages page anchor

Register a callback that TAC calls when a user message is ready for processing. The callback receives the user's message, a ConversationSession with context, and an optional memory response. Return the response string and TAC automatically sends it through the correct channel.

PythonTypeScript
1
from tac.models.session import ConversationSession
2
from tac.models.tac import TACMemoryResponse
3
4
conversation_history: dict[str, list] = {}
5
6
async def handle_message_ready(
7
message: str,
8
context: ConversationSession,
9
memory: TACMemoryResponse | None,
10
) -> str | None:
11
conv_id = context.conversation_id
12
13
if conv_id not in conversation_history:
14
conversation_history[conv_id] = []
15
16
conversation_history[conv_id].append({"role": "user", "content": message})
17
18
# Process with your LLM (see following sections)
19
llm_response = await generate_response(conversation_history[conv_id])
20
21
conversation_history[conv_id].append({"role": "assistant", "content": llm_response})
22
23
return llm_response
24
25
tac.on_message_ready(handle_message_ready)

Enhance your system prompt with Conversation Memory

enhance-your-system-prompt-with-conversation-memory page anchor

Define your agent's role and behavior through a system prompt. You can combine a static prompt with dynamic memory context.

PythonTypeScript

The Python SDK provides MemoryPromptBuilder to format memory data into a prompt string:

1
from tac.adapters.prompt_builder import MemoryPromptBuilder
2
3
SYSTEM_PROMPT = "You are a helpful customer service agent. Be concise and friendly."
4
5
async def handle_message_ready(
6
message: str,
7
context: ConversationSession,
8
memory: TACMemoryResponse | None,
9
) -> str | None:
10
conv_id = context.conversation_id
11
12
if conv_id not in conversation_history:
13
conversation_history[conv_id] = []
14
15
conversation_history[conv_id].append({"role": "user", "content": message})
16
17
# Build memory context from profile traits and conversation history
18
memory_context = MemoryPromptBuilder.build(
19
memory_response=memory,
20
context=context,
21
)
22
23
# Combine your prompt with memory context
24
if memory_context:
25
system_prompt = f"{SYSTEM_PROMPT}\n\n{memory_context}"
26
else:
27
system_prompt = SYSTEM_PROMPT
28
29
# Pass to your LLM as the system message
30
messages = [
31
{"role": "system", "content": system_prompt},
32
*conversation_history[conv_id],
33
]
34
35
response = await openai_client.chat.completions.create(
36
model="gpt-4o-mini",
37
messages=messages,
38
)
39
40
llm_response = response.choices[0].message.content
41
conversation_history[conv_id].append({"role": "assistant", "content": llm_response})
42
43
return llm_response

This approach works with any LLM provider. For OpenAI specifically, the Python SDK provides a built-in adapter that retrieves memory context automatically.

PythonTypeScript
1
from openai import AsyncOpenAI
2
from tac.adapters.openai import with_tac_memory
3
4
openai_client = AsyncOpenAI()
5
6
async def handle_message_ready(
7
message: str,
8
context: ConversationSession,
9
memory: TACMemoryResponse | None,
10
) -> str | None:
11
# Wrap the OpenAI client — memory context is added to your messages before each LLM call
12
client = with_tac_memory(openai_client, memory, context)
13
14
response = await client.chat.completions.create(
15
model="gpt-4o-mini",
16
messages=conversation_history[context.conversation_id],
17
)
18
19
llm_response = response.choices[0].message.content
20
21
return llm_response

Use Enterprise Knowledge bases to give your agent grounded responses.

PythonTypeScript
1
from tac.tools import create_knowledge_tool
2
3
knowledge_tool = await create_knowledge_tool(
4
knowledge_client=tac.knowledge_client,
5
knowledge_base_id="know_knowledgebase_xxxxx",
6
)
7
8
# Add to your LLM's tool list alongside custom tools
9
tools = [check_order_status, knowledge_tool]

The previous examples wait for the full LLM response before sending it to the caller. For Voice conversations, you can stream tokens from your LLM as they're generated so the caller hears the response sooner. Pass an async generator to send_response instead of a complete string.

PythonTypeScript
1
from collections.abc import AsyncGenerator
2
3
async def handle_message_ready(
4
message: str,
5
context: ConversationSession,
6
memory: TACMemoryResponse | None,
7
) -> str | None:
8
conv_id = context.conversation_id
9
10
if conv_id not in conversation_history:
11
conversation_history[conv_id] = []
12
13
conversation_history[conv_id].append({"role": "user", "content": message})
14
15
async def stream_tokens() -> AsyncGenerator[str, None]:
16
response_tokens = []
17
18
stream = await openai_client.chat.completions.create(
19
model="gpt-4o-mini",
20
messages=conversation_history[conv_id],
21
stream=True,
22
)
23
24
async for chunk in stream:
25
if chunk.choices and chunk.choices[0].delta.content:
26
token = chunk.choices[0].delta.content
27
response_tokens.append(token)
28
yield token
29
30
full_response = "".join(response_tokens)
31
conversation_history[conv_id].append({"role": "assistant", "content": full_response})
32
33
if context.channel == "voice":
34
await voice_channel.send_response(conv_id, stream_tokens())
35
elif context.channel == "sms":
36
llm_response = await generate_response(conversation_history[conv_id])
37
conversation_history[conv_id].append({"role": "assistant", "content": llm_response})
38
await sms_channel.send_response(conv_id, llm_response)
39
40
tac.on_message_ready(handle_message_ready)

TAC includes a built-in server that sets up webhook and WebSocket routes for your configured channels.

PythonTypeScript
1
from tac.server import TACFastAPIServer
2
3
if __name__ == "__main__":
4
server = TACFastAPIServer(
5
tac=tac,
6
voice_channel=voice_channel,
7
messaging_channels=[sms_channel],
8
)
9
server.start()

The server registers routes based on the channels you configure:

Voice routes (when voice_channel is provided):

  • POST /twiml — Incoming call handler
  • WebSocket /ws — Streaming via Conversation Relay
  • POST /conversation-relay-callback — Completion callback

Messaging route (when messaging_channels is provided):

  • POST /webhook — Webhook for SMS and other messaging channels

Conversation Intelligence route (when cintel_webhook_path is set on TACServerConfig):

  • POST <cintel_webhook_path> — Conversation Intelligence events (for example, /ci-webhook). Disabled by default.