Core concepts
This guide describes the core concepts of Twilio Agent Connect (TAC) and explains how its components interact to support conversational AI applications.
TAC integrates with Twilio communication channels — Voice, SMS, WhatsApp, RCS, and Chat — to connect users to your agent. All channels share a unified callback interface, so you handle messages from any channel with the same function. For channel configuration and examples, see Channels.
Conversations represent ongoing interactions between participants across communication channels. TAC connects to Twilio services to retrieve conversation data and context, making it available to your agent through ConversationSession objects.
TAC retrieves context from Twilio services and passes it to your callbacks through the ConversationSession object. The callback also receives a memory parameter containing observations, summaries, and communication history (when Conversation Memory is configured).
1async def handle_message_ready(2message: str,3context: ConversationSession,4memory: TACMemoryResponse | None,5) -> str | None:6# Access conversation context7print(f"Conversation ID: {context.conversation_id}")8print(f"Profile ID: {context.profile_id}")9print(f"Channel: {context.channel}") # "voice", "sms", "whatsapp", "rcs", or "chat"10print(f"Started at: {context.started_at}")1112# Access profile data (if Conversation Memory enabled)13if context.profile:14print(f"User traits: {context.profile.traits}")1516# Access memory (observations, summaries, communications)17if memory:18print(f"Observations: {memory.observations}")
TAC integrates with Conversation Memory to provide persistent context across conversations.
TAC uses a dual-mode retrieval strategy with automatic fallback:
- Memory API mode (when Conversation Memory is configured): TAC looks up the customer profile, then retrieves observations, summaries, and communication history from the Memory API. This provides the richest context for your agent.
- Fallback mode (when Conversation Memory is not configured): TAC retrieves basic communication history directly from the Conversation Orchestrator API. Observations and summaries are not available in this mode, but your agent still has access to recent messages.
This fallback is automatic — you don't need to handle it in your code. Your on_message_ready callback always receives a TACMemoryResponse object, regardless of which mode is active. If the Memory API is unavailable at runtime, TAC logs a warning and falls back to the Conversation Orchestrator API for that request.
Conversation Memory integration is optional. When configured, TAC retrieves memory for each session:
1async def handle_message_ready(2message: str,3context: ConversationSession,4memory: TACMemoryResponse | None,5) -> str | None:6if memory:7# Access observations, summaries, communications8for obs in memory.observations:9print(obs.content)
You can access user data through profile traits:
1profile = context.profile23if profile and profile.traits:4# Traits are organized by groups5contact_group = profile.traits.get("Contact", {})6email = contact_group.get("email")7phone = contact_group.get("phone")89preferences_group = profile.traits.get("Preferences", {})10language = preferences_group.get("language", "en")
Traits from Conversation Memory are organized into logical groups (for example, "Contact", "Preferences"). The data structure comes pre-organized from Conversation Memory, and you can access it through context.profile.traits as shown in the example above.
To update profiles, use the Conversation Memory Console or the Conversation Memory API. TAC can only read profiles.
TAC includes a tool system that lets you define functions your agent can call during conversations. Tools follow the standard function-calling pattern used by LLM providers: you define a name, description, and JSON schema, and TAC formats them for your provider.
TAC includes built-in tools for common operations:
- Knowledge search: Query Enterprise Knowledge bases and return relevant results to your agent.
- Escalation: Transfer the conversation to a human agent in Twilio Flex.
- Memory retrieval: Search and retrieve customer observations, summaries, and conversation history.
You can also define custom tools with the @function_tool decorator (Python) or defineTool function (TypeScript). In Python, custom tools can receive the ConversationSession as a parameter, giving them access to profile data, conversation state, and channel information.
TAC integrates with Enterprise Knowledge to give your agent access to structured knowledge bases. When a user asks a question, your agent can search relevant knowledge bases and use the results to generate grounded, accurate responses.
Knowledge search is exposed as a tool that you add to your LLM's tool list. The tool handles the search request, queries the knowledge base, and returns matching results that your agent can reference in its response.
TAC supports transferring conversations from your AI agent to a human agent when the situation requires human intervention. Escalation preserves the conversation context so the human agent has full visibility into the interaction history.
The handoff flow uses the built-in create_studio_handoff_tool, which routes conversations to a Twilio Studio Flow. Set TWILIO_STUDIO_HANDOFF_FLOW_SID in your environment, create the tool with the TAC instance and session, and add it to your LLM's tools. When the LLM calls the tool:
- Voice: The tool stores the handoff payload on the session. After the LLM's final response plays, the Voice channel sends a WebSocket
endmessage that routes the call to your Studio Flow. - Digital: The tool
POSTs the handoff payload directly to the Studio Flow Executions API.
For a complete guide, see Escalate to a human agent.
TAC supports two deployment modes to accommodate different use cases:
| Full Orchestrator mode (recommended) | ConversationRelay-only mode (voice-only) | |
|---|---|---|
| Requires | Conversation configuration with linked memory store | None |
| Channels | Voice, SMS, RCS, WhatsApp, Chat | Voice only |
| Memory | Automatic retrieval with observations, summaries, and traits | Not supported |
| Intelligence | Full support for real-time and post-conversation analysis | Not supported |
| Use when | Production agents that need customer memory, multi-channel support, or conversation intelligence | Simple Voice IVR systems, quick prototypes, or cost-optimized deployments without persistent memory |
1if tac.is_orchestrator_enabled():2# Full Orchestrator mode - memory available3memory = await tac.retrieve_memory(context)4else:5# ConversationRelay-only mode - handle without memory6pass
TAC does not run your AI logic. You provide the LLM application and control how it generates responses. TAC handles the infrastructure — channels, memory, tools, and session management — while you provide the agent.
TAC does not make decisions about what your agent should do. You define and control:
- When to escalate to human agents.
- How to interpret user intents.
- What actions to take based on conversation state.
- Custom validation or processing rules.
- Quickstart: Build and test a TAC application from scratch.
- Add TAC to your agent: Connect your agent and configure channels, tools, knowledge, and memory.
- Channels: Learn about channel configuration.