Skip to contentSkip to navigationSkip to topbar
Rate this page:
On this page

AI Assistants Overview


Twilio Alpha - Developer Preview

AI Assistants is a Twilio Alpha(link takes you to an external page) project that's in Developer Preview. To request access, please join the waitlist(link takes you to an external page).

View the current limitations for details about feature limits during developer preview.

Twilio AI Assistants are an opinionated framework to build and host conversational AI Assistants for your customer-facing use cases.

Twilio AI Assistant core functionality consists of:

  • Customer Memory: During conversations with customers, Assistants create a customer profile that you can augment, reference, and build upon in future interactions using Twilio Segment(link takes you to an external page) .
  • Tools: Assistants can make API requests to interact with other systems using interfaces that you define.
  • Knowledge sources: Your Assistant can use unstructured knowledge sources such as websites and databases to provide additional context for answering users' questions.
  • Human handoff: If an Assistant can't answer a question or there are topics you don't want it to handle, the Assistant can automatically hand over the conversation to a human agent in the Twilio Flex contact center .
  • Guardrails and monitoring: Assistants have a set of prompt injection detection rules, content moderation, and hallucination measurements that act as a safeguard, ensuring the trust and observability of each interaction.

AI Assistants are built on top of various Large Language Models (LLMs), including models from OpenAI amongst others.

Read on to learn about the AI Assistant features, or check out the Quickstart to see how to build your first Assistant.

Assistant features

assistant-features page anchor

Base configuration

base-configuration page anchor

Your AI Assistant's base configuration consists of its name, base model, and personality prompt. You can also optionally provide "memory" through Twilio's CustomerAI(link takes you to an external page).

Below are the base configuration components:

  • Name: This is currently only used for you to identify your Assistant in the Console UI. If you want your Assistant to be able to use this name to refer to itself, make sure you add that to its personality prompt.
  • Base model: Currently only OpenAI GPT-4. In the future, you will be able to use different LLM models and modify parts of our model "constellation" to use the best model for particular parts of the Assistant's flow.
  • Personality prompt: This defines the purpose of your Assistant and can be used to define the tone and voice of your Assistant or anything else you want it to keep in mind. This should provide a high-level overview of the purpose and tone; don't be too prescriptive here about what you want your Assistant to do. Best practices in crafting your prompt include:
    • Be concise and clear in your prompts, avoiding ambiguity.
    • Align the tone and style of prompts with your brand's voice and the Assistant's persona.
    • Encourage interaction by asking open-ended questions and providing options for further exploration.
  • Customer AI: Assistants are built on top of Twilio's CustomerAI. You can choose to integrate your Assistant with Segment so that your Assistant is "customer aware".

Tools are APIs that your Assistant can use to accomplish tasks and solve problems.

Tools should be standalone APIs (similar to webhooks) that are purpose-built for a task. An Assistant chooses from its available Tools when answering questions.

Tools differ from Knowledge in that Tools accomplish specific tasks (for example, looking up the weather or finding directions to a location), whereas Knowledge entries are unstructured data sources that the AI Assistant can reference to ask additional questions (such as your company's policies or documentation).

Tool attributes

tool-attributes page anchor

A Tool definition consists out of the following attributes:

NameThe name for your Tool. This needs to be unique as it will be used by your Assistant internally to reference this tool. You can think about it like a "function name".Google Maps
DescriptionThis description will be the primary way your Assistant will determine when to use this tool. If you see your tool not being used as you expected this is the best place to start iterating on it.Use this to fetch information about a location from Google Maps.
InputThis field describes what information you want the Assistant to pass to the webhook. The format is similar to TypeScript types to define the various fields you want to have sent.{ name: string; }
HTTP MethodSpecifies whether you want the API request to be an HTTP GET or POST request.GET
Webhook URLThe URL of the API webhook you want it to trigger.
Verified EndpointWhether you want the AI Assistant to perform a 2FA verification check via SMS. Requires that your Segment profile contains a phone or phone_number trait.[unchecked]
Advanced SettingsUnder advanced settings you can specify if you want the result of a tool to be used as Final Response. That will result in the Assistant taking the output of the HTTP request and directly responding to the customer. This is useful if your Tool returns a specific message you want to communicate with the customer. However, this means the Tool cannot be used as part of a series of Tools.

The Webhooks will automatically pass the following information as HTTP headers to the request:

X-Session-IdThe session ID of your conversation. You can use this for example to identify if the same tool was called multiple times from the same conversation.
X-IdentityThe Segment Identity that is used for your Assistant interaction. If you use the Assistant Simulator, this value will be the same as what you put into the User Identity

Security and permissions

security-and-permissions page anchor

Right now, the only authentication mode supported for Tools is OAuth 2 with password type grants. If you need a different mode of authentication to interact with an API, it's recommended to create a proxy API endpoint that Assistants can trigger. For example, rather than giving direct access to the Google Maps API, you should create your own endpoint that calls the Google Maps API and give your Assistant access to that endpoint instead.

If you want to restrict actions from certain users interacting with an Assistant, or if you want to understand on whose behalf the Assistant is trying to trigger a tool, check the X-Identity header.

With Knowledge, you can give your Assistant access to unstructured data such as knowledge bases, policies, and documentation.

The following knowledge types are currently available to use with your Assistant:

  • Web: Provide the link to a publicly accessible website. During upload, the Agent will crawl the website. You can specify a crawling depth up to 3 levels down. During developer preview, there is a 15 page maximum for websites. Pages outside of the specified domain will not be crawled.
  • Text: You can specify a manual chunk of text and upload that directly.
  • Files: Currently supports CSV, JSON-L, and Markdown files.
  • Vector databases: You can bring your own vector database. Right now the only valid option is a Pinecone database. If you do this, make sure you use a dimension of 1536.

After you've defined Knowledge, the Assistant will determine if a Knowledge entry might be helpful to generate an answer to a customer's question. This works similarly to how Tools are used. However, instead of generating structured data to interface with the Tool, the Assistant will generate a question and pull relevant information from the unstructured data that you uploaded.

Knowledge entry attributes

knowledge-entry-attributes page anchor

A Knowledge Entry consists out of the following attributes:

NameThe name for your Knowledge. This needs to be unique as it will be used by your Assistant internally to reference this knowledge entry.FAQ
DescriptionThis description is the primary way your Assistant determines when to use this entry and how to phrase a question to it. If your Assistant isn't using the entry as you expected, this is the best place to start iterating.Use this to find answers to frequently asked questions about Twilio.
ConfigurationThis configuration will differ depending on the Knowledge type.
Advanced SettingsSpecify if you want the result from querying this Knowledge to be used as the final response to the customer. If you turn this on, the Assistant won't be able to combine information from multiple knowledge entries.

Connect your AI Assistant to your preferred channels

Every Assistant comes with a REST API that you can use to trigger a message. Aside from a Body you can pass in a SessionId of your choice (which will be used to maintain message history) and an Identity.

Send a message using curl

send-a-message-using-curl page anchor

curl -X POST -H 'Content-Type:application/json' \
_10$ASSISTANT_SID/Messages \
-d '{"Identity":"user_id:example", "SessionId": "demo", "Body":"Ahoy there"}' \

The Simulator allows you to interact with your AI Assistant via the Twilio Console for testing, demo, and debugging purposes. You can find the Simulator at the top right of your Assistant Configuration screen in the Console.

You can define an identity and create a new Simulator session to chat with your Assistant. The view will also show you the output of the Tools the Assistant uses. Each new Simulator session starts a new conversation history.

You can use the Simulator to test a wide range of queries, including edge cases, to evaluate the Assistant's responses and adaptability, and to simulate different user personas and scenarios.

While AI Assistants are in developer preview, there are feature limitations and system limits in place.

Assistant configuration limits:

  • Five Tools, with a maximum of 25 input properties
  • Five Knowledge Sources
    • Website sources can have a maximum of 15 pages
  • Names for Tools and Knowledge Sources can be no longer than six tokens (~30 characters)
  • Descriptions for Tools and Knowledge Sources can be no longer than 60 tokens (~300 characters)
  • Personality prompts can be no longer than 400 tokens (~2000 characters)

Assistant usage:

  • 1000 messages maximum per Twilio account

If you want to exceed these limitations, please reach out to

Rate this page: