Introducing Twilio AI Assistants: a platform to build customer-aware autonomous assistants

Introducing Twilio AI Assistants

AI Assistants is our first project launched under the Twilio Alpha program. In the spirit of experimentation and exploration, we want to share what we've been up to and are excited to get this into your hands as a Developer Preview.

AI, and specifically Large Language Models (LLMs), have been evolving rapidly in recent years. We've seen a rise in discussions surrounding autonomous agents, systems that can answer complex questions through reasoning, pose relevant follow-up questions, and perform tasks without the need for rigid decision trees or intent-based training.

However, building progressively more complex systems on a path to fully autonomous agents can be complex – and even overwhelming. From dealing with handling memory to provide the AI conversation context, to over-designing and implementing complex Retrieval-Augment Generation (RAG) pipelines to provide additional domain knowledge to your system, to figuring out how to stitch everything together with your different systems and channels, to measuring LLM hallucinations and prompt injection detection, you quickly find yourself down a rabbit hole. We know because that's where we found ourselves.

Today, as a project within the Twilio Alpha program, we are excited to announce our Developer Preview for Twilio AI Assistants: a platform to build customer-aware autonomous agents that we call "Assistants".

What is Twilio AI Assistants?

Twilio AI Assistants is an opinionated framework to build and host Assistants for customer-facing use cases.

At the core of Twilio AI Assistants resides a distinctive customer-aware approach facilitated by Customer Memory. This feature will not only automatically handle the conversation history between the user and the Assistant, but behind the scenes, use the AI Perception Engine and AI Personalization Engine to create a customer profile that you can augment and reference using Segment.

Beyond its customer-centric capabilities, AI Assistants can leverage Tools that you define to make API requests to interact with other systems and access Knowledge sources to tap domain expertise without you having to manage your own RAG pipeline. If an Assistant can't answer a question or you don't want it to handle certain topics, AI Assistants can automatically hand over the conversation to a human agent in the Twilio Flex contact center.

Moreover, the Assistants are equipped with out-of-the-box Guardrails - a set of prompt injection detection rules, content moderation, and hallucination measurements (coming soon) that act as a safeguard, ensuring the trust and observability of each interaction.

AI Assistants are built on top of various state-of-the-art Large Language Models, including models from OpenAI, amongst others. We at Twilio are constantly exploring how we can use the latest advancements in AI so you can focus on building great customer experiences rather than subscribing to AI newsletters and keeping up with the blistering pace of AI advancements.

Twilio AI Assistants Use Cases

During conversations with customers, three primary use cases stood out for AI Assistants:

1) Support Automation: AI Assistants offer a robust solution for automating service requests, alleviating the burden on customer support teams. Through seamless integration with platforms like Flex, customers can initially engage with AI Assistants and smoothly transition to human agents if necessary.

2) Pre-sales, Lead Qualification, and Growth Marketing: Leveraging the conversational capabilities of AI Assistants, businesses can streamline pre-sales interactions, lead qualification processes, and growth marketing strategies. Imagine a scenario where AI Assistants replace traditional form fills with dynamic conversational experiences, enabling the intelligent gathering and analysis of customer data.

3) Self-Service Product Assistance: AI Assistants can prove invaluable in guiding customers through product adoption phases, enhancing user experience, and driving conversion rates. Utilizing the Customer Memory of an Assistant enables personalized onboarding experiences for each user's journey. Gathering crucial data during onboarding can then be utilized for customized marketing strategies in the future.

While these use cases were notable in our initial discussions, we're confident that you'll uncover even more. AI Assistants is designed for versatility, and we're excited to witness the innovations you'll create!

Building an Assistant

Initially, you can configure your Assistant through our configuration UI. There, you can give your Assistant a personality, turn on Customer Memory, and configure its capabilities through Tools and Knowledge. We plan to add the ability to deploy via CLI or API later in our Developer Preview so you can manage your Assistant from your own workflows.

Customer Memory

Twilio AI Assistants is built on top of what we call Customer Memory.

AI Assistants will automatically keep track of conversation history and attempt to learn about a customer as they converse with them. Powered by the AI Perception Engine and AI Personalization Engine Assistants can reference any data you have in a Segment Profile, but also build a customer profile even if you are not using Segment yet.

And since Customer Memory is built on top of Segment, you'll be able to use the data outside of AI Assistants. Whether you use it in your next marketing campaign, help your contact center agents more effectively answer questions using Unified Profiles, or pipe AI Assistants' data into your own data warehouse, every conversation with AI Assistants helps you better understand your users.

Tools

With Tools, your Assistant can reach out to other systems to retrieve additional data or perform actions. AI Assistants does this by making HTTP requests to the API endpoints you provide. All you have to do is give AI Assistants a URL, the format of the data you expect AI Assistants to send, and a description of when you want AI Assistants to use the Tool, and your AI Assistant will determine when to use the Tool on its own.

Knowledge

Managing domain knowledge for LLMs using RAG pipelines is important to reduce the risk of hallucinations, control what data the AI has access to, and be confident your data is always up-to-date.

Building a RAG pipeline is hard, though! From loading the data to picking the right embedding and chunking strategies to actually wiring up your pipeline, each step requires careful decision making.

AI Assistants come with a RAG system out of the box. All you need to do is give us the websites and files you want to process and a description of when you want AI Assistants to use them, and we will take care of the rest.

Security

AI Assistants comes with a set of default safety features such as:

• Prompt injection protection: to reduce the risk of malicious messages manipulating the behavior of your assistant
• Content moderation: to reduce the risk of inappropriate content sent to or generated by your Assistant
• Observability: to monitor the performance, behavior, and health of your Assistant with guardrail metrics
• Type-safe Tool input validation: to avoid AI Assistants sending invalid data to your APIs
• Webhook signatures for API requests: to make sure you can validate requests coming from AI Assistants and control what actions can be taken

We plan to continue our exploration of the most critical safety mechanisms and guardrails in the coming months.

Connecting your Assistant

Within our AI Assistants console, you can use the simulator to test your Assistant as you build. Once you are ready to introduce your Assistant to the world, it’s compatible with a wide range of channels – SMS, WhatsApp, web chat, mobile – with more to come.

SMS / WhatsApp

AI Assistants is compatible with Twilio Programmable Messaging both for SMS and WhatsApp. Check out our documentation to learn more about how to integrate your AI Assistant with Twilio Programmable Messaging.

Web Chat

If you are using React, you can use our SDK to build a web chat for your Assistant in a few lines of code:

If you want to customize the UI or use other JavaScript frameworks, you'll be able to use the individual building blocks from our SDK.

iOS / Android

AI Assistants integrates directly with Twilio Conversations. By using the Twilio Conversations SDKs for iOS and Android, you'll be able to connect your AI Assistant to your existing iOS and Android applications.

Custom Channels

We wouldn't be Twilio if we didn't give you the opportunity to wire up whatever you can imagine. You can also use our API directly:

The API will also be exposed throughout our various helper libraries across Python, Node.js, Go, Ruby, Java, .NET, and PHP.

What's Next

Starting today, we are opening up our Developer Preview waitlist. We want to make sure we build the best possible experience for you all, and as a result we'll be onboarding developers on a rolling basis over the coming months. We can’t wait to get your feedback and have you help shape AI Assistants.

We still have many more things planned for AI Assistants, and can’t wait to share more with you.

If you want to be part of the Developer Preview, you can follow along with Twilio Alpha and sign up for our waitlist today!

Rachel Baskin is a Principal Project Manager on the Emerging Technology and Innovation team. Her passions are solving customer pain points and integrating product development with analytics for data-driven decision-making. She also loves playing Settlers of Catan and watching Survivor. Connect with Rachel at rbaskin [at] twilio.com.