Level up your Twilio API skills in TwilioQuest, an educational game for Mac, Windows, and Linux. Download Now


Rate this page:

Thanks for rating this page!

We are always striving to improve our documentation quality, and your feedback is valuable to us. How could this documentation serve you better?

Twilio Autopilot Console Quickstart

With Twilio's Autopilot, you can build messaging bots and virtual voice assistants powered by Twilio's own Natural Language Processing engine.

In this quickstart, we're going to build a simple voice assistant that can tell a caller a joke if they ask for one, all from inside the Twilio console.

Looking for the code? Check out the Python version of our Autopilot quickstart (more languages coming soon!)

Let's get started!

I'm ready to create my virtual assistant!

Create your first virtual assistant

To create your first virtual assistant, navigate to the Autopilot home in your Twilio console and click on "Create a new Assistant".

Create your first assistant in the Autopilot console

Give your assistant the name "quickstart" and click Submit.

Create your new assistant

Success! You should see an overview page for your quickstart assistant. We'll start by updating our assistant's most basic task: saying hello.

Show me how it's done

Update your assistant's first task

Tasks are the basic building block for Twilio's Autopilot as they represent what your assistant can do.

Find the default task and update its output

When you created your assistant, it came with a default task that tells the assistant on how to first interact with people. This hello_world task is, by default, set as your assistant's initiation action and fallback default.

The initiation action default tells your assistant which task to perform when it needs to begin the conversation, e.g., when someone calls it. The fallback default tells your assistant which task to perform if it cannot map a user's input to any of its available tasks. You can find these settings in the defaults panel of the Task view in the Autopilot console.

From your quickstart Assistant summary page, click on Task Builder to find it.

Create a task in the Autopilot console with Task Builder

Click on the "hello_world" task to see what your assistant will say if you call it. Update the message to say a friendly greeting.

  "actions": [
      "say": "Hello, world!"

Save your changes.

Let's add a second task

Add a task

Most of the time we'll want an assistant that can handle more than one type of interaction. So far our assistant is polite, but can it answer a question?

We'll have our assistant handle one inquiry to start: "tell me a joke."

Create a 'tell-joke' task

In the Assistant Task Builder, click on the red plus (+) button to create a new task. Give your task the name "tell-joke" and save.

Click on your new task in the task list. You'll see the default JSON actions for any new task, "This is your new task."

Let's start by updating this task to tell our visitor a joke:

  "actions": [
      "say": "I was going to look for my missing watch,  but I could never find the time."

Save your updated task.

Add samples to the task to train your assistant

Now we need to train our assistant to listen for key words and phrases that show that our caller wants to hear a joke. To do this, we'll add some samples to this new task so that our assistant can distinguish it from the 'hello_world' task.

Go to the Natural Language Router view for your assistant. This is where you can add training samples to your assistant's tell-joke task.

Add samples to your assistant's task via the Autopilot console

It's best to start with at least ten training samples for a given task that requires your assistant to map human language to a set of actions.

You can click to expand the input field for your samples and enter the following words and phrases:

Tell me a joke
I'd like to hear a joke
Do you know any good jokes?
Tell joke
Tell me something funny
Make me laugh
I want to hear a joke
Can I hear a joke?
I like jokes
I'd like to hear a punny joke

Select the task you're adding samples to, tell-joke, from the drop-down. Then click Add Sample to build out your collection of training samples.

You've just given your assistant a slightly bigger vocabulary for its first task.

Close the Samples view: it's time to tell your assistant to listen for input.

Show me how my assistant listens for input

Listen for a prompt

Since our hello_world task will execute when our assistant first connects with a human over the phone, we're going to update it to introduce itself properly and listen for input.

Navigate back to task builder in the Autopilot console and open up the hello_world task. Update the say action to greet your caller, introduce your virtual assistant, and prompt the user to say something:

"say": "Hello! I'm your virtual assistant. What would you like help with today?"

We've mentioned the say action throughout this quickstart. Actions instruct an assistant on how to perform a given task and are defined in an array, as we've seen with the say in both of our tasks. You can see the full list of available task actions here.

For our assistant to collect our caller's response, we're going to leverage Autopilot's listen action. This action instructs the assistant to wait for additional input, keeping the dialogue alive. Without this action, the dialogue would simply end after the assistant greets our caller.

Update the JSON for the hello_world task to include the "listen" action and set it to true. The full JSON for hello_world should now look like this:

  "actions": [
      "say": "Hello! I'm your virtual assistant. What would you like help with today?"
      "listen": true

Save the updated initiation task. We're ready to put our assistant to the test.

Show me how it works

Create your assistant's model build

To work these changes into the machine learning model for your quickstart assistant, you need to create a new Model Build.

From the Natural Language Router view you've been working in, click on Build Models.

Autopilot's Natural Language Router console view: build your model

Then, click on the red "Create your first Model Build" button and give your build a new name: we'll version our model and name it v0.1 and submit.

Create a new model build for your assistant

Once your build's status reads "Completed," you can test your assistant out with the Assistant Inspector.

Test with a Query

As you build your assistant, you can test it out directly via the console.

Navigate to your assistant logs and click on Create your first Query

Navigate to your assistant logs and create your first query

Query by entering the phrase "Tell me a joke" and submit your query.

The natural language understanding created from your samples should identify that this expression corresponds to the tell-joke task. You can leverage this query feature as you continue to build out tasks for your assistant to see how it will interact with your users.

Now that we've taught our assistant how to listen for a request and tell a joke, it's time to test this out in the real world!

I'm ready to call my assistant

Connect your assistant to a phone number and try it out

Now that your assistant is configured and trained on a task, we can test it out with a phone call.

If you don't already have a Twilio phone number that you'd like to connect to your assistant, go to the phone numbers page in the console and find a number you like with Voice calling capabilities.

Purchase a voice-enabled phone number from Twilio

Once you have your number, select it from your list of phone numbers. Scroll down to the Voice & Fax section. Next to A Call Comes In, choose Webhook from the drop-down.

Enter the following URL to set the Voice webhook URL for your assistant:


Replace the placeholder value for <ACCOUNT_SID> with your unique Twilio Account SID, which you can find on your console home page. Replace <ASSISTANT_SID> with the unique SID for your quickstart assistant – you can see this value in the Assistant list in your console: it will start with UA.

Save your settings. Give this phone number a call to connect to your assistant and ask to hear a joke!

What's next?

What's next?

Congratulations! You've built your first assistant with Twilio.

Most virtual assistants need to do more complex tasks than telling one joke, though. Check out these other tutorials and docs to go deeper with Autopilot:

  • Learn how to make a more complex assistant that can gather and track user input in our Deep Table Reservations tutorial.
  • See how to connect your assistant to Slack.
  • Want to connect your new assistant to Alexa? Find step-by-step instructions here.
  • We leveraged Autopilot's say and listen actions in this quickstart, but there's a lot more Autopilot can do! Learn about the other available actions here.

We can't wait to see what you build!

Rate this page:

Need some help?

We all do sometimes; code is hard. Get help now from our support team, or lean on the wisdom of the crowd browsing the Twilio tag on Stack Overflow.