Menu

Expand
Rate this page:

Twilio Autopilot Python Quickstart

With Twilio's Autopilot, you can build messaging and voice bots powered by Python and Twilio's own Natural Language Processing engine.

In this quickstart, we're going to build a simple bot (that you can also test in the Autopilot console) that can tell a user what clothes are available for purchase and detect intent, as well as extract entities from a user's utterance.

Show me how it's done

If you already have a Twilio account and a voice-enabled Twilio phone number, you’re all set here! Feel free to jump to the next step.

Before you can build a bot with Twilio, you'll need to sign up for a Twilio account or sign into your existing account. For this quickstart, you'll also need a Twilio phone number that's capable of making calls.

You can sign up for a free Twilio trial account here.

  • When you sign up, you'll be asked to verify your personal phone number. This helps Twilio verify your identity and also allows you to make calls to your phone from your Twilio account while in trial mode.
  • Once you verify your number, you'll be asked to create a project. For the sake of this tutorial, you can click on the "Learn and Explore" template. Give your project a name, or just click "skip remaining steps" to continue with the default.
  • Once you get through the project creation flow, you'll arrive at your project dashboard in the Twilio Console. This is where you'll be able to access your Account SID, authentication token, find a Twilio phone number, and more.

If you don't currently own a Twilio phone number with Voice functionality, you'll need to purchase one. After navigating to the Buy a Number page, check the "Voice" box and click "Search."

Buy a voice compatible number with Twilio

You’ll then see a list of available phone numbers and their capabilities. Find a number that you like and click "Buy" to add it to your account.

voice_compatible-number.png

Now that you have a Twilio account and a programmable phone number, you can start building your bot! To make building with Autopilot easier, we'll check that we have Python installed properly next.

Take me through the setup

If you’ve gone through one of our other Python Quickstarts already and have Python installed, you can skip this step and get straight to creating your first bot.

To build and manage your bot with Python you'll need to have a recent version of Python (3.5 or higher).

Checking your Python installation

If you are on a Mac or a Linux/Unix machine, you probably already have Python
installed. On Windows, you typically would not have Python installed out of the box.

Try running the following command from the command line to check your installation:

python --version

You should see a response similar to this:

Python 3.7.5

Any version of Python above 3.5 should work well for this project.

Installing Python on Windows

Windows users can follow this excellent tutorial for installing Python on Windows, or follow the instructions from Python's documentation.

Installing Python on OS X or Linux

We recommend installing Python following the guide for Mac OS X or your Linux distribution.

All set! Let's create that bot now.

Create your bot

Now you can spin up a new bot. You could use a ready-made template that comes with pre-trained samples and already-created tasks, but let's make it from scratch for the purposes of this guide.

A bot is a conversational application that you can deploy across multiple channels, like SMS and Voice. Each bot is independent of other bots you create and has a unique set of tasks that it can do.

In your Autopilot console, select Create a new Bot.

Create a new bot image

Then scroll down and select Start from scratch.

Start from Scratch

You will have to give your bot a name, but do not worry! You can change it later. Then click Create a bot. Congratulations! You've just created an Autopilot bot.

Tell me about how a bot interacts with users

Create and train one of your bot's tasks

A Twilio Autopilot bot runs on tasks.

A task is a set of actions your bot needs to carry out when interacting with users. Tasks are modeled on something your end user wants or needs, and the specific things your bot needs to do to help them out.

A bot usually has many tasks that power it. These could be simple tasks like confirm or cancel or more complex tasks like make-a-reservation.

What are tasks made up of?

Autopilot Actions

Actions instruct the bot on how to perform a given task.

When a bot executes a task, it will run through all specified actions and then end the interaction. Most bots will have many tasks that each contain multiple actions. You can imagine a bot that says something, listens for input, collects some data, and then hands off the caller to an agent.

Your bot comes with four pre-trained, pre-built tasks. We'll start by modifying the greeting task, which just says a brief introduction including what items the user can buy. We'll then make a slightly more complicated task called buy_clothes that says a brief response and listens for user input.

Modify an existing task with static JSON via the console

Make a task with Static JSON

Click the Program button corresponding to the greeting task and replace the JSON in the Action Bin by copying and pasting the JSON code below.

{
    "actions": [
        {
            "say": "Hello, we have over one hundred shirts, shoes, pants, skirts, and dresses you can buy! What can I help you with today?"
        },
        {
            "listen": true
        }
    ]
}

Now click Save edits below and click All Tasks to add your new task to your bot.

Next, we will use static JSON to program the task. After that, we will train the task with a few samples. Once the task is trained, we can test the bot in the chat simulator.

All of this is done through the web console.

Add a new task

Click Add a task next to the red plus button to add a new task, then name it buy_clothes before clicking the red Add task button. Now it's time to program your new task with static JSON.

Add task header

Program a task with static JSON via the console

Press the Program button corresponding to buy_clothes and put in the following JSON:

{
    "actions": [
        {
            "say": "What clothes would you like to order?"
        },
        {
            "listen": true
        }
    ]
}

Click Save edits again and now it's time to move on to training these tasks with different samples.

Show me how to train my bot via the console

Train your bot

Now we need to train our bot to listen for key words and phrases that show that our user wants to know what clothes are available to buy or to order different clothes. To do this, we'll add some samples to this new task so that our bot can distinguish it from other tasks, like the buy_clothes task.

A sample is the training data your bot uses to understand user input. Samples define how people might describe a task when speaking or typing, and are necessary for your bot to learn how to interact with your users.

Usually, it's best to provide at least ten samples for any given task so that your bot can map human input to the task. However, our greeting task will serve a special purpose for our bot: we'll set this task as the bot's default tasks so that your bot will say this phrase when anyone calls in. Because your bot will use this task before a user starts interacting with it, it's okay to just have a few samples.

Next, click the Switch to train task button.

Here you can add samples which are phrases to trigger your buy_clothes task. Add "I'd like to buy some clothes" and then click Add sample.

add samples

Feel free to add a few more similar phrases, like maybe "I need some clothes", "Can I purchase clothing", and more. Have fun and get creative!

The greeting task already has fourteen samples like "hello", "hi", and "how's it going" built-in and pre-trained. You could add some more like "what does this bot do" if you wish.

Now before we can test our bot's new tasks, we have to build the model.

Build the model

Build the model

Before your bot can update and leverage the samples we just added, you'll need to create a Model Build. You'll see that alert whenever you add new samples, fields, field values, or change the name of a task. Creating a new Model Build will incorporate these changes into the machine learning model that powers your bot. You can do this by clicking on the red Build model button at the top.

create model build

Now it's time to test our bot in the chat simulator!

Test your bot with the chat simulator

On the left side, select Simulator and type in a greeting to your bot! You should also see the task that was triggered by your user input on the right, as shown below.

greeting_and_buy_clothes_q.gif

If the user input triggered the incorrect task, you could then add it as a sample to the correct task right on that same page! Now let's see how Fields work using that same buy_clothes task.

Show me how Fields work!

Create Fields

A Field is key information expressed by the end-user, such as a time, date, or number, that is usually essential for the bot to complete its task. Each Task can have a distinct set of Fields associated with it. Fields are then used in Samples to train the model on where to look for the given attributes.

There are Field Values which are the key pieces of data that people say. Field Names are the labels for a type of value. Field Types are the values represented by a field name. For our buy_clothing task, a Field Value could be "pants" or "shirt", a Field Name could be "clothing", and we could make a custom Field Type called "clothing_type."

Autopilot comes with built-in Field Types that let you create fields of a common data type, such as Date, Time, Number, and more so you don't have to define them yourself as a custom Field Type. We recommend you use built-in fields whenever possible because they are optimized to understand this type of data.

Let's now create a custom field type in the console and train it with some values. Click on Field Types on the lefthand side and scroll down to You have no custom field types. Under Custom Field Type type "Clothing" and under A Value For This Type put "pants".

custom field types

Click the red Create field type button and add more values like shorts, bottoms, skirts, dresses, shoes, and more!

Add Fields to a Task

Build a new model to implement the changes you've just made to your Autopilot bot. Now to add a Field to a Task, go back to Tasks and select your buy_clothes task. Click on Train and then select Fields instead of Samples.

Add Fields to a Task

First we'll add a built-in Field to the task. Under Field Name, type in "quantity" and under Field Type select Twilio.NUMBER. Then click the red Add field button.

Add built-in field

To add the Custom Field Clothing to the task, type "clothing" under Field Name and Clothing under Field Type. Tada! You've just created both a custom Field and a built-in Field to one of your Autopilot tasks.

Back under your buy_clothing task, modify the samples to reflect the fields, as shown in the gif below: "I wish to purchase two skirts" would turn into "I wish to purchase {quantity} {clothing}".

edit_samples_for_fields.gif

If you were to build the model and test your bot in the simulator again but with a message that includes these fields, you'd see the fields get displayed in the simulator like so:

fields_recognized_in_simulator.gif

Program the same task dynamically in Python

Program Tasks Dynamically

Now let's see how the same task can be programmed by rendering JSON from your web server in Python with Flask. The task will still say a brief expression and then listen for user input.

You will need to start by creating a new directory for your application. Inside that directory, create two files - requirements.txt and app.py. With the requirements.txt, we will add the Python web framework Flask to our project. Our application code will go into the app.py Python file.

The requirements.txt should look like this:

Flask

To start with, your app.py file can simply be:

from flask import Flask

app = Flask(__name__)

@app.route('/')
def home():
    return 'Hello World!'

We will add more to this Python file as we go.

Before doing anything else, install the Flask package with the following command:

python -m pip install -r requirements.txt

If you would like, test your Python installation by running this command:

flask run

And then when you visit this URL: http://localhost:5000 - you should see Hello World in your web browser.

Instead of providing JSON in the Autopilot console, we are going to serve JSON from our Python server. For that purpose, we are going to use Flask's send_file method to send a file named dynamicsay.json.

Create a file named dynamicsay.json in the same directory as the app.py file, and then copy the following contents into the file:

{
  "actions": [
      {
          "say": "What clothes would you like to order?"
      },
      {
          "listen": true
      }
  ]
}

In your app.py file, add the following code:

from flask import send_file

@app.route('/dynamicsay', methods=['POST'])
def dynamic_say():
    return send_file('dynamicsay.json')

Next, start up ngrok to get a public URL for your server. If you're new to ngrok, see our documentation on How to Install ngrok.

ngrok http 5000

You will end up with an external URL like https://3dd78ddd.ngrok.io for your application.

Now go back to your Autopilot Tasks and click program. Replace the JSON code in the ActionBin of buy_clothes with this:

{
    "actions": [
        {
            "redirect": "https://your-ngrok-subdomain-here.ngrok.io/dynamicsay"
        }
    ]
}

Don't forget to replace the subdomain in the redirect URL with the one from the ngrok URL!

Click Save edits and you've made that same task use your Python web application. You could test in the chat simulator again:

simulator_show_dynamic_action.gif

Tell me about Autopilot’s request to my application

Now, what happens when a Task is triggered? Autopilot makes a request to the URL configured to the Task that will always include certain parameters such as AccountSid, AssistantSid, DialogueSid, UserIdentifier, CurrentInput, CurrentTask, FieldValue, FieldType, and more. If any Fields are recognized in the intent, they are given back to your application as a key-value pair of the field value or field type recognized.

Autopilot will send these parameters either as POST parameters, in the format application/x-www-form-urlencoded, or URL query parameters, depending on which HTTP method you've configured. More information on the Autopilot Request can be found here.

You may have noticed that we need a task to redirect to after buy_clothes. Let's use a Collect flow for this new task.

What is a Collect flow?

Create a Collect flow

Autopilot's Collect action lets you ask questions to users and efficiently collect answers in the bot's memory, similar to how web forms are used. You define the field name and type for each question and Collect renders a conversational flow to ask the user the questions. The answers are then collected in the bot's memory and sent via the request parameters on each Autopilot webhook.

Time to make a Collect flow!

Below is a code listing for a Collect flow named collect_clothes_order which asks a series of four questions.

Each question is given a name and a type.

The name is how we can receive and reference the answer from the user and the type is the type of answer we expect to receive. If a user's answer is not a number when a number is the type expected, then the user will be asked to clarify their answer.

Now replace the contents of the dynamicsay.json file with the following JSON. At the end of the JSON file, replace the sample ngrok URL with your ngrok URL, or Collect won't work. You'll see an error appear in the Twilio debugger if the URL isn't correct.

{
  "actions": [
    {
      "collect": {
        "name": "collect_clothes_order",
        "questions": [
          {
            "question": "What is your first name?",
            "name": "first_name",
            "type": "Twilio.FIRST_NAME"
          },
          {
            "question": "What type of clothes would you like?",
            "name": "clothes_type",
            "type": "Clothing"
          },
          {
            "question": "How many would you like to order?",
            "name": "num_clothes",
            "type": "Twilio.NUMBER"
          },
          {
            "question": "What country is your shipping address in?",
            "name": "shipping_country",
            "type": "Twilio.COUNTRY"
          }
        ],
        "on_complete": {
          "redirect": "https://your-ngrok-subdomain-here.ngrok.io/collect"
        }
      }
    }
  ]
}

This receives the user input, saves it as variables, and returns a response to the user showing them their information was received. Let's now parse the answers collected by the flow from the Memory object in Autopilot’s request.

Parse answers from the Collect flow in Autopilot's request

We need to add another route to app.py for the collect path that will generate the JSON for the Say Action to redirect to our Collect flow. This code parses the answers out of the Memory parameter (which is a JSON construct), and then echoes them back to the user in a confirmation message.

import json
from flask import jsonify, request

@app.route('/collect',  methods=['POST'])
def collect():
    memory = json.loads(request.form.get('Memory'))

    answers = memory['twilio']['collected_data']['collect_clothes_order']['answers']

    first_name = answers['first_name']['answer']
    clothes_type = answers['clothes_type']['answer']
    num_clothes = answers['num_clothes']['answer']

    message = (
        f'Ok {first_name}. Your order for {num_clothes} {clothes_type} is now confirmed.'
        f' Thank you for ordering with us'
    )

    return jsonify(actions=[{'say': {'speech': message}}])

Now what if they respond incorrectly to one? Never fear! Autopilot is here!

What if a user responds incorrectly to a Collect question?

Use Collect's Validate Instruction

The Collect action lets you validate user input with the Validate instruction. You can pass it a list of allowed values, a webhook to validate the value being collected (this is useful when validating fields like order numbers or available dates that can only be validated against your business logic), the messages when the input is invalid, the messages when the question is answered successfully, and the maximum number of attempts Autopilot should ask the user the question. More on Validate can be found here.

Let's modify the contents of the JSON response in dynamicsay.json to include the Validate action.

{
  "actions": [
    {
      "collect": {
        "name": "collect_clothes_order",
        "questions": [
          {
            "question": "What is your first name?",
            "name": "first_name",
            "type": "Twilio.FIRST_NAME"
          },
          {
            "question": "What type of clothes would you like?",
            "name": "clothes_type",
            "type": "Clothing",
            "validate": {
              "on_failure": {
                "messages": [
                  {
                    "say": "Sorry, that's not a clothing type we have. We have shirts, shoes, pants, skirts, and dresses."
                  }
                ],
                "repeat_question": true
              },
              "on_success": {
                "say": "Great, I've got the clothing type you want."
              },
              "max_attempts": {
                "redirect": "task://collect_fallback",
                "num_attempts": 3
              }
            }
          },
          {
            "question": "How many would you like to order?",
            "name": "num_clothes",
            "type": "Twilio.NUMBER"
          }
        ],
        "on_complete": {
          "redirect": "https://your-ngrok-subdomain-here.ngrok.io/collect"
        }
      }
    }
  ]
}

What did we add? For the second question in the Collect flow, we added the Validate Action that has a message to respond with if the user inputs an answer that our Autopilot bot does not recognize. We set repeat_question to true and then max_attempts to three so that the user only has three tries to answer the question correctly. If their three tries are up, the task will redirect to our already-made collect_fallback task. If the user answers correctly, the bot will respond with "Great, I've got the clothing type you want" and continue going through each question in the Collect flow.

Test out our completed bot with Collect and Validate

collect_final.gif

What's next?

Congratulations, you just built your first Twilio bot with Autopilot!

This quickstart taught you the basics of building a simple Autopilot-powered bot and then built upon different Autopilot concepts. Autopilot also powers bots that can collect data and route users through complex flows.

Go deeper with Twilio Autopilot with the following tutorials and reference docs:

We can't wait to see what you build!

Rate this page:

Need some help?

We all do sometimes; code is hard. Get help now from our support team, or lean on the wisdom of the crowd browsing the Twilio tag on Stack Overflow.

        
        
        

        Thank you for your feedback!

        We are always striving to improve our documentation quality, and your feedback is valuable to us. How could this documentation serve you better?

        Sending your feedback...
        🎉 Thank you for your feedback!
        Something went wrong. Please try again.

        Thanks for your feedback!

        Refer us and get $10 in 3 simple steps!

        Step 1

        Get link

        Get a free personal referral link here

        Step 2

        Give $10

        Your user signs up and upgrade using link

        Step 3

        Get $10

        1,250 free SMSes
        OR 1,000 free voice mins
        OR 12,000 chats
        OR more