Rate this page:

Twilio Autopilot Java Quickstart

With Twilio's Autopilot, you can build messaging and voice bots powered by Java and Twilio's own Natural Language Processing engine.

In this quickstart, we're going to build a simple bot (that you can also test in the Autopilot console) that can tell a user what clothes are available for purchase and detect intent, as well as extract entities from a user's utterance.

Show me how it's done

Sign up for Twilio and get a phone number

If you already have a Twilio account and a voice-enabled Twilio phone number, you’re all set here! Feel free to jump to the next step.

Before you can build a bot with Twilio, you'll need to sign up for a Twilio account or sign into your existing account. For this quickstart, you'll also need a Twilio phone number that's capable of making calls.

You can sign up for a free Twilio trial account here.

  • When you sign up, you'll be asked to verify your personal phone number. This helps Twilio verify your identity and also allows you to make calls to your phone from your Twilio account while in trial mode.
  • Once you verify your number, you'll be asked to create a project. For the sake of this tutorial, you can click on the "Learn and Explore" template. Give your project a name, or just click "skip remaining steps" to continue with the default.
  • Once you get through the project creation flow, you'll arrive at your project dashboard in the Twilio Console. This is where you'll be able to access your Account SID, authentication token, find a Twilio phone number, and more.

If you don't currently own a Twilio phone number with Voice functionality, you'll need to purchase one. After navigating to the Buy a Number page, check the "Voice" box and click "Search."

Search for a Phone Number.png

You’ll then see a list of available phone numbers and their capabilities. Find a number that you like and click "Buy" to add it to your account.

Buy a Phone Number.png

Now that you have a Twilio account and a programmable phone number, you can start building your bot! To make building with Autopilot easier, we'll check that we have Java and Gradle installed properly next.

Take me through the setup

Install Java and Gradle

If you’ve gone through one of our other Java Quickstarts already and have Java and Gradle installed, you can skip this step and get straight to creating your first bot.

To build and manage your bot with Java you'll need to have a recent version of Java (Java 8 higher) and the Gradle build tool installed.

Checking your Java installation

You may already have the Java SE Development Kit installed.

Try running the following command from the command line to check your installation:

javac -version

You should see a response similar to this:

javac 1.8.0_92

Any version of Java above 1.8 (which is Java 8) should work well for this project.

Installing Java

If you need to install Java, follow our guide for How to set up your Java and Servlets development environment.

Install Gradle

While you can certainly use an IDE or the Maven build tool, we will use the Gradle build tool in this guide. Follow the installation instructions for Gradle for your platform.

All set! Let's create that bot now.

Create your bot

Now you can spin up a new bot. You could use a ready-made template that comes with pre-trained samples and already-created tasks, but let's make it from scratch for the purposes of this guide.

A bot is a conversational application that you can deploy across multiple channels, like SMS and Voice. Each bot is independent of other bots you create and has a unique set of tasks that it can do.

In your Autopilot console, select create a new Bot by selecting Start from scratch

A screenshot of a portion of the Twilio Console. In the left column we see the Develop panel selected, with Autopilot expanded and "your bots" selected. On the right, the screen reads "Select a bot." Below this we read "Create a bot" with the options of "Browse templates" and, below it, "Start from scratch"

You will have to give your bot a unique name, but do not worry! You can change it later. Then click Create a bot. Congratulations! You've just created an Autopilot bot.

Tell me about how a bot interacts with users

Create and train one of your bot's tasks

A Twilio Autopilot bot runs on tasks.

A task is a set of actions your bot needs to carry out when interacting with users. Tasks are modeled on something your end user wants or needs, and the specific things your bot needs to do to help them out.

A bot usually has many tasks that power it. These could be simple tasks like confirm or cancel or more complex tasks like make-a-reservation.

What are tasks made up of?

Autopilot Actions

Actions instruct the bot on how to perform a given task.

When a bot executes a task, it will run through all specified actions and then end the interaction. Most bots will have many tasks that each contain multiple actions. You can imagine a bot that says something, listens for input, collects some data, and then hands off the caller to an agent.

Your bot comes with four pre-trained, pre-built tasks. We'll start by modifying the greeting task, which just says a brief introduction including what items the user can buy. We'll then make a slightly more complicated task called buy_clothes that says a brief response and listens for user input.

Modify an existing task with static JSON via the console

Make a task with Static JSON

Click the Program button corresponding to the greeting task and replace the JSON in the Action Bin by copying and pasting the JSON code below.

    "actions": [
            "say": "Hello, we have over one hundred shirts, shoes, pants, skirts, and dresses you can buy! What can I help you with today?"
            "listen": true

Now click Save edits below and click All Tasks to add your new task to your bot.

Next, we will use static JSON to program the task. After that, we will train the task with a few samples. Once the task is trained, we can test the bot in the chat simulator.

All of this is done through the web console.

Add a new task

Click the blue Add a task button to add a new task, then name it buy_clothes before clicking the blue Add button. Now it's time to program your new task with static JSON.

Screenshot of the Twilio console for Autopilot screen for "create tasks for your bot." We've entered "buy_clothes" into the field, and there's a blue "Add" button to the right.

Program a task with static JSON via the console

Press the Program button corresponding to buy_clothes and put in the following JSON:

    "actions": [
            "say": "What clothes would you like to order?"
            "listen": true

Click Save edits again and now it's time to move on to training these tasks with different samples.

Show me how to train my bot via the console

Train your bot

Now we need to train our bot to listen for key words and phrases that show that our user wants to know what clothes are available to buy or to order different clothes. To do this, we'll add some samples to this new task so that our bot can distinguish it from other tasks, like the buy_clothes task.

A sample is the training data your bot uses to understand user input. Samples define how people might describe a task when speaking or typing, and are necessary for your bot to learn how to interact with your users.

Usually, it's best to provide at least ten samples for any given task so that your bot can map human input to the task. However, our greeting task will serve a special purpose for our bot: we'll set this task as the bot's default tasks so that your bot will say this phrase when anyone calls in. Because your bot will use this task before a user starts interacting with it, it's okay to just have a few samples.

Next, click the Train link next to the buy_clothes task to see the training view.

Here you can add samples which are phrases to trigger your buy_clothes task. Add "I'd like to buy some clothes" and then click Add sample.

The task training view for "buy_clothes." Under "What would people say to trigger the task: buy_clothes," we've entered "I'd like to buy some clothes". A blue arrow points to the blue "add sample" button on the right.

Feel free to add a few more similar phrases, like maybe "I need some clothes", "Can I purchase clothing", and more. Have fun and get creative!

The greeting task already has fourteen samples like "hello", "hi", and "how's it going" built-in and pre-trained. You could add some more like "what does this bot do" if you wish.

Now before we can test our bot's new tasks, we have to build the model.

Build the model

Build the model

Before your bot can update and leverage the samples we just added, you'll need to create a Model Build. You'll see that alert whenever you add new samples, fields, field values, or change the name of a task. Creating a new Model Build will incorporate these changes into the machine learning model that powers your bot. You can do this by clicking on the button that reads Build model, which sits beneath your task list:

A screenshot of the task list view for an Autopilot bot. Underneath the task list, which starts with "buy_clothes," we see a warning sign that reads "your edits need a model build." Beneath that text is a button labeled "Build model" with a blue arrow pointing to it.

Now it's time to test our bot in the chat simulator!

Test your bot with the chat simulator

To the right of where you clicked "Build model" there's a link that reads Simulate.

Trigger a simulation of your bot by selecting "Simulator" in the left-hand navigation, or the link to the right of the "build model" button

Similarly, you can select Simulator from the left-hand navigation in the console to test out your model and type in a greeting to your bot!

You should also see the task that was triggered by your user input on the right, as shown below.


If the user input triggered the incorrect task, you could then add it as a sample to the correct task right on that same page! Now let's see how Fields work using that same buy_clothes task.

Show me how Fields work!

Create Fields

A Field is key information expressed by the end-user, such as a time, date, or number, that is usually essential for the bot to complete its task. Each Task can have a distinct set of Fields associated with it. Fields are then used in Samples to train the model on where to look for the given attributes.

There are Field Values which are the key pieces of data that people say. Field Names are the labels for a type of value. Field Types are the values represented by a field name. For our buy_clothing task, a Field Value could be "pants" or "shirt", a Field Name could be "clothing", and we could make a custom Field Type called "clothing_type."

Autopilot comes with built-in Field Types that let you create fields of a common data type, such as Date, Time, Number, and more so you don't have to define them yourself as a custom Field Type. We recommend you use built-in fields whenever possible because they are optimized to understand this type of data.

Let's now create a custom field type in the console and train it with some values. Click on Field Types on the lefthand side and scroll down to You have no custom field types. Under Custom Field Type type "Clothing" and under A Value For This Type put "pants".

Field Types page for Autopilot bot. At the bottom of the page we've highlighted the fields "Custom field type" and "a value for this type" where we've entered "Clothing" and "pants," respectively. To the right of these fields is a blue button reading "Create field type"

Click the blue Create field type button and add more values like shorts, bottoms, skirts, dresses, shoes, and more!

Add Fields to a Task

Build a new model to implement the changes you've just made to your Autopilot bot. Now to add a Field to a Task, go back to Tasks and select your buy_clothes task. Click on Train and then select Fields instead of Samples.

The Tasks panel for our Autopilot bot is selected in the left navigation panel, and we've toggled to the "Fields" section of the buy_clothes task.

First we'll add a built-in Field to the task. Under Field Name, type in "quantity" and under Field Type select Twilio.NUMBER from the dropdown.Then click the blue Add field button.

Under "field name" we've entered "quantity" into the empty box, and selected "Twilio.NUMBER" from the dropdown under "Field type." To the right of these fields is a blue Add field button.

To add the Custom Field Clothing to the task, type "clothing" under Field Name and Clothing under Field Type. Tada! You've just created both a custom Field and a built-in Field to one of your Autopilot tasks.

Back under your buy_clothes task, modify the samples to reflect the fields, as shown in the gif below: "I wish to purchase two skirts" would turn into "I wish to purchase {quantity} {clothing}".


If you were to build the model and test your bot in the simulator again but with a message that includes these fields, you'd see the fields get displayed in the simulator like so:


Program the same task dynamically in Java

Program Tasks Dynamically

Now let's see how the same task can be programmed by rendering JSON from your web server in Java. The task will still say a brief expression and then listen for user input.

You will need to start by creating a new directory for your application. Open up a command prompt or terminal, and then start a new Gradle project with gradle init:

gradle init

Gradle will ask a series of questions to set up your project.

For the first question, choose to create an application.

Select type of project to generate:
  1: basic
  2: application
  3: library
  4: Gradle plugin
Enter selection (default: basic) [1..4] 2

You can stick with defaults (press enter) for the language, DSL, and test framework options.

Select implementation language:
  1: C++
  2: Groovy
  3: Java
  4: Kotlin
  5: Swift
Enter selection (default: Java) [1..5] 

Select build script DSL:
  1: Groovy
  2: Kotlin
Enter selection (default: Groovy) [1..2] 

Select test framework:
  1: JUnit 4
  2: TestNG
  3: Spock
  4: JUnit Jupiter
Enter selection (default: JUnit 4) [1..4] 

The project name should be AutopilotApp.

Here is the remaining output of that gradle init command.

Project name (default: java): AutopilotApp
Source package (default: AutopilotApp): 

> Task :init
Get more help with your project:

2 actionable tasks: 2 executed

Gradle will create several files and directories for your project so you can get started.

We will need to modify two files - build.gradle and src/main/java/AutopilotApp/ Within the build.gradle, we will add the Spark Java web framework to our project (not to be confused with Apache Spark, for data processing). Our application code will go into the source file.

The build.gradle should look like this (we add dependencies for Spark, and for the Velocity template language):

 * This file was generated by the Gradle 'init' task.
 * This generated file contains a sample Java project to get you started.
 * For more details take a look at the Java Quickstart chapter in the Gradle
 * User Manual available at

plugins {
    // Apply the java plugin to add support for Java
    id 'java'

    // Apply the application plugin to add support for building a CLI application.
    id 'application'

repositories {
    // Use jcenter for resolving dependencies.
    // You can declare any Maven/Ivy/file repository here.

dependencies {
    // This dependency is used by the application.
    implementation ''

    implementation 'com.sparkjava:spark-core:2.8.0'
    implementation 'com.sparkjava:spark-template-velocity:2.7.1'
    implementation ''

    // Use JUnit test framework
    testImplementation 'junit:junit:4.12'

application {
    // Define the main class for the application.
    mainClassName = 'AutopilotApp.App'

To start with, your file can simply be:

package AutopilotApp;

import static spark.Spark.*;

public class App {
    public static void main(String[] args) {
        get("/", (req, res) -> "Hello World");

We will add more to this Java application as we go.

Before doing anything else, try running your Java web application with the following command:

gradle run

And then when you visit this URL: http://localhost:4567 - you should see Hello World in your web browser.

Instead of providing JSON in the Autopilot console, we are going to serve JSON from our Java server. For that purpose, we are going to use the Spark web framework's Velocity template engine to send a file named dynamicsay.json. We won't actually change the contents of the JSON file with this example, but you easily could.

Create a file named dynamicsay.json in the src/main/resources directory, and then copy the following contents into the file:

  "actions": [
          "say": "What clothes would you like to order?"
          "listen": true

In your file, add the following code to the main() method:

post("/dynamicsay", (req, res) -> {
    Map<String, Object> model = new HashMap<>();
    return new VelocityTemplateEngine().render(
        new ModelAndView(model, "dynamicsay.json")

Also, add the following import statements to the top of the file:

import spark.template.velocity.VelocityTemplateEngine;
import spark.ModelAndView;

import java.util.HashMap;
import java.util.Map;

Now, you can stop your gradle run command with Ctrl-C, and then re-run it with gradle run again.

Next, start up ngrok to get a public URL for your server. If you're new to ngrok, see our documentation on How to Install ngrok.

ngrok http 4567

You will end up with an external URL like for your application.

Now go back to your Autopilot Tasks and find the buy_clothes task - click the program link for that task. In the screen that appears, replace the JSON code in the ActionBin of buy_clothes with this:

    "actions": [
            "redirect": ""

Don't forget to replace the subdomain in the redirect URL with the one from the ngrok URL!

Click Save edits and you've made that same task use your Java web application. You could test in the chat simulator again:


Tell me about Autopilot’s request to my application

Now, what happens when a Task is triggered? Autopilot makes a request to the URL configured to the Task that will always include certain parameters such as AccountSid, AssistantSid, DialogueSid, UserIdentifier, CurrentInput, CurrentTask, FieldValue, FieldType, and more. If any Fields are recognized in the intent, they are given back to your application as a key-value pair of the field value or field type recognized.

Autopilot will send these parameters either as POST parameters, in the format application/x-www-form-urlencoded, or URL query parameters, depending on which HTTP method you've configured. More information on the Autopilot Request can be found here.

You may have noticed that we need a task to redirect to after buy_clothes. Let's use a Collect flow for this new task.

What is a Collect flow?

Create a Collect flow

Autopilot's Collect action lets you ask questions to users and efficiently collect answers in the bot's memory, similar to how web forms are used. You define the field name and type for each question and Collect renders a conversational flow to ask the user the questions. The answers are then collected in the bot's memory and sent via the request parameters on each Autopilot webhook.

Time to make a Collect flow!

Below is a code listing for a Collect flow named collect_clothes_order which asks a series of four questions.

Each question is given a name and a type.

The name is how we can receive and reference the answer from the user and the type is the type of answer we expect to receive. If a user's answer is not a number when a number is the type expected, then the user will be asked to clarify their answer.

Now replace the contents of the dynamicsay.json file with the following JSON. At the end of the JSON file, replace the sample ngrok URL with your ngrok URL, or Collect won't work. You'll see an error appear in the Twilio debugger if the URL isn't correct.

  "actions": [
      "collect": {
        "name": "collect_clothes_order",
        "questions": [
            "question": "What is your first name?",
            "name": "first_name",
            "type": "Twilio.FIRST_NAME"
            "question": "What type of clothes would you like?",
            "name": "clothes_type",
            "type": "Clothing"
            "question": "How many would you like to order?",
            "name": "num_clothes",
            "type": "Twilio.NUMBER"
            "question": "What country is your shipping address in?",
            "name": "shipping_country",
            "type": "Twilio.COUNTRY"
        "on_complete": {
          "redirect": ""

This receives the user input, saves it as variables, and returns a response to the user showing them their information was received. Let's now parse the answers collected by the flow from the Memory object in Autopilot’s request.

Parse answers from the Collect flow in Autopilot's request

We need to add another route to our Java application for the collect path that will generate the JSON for the Say Action to redirect to our Collect flow. This code parses the answers out of the Memory parameter (which is a JSON construct), and then echoes them back to the user in a confirmation message.

post("/collect", (req, res) -> {
    String memory = req.queryParams("Memory");
    if (memory != null) {
        return createMessage(memory);
    return "OK";

We do need to parse the JSON contained in the memory, and then turn that into a message.

private static String createMessage(String memory) {
    JsonObject root = new JsonParser()
    JsonObject answers = root.getAsJsonObject("twilio")
    String firstName = answers.getAsJsonObject("first_name")
    String clothesType = answers.getAsJsonObject("clothes_type")
    String numClothes = answers.getAsJsonObject("num_clothes")

    String fmt = "Ok %s. Your order for %s %s is now confirmed. Thank you for ordering with us";
    String message = String.format(fmt,
                        firstName, numClothes, clothesType);
    return message;

Now what if they respond incorrectly to one? Never fear! Autopilot is here!

What if a user responds incorrectly to a Collect question?

Use Collect's Validate Instruction

The Collect action lets you validate user input with the Validate instruction. You can pass it a list of allowed values, a webhook to validate the value being collected (this is useful when validating fields like order numbers or available dates that can only be validated against your business logic), the messages when the input is invalid, the messages when the question is answered successfully, and the maximum number of attempts Autopilot should ask the user the question. More on Validate can be found here.

Let's modify the contents of the JSON response in dynamicsay.json to include the Validate action.

  "actions": [
      "collect": {
        "name": "collect_clothes_order",
        "questions": [
            "question": "What is your first name?",
            "name": "first_name",
            "type": "Twilio.FIRST_NAME"
            "question": "What type of clothes would you like?",
            "name": "clothes_type",
            "type": "Clothing",
            "validate": {
              "on_failure": {
                "messages": [
                    "say": "Sorry, that's not a clothing type we have. We have shirts, shoes, pants, skirts, and dresses."
                "repeat_question": true
              "on_success": {
                "say": "Great, I've got the clothing type you want."
              "max_attempts": {
                "redirect": "task://collect_fallback",
                "num_attempts": 3
            "question": "How many would you like to order?",
            "name": "num_clothes",
            "type": "Twilio.NUMBER"
        "on_complete": {
          "redirect": ""

What did we add? For the second question in the Collect flow, we added the Validate Action that has a message to respond with if the user inputs an answer that our Autopilot bot does not recognize. We set repeat_question to true and then max_attempts to three so that the user only has three tries to answer the question correctly. If their three tries are up, the task will redirect to our already-made collect_fallback task. If the user answers correctly, the bot will respond with "Great, I've got the clothing type you want" and continue going through each question in the Collect flow.

Test out our completed bot with Collect and Validate


What's next?

Congratulations, you just built your first Twilio bot with Autopilot!

This quickstart taught you the basics of building a simple Autopilot-powered bot and then built upon different Autopilot concepts. Autopilot also powers bots that can collect data and route users through complex flows.

Go deeper with Twilio Autopilot with the following tutorials and reference docs:

We can't wait to see what you build!

Rate this page:

Need some help?

We all do sometimes; code is hard. Get help now from our support team, or lean on the wisdom of the crowd by visiting Twilio's Stack Overflow Collective or browsing the Twilio tag on Stack Overflow.


        Thank you for your feedback!

        Please select the reason(s) for your feedback. The additional information you provide helps us improve our documentation:

        Sending your feedback...
        🎉 Thank you for your feedback!
        Something went wrong. Please try again.

        Thanks for your feedback!

        Refer us and get $10 in 3 simple steps!

        Step 1

        Get link

        Get a free personal referral link here

        Step 2

        Give $10

        Your user signs up and upgrade using link

        Step 3

        Get $10

        1,250 free SMSes
        OR 1,000 free voice mins
        OR 12,000 chats
        OR more