How to Build an AI Messaging App with Spacy, Twilio, and Flask

February 15, 2024
Written by
Python Code Nemesis
Opinions expressed by Twilio contributors are their own
Reviewed by
Diane Phan

The power of natural language processing (NLP) allows you to command your computer to effortlessly carry out your instructions and send personalized messages to specific numbers on your behalf. Without the aid of NLP, you'd likely have to adhere to a specific command format and employ regex. This recognition of entities in various sentence structures enhances the application's user-friendliness and intuitiveness, allowing users to interact more naturally.

In this article, we will explore how to leverage the power of Spacy NLP , Twilio , and Flask to develop a system capable of identifying entities within sentences in order to send messages through Twilio, and offering a user-friendly chat interface for seamless interaction.


  • Python 3.x installed.
  • Node.js(v16.13.2) and npm installed.
  • An SMS enabled cell phone to test out the application.
  • A Google account to access Google Colab .

Get a free Twilio number

Twilio provides a number of services and APIs that enable developers to build communication functionality into their applications. In this article, we will be leveraging the power of Programmable Messaging , which enables developers to send and receive SMS and MMS messages.

If you are new to Twilio, follow the steps below to get started. If you already have a Twilio account and Twilio phone number, feel free to skip to the next section.

To get a free Twilio number, sign up for a Twilio account. Sign up for a free Twilio account here . To learn more about Twilio trial accounts, visit this documentation page.

To test a trial account by sending SMS or calls to someone else, you will need to verify your phone number first. Click on the "verify their number" hyperlink to navigate to that screen and verify the number with a one-time password.

Enter your phone number and select a preference for verification.

After phone number verification is complete, the number will be displayed on the Verified Caller IDs screen.

On the Twilio Console , your account information, including the Account SID, the Auth Token, and your Twilio Phone number, will be visible. This will be useful when creating the Flask REST API to send an automated SMS to a number from Twilio.

Set up the project

Once the Twilio Account setup is complete, the next step is to create an AI model and train it with our own custom training samples.

For convenient code maintenance and versioning, you can create a GitHub repository. It can be cloned locally, and all changes made there, which can be pushed to the remote repository periodically. This is so that no progress is lost and changes can be easily rolled back.

To do this, create a new GitHub repository called "Python_Twilio_AI_Messaging_Chatbot". All the code for the next steps is available on this GitHub repository .

Next, follow the instructions in the GitHub documentation to generate a fine-grained personal access token for GitHub. Choose "Python_Twilio_AI_Messaging_Chatbot" under Repository access by selecting Only select repositories and searching for this repository in the list. Then, under Permissions, scroll down to the Contents section and select Read and write in the dropdown field.

Once you have generated your personal access token, copy it and keep it in a safe place, as it will not be shown again.

Clone the repository locally by running the following command, replacing YOUR_GITHUB_USERNAME and YOUR_PERSONAL_ACCESS_TOKEN with your own username and personal access token, respectively.

git clone

Once you have cloned the repository, change into the Python_Twilio_AI_Messaging_Chatbot directory:

cd Python_Twilio_AI_Messaging_Chatbot

To set up the remote repository to keep track with the main branch, run this command:

git remote set-url origin

Use Named Entity Recognition (NER) in the application

Named Entity Recognition (NER) is a natural language processing (NLP) task that involves identifying and classifying named entities in text. Named entities are specific words or phrases that represent entities such as people, organizations, locations, dates, quantities, and other types of proper nouns.

The goal of NER is to automatically locate and categorize these named entities within a given text. This task is crucial in various NLP applications, including information extraction, question answering systems, sentiment analysis, and machine translation. By identifying and classifying named entities, NER enables a deeper understanding of the text and helps extract valuable information from unstructured data.

NER is usually implemented using machine learning techniques, particularly supervised learning algorithms. These algorithms are trained on annotated datasets where human annotators label the named entities in a corpus of text. Various machine learning approaches, such as rule-based methods, Hidden Markov Models (HMMs), Conditional Random Fields (CRFs), and deep learning models like Recurrent Neural Networks (RNNs) and Transformer-based architectures, can be used for NER.

Use Spacy to train custom NER models

Spacy , a popular NLP library, implements Named Entity Recognition (NER) using a combination of rule-based matching and statistical models. The models learn to predict the named entity labels based on the contextual information of the words in a sentence. This library provides the capability to train your own custom NER models using your annotated training data.

Spacy provides pretrained models that are already trained on large corpora and can recognize a wide range of named entities. These models are trained using labeled data and can be conveniently loaded into your application.

This library also allows you to define custom rules to match specific patterns of words or phrases as named entities. These rules can be based on lexical patterns, part-of-speech tags, or other linguistic features. Rule-based matching is useful when you have a small set of specific entities that you want to extract.

Train Custom NER Models

To train a custom NER model in Spacy, you need to provide annotated training data where each entity in the text is labeled with its corresponding entity type. Spacy uses a machine learning algorithm, such as a convolutional neural network (CNN) or a transformer-based architecture, to learn the patterns and features that distinguish different named entity types. Training a custom NER model involves iterating over the training data and optimizing the model’s parameters to improve its entity recognition performance.

Fine-tune and evaluate

After training a custom NER model, you can fine-tune it on your specific domain or task by providing additional labeled examples. Spacy provides evaluation metrics to assess the performance of the model, such as precision, recall, and F1-score, which help measure the model’s ability to correctly identify named entities.

Using the data we have about people's phone numbers and text messages, we need to apply Spacy's pre-trained models to achieve good results with fewer training iterations.

Train your custom named entity recognition model

Google Colab lets you train machine learning models online with great speed. Create a new Google colab notebook through Google's research dashboard and add the following code:

# load spacy
import spacy
nlp = spacy.load("en_core_web_sm")
import spacy.cli"en_core_web_lg")

Now add your training samples. The training samples are stored as tuples, where the first element in the tuple is a string that represents a sample input sentence.

The next element in the tuple is the dictionary with the key “entities”. The entities key has the list of the named entities in the input sentence. The first element is the starting index of the named entity in the string and the second element is the ending index of the named entity in the string, for example, RECIPIENT_PHONE or MESSAGE.

The third element is the actual named entity that the word between those two indices can be classified as. Let's take an example for better understanding.

"("Send a message to +15551234567 with the message 'Hello'", {"entities": [(18, 30, "RECIPIENT_PHONE"), (49, 54, "MESSAGE")]})"

Here "Send a message to +15551234567 with the message 'Hello'" is the sentence. The word from index 18 till before 30, which is +15551234567 is marked as RECIPIENT_PHONE. The word from index 49 till before 54 is Hello, which is marked as the entity MESSAGE.

Add the following sample training data to your Google Colab notebook:

train = [
    ("Send a message to +15551234567 with the message 'Hello'", {"entities": [(18, 30, "RECIPIENT_PHONE"), (49, 54, "MESSAGE")]}),
    ("Send a message to +15551234567 with the message: 'Hello'", {"entities": [(18, 30, "RECIPIENT_PHONE"), (50, 55, "MESSAGE")]}),
    ("Text +447891234567: 'Can you call me?'", {"entities": [(5, 18, "RECIPIENT_PHONE"), (20, 36, "MESSAGE")]}),
    ("Text +447891234567 'Can you call me?'", {"entities": [(5, 18, "RECIPIENT_PHONE"), (23, 36, "MESSAGE")]}),
    ("Send a text to +919876543210: 'I'll be there soon'", {"entities": [(15, 28, "RECIPIENT_PHONE"), (31, 49, "MESSAGE")]}),
    ("Send a text to +919876543210 'I'll be there soon'", {"entities": [(15, 28, "RECIPIENT_PHONE"), (30, 48, "MESSAGE")]}),
    ("Send the message 'Do you have any plans tonight?' to +336987654321.", {"entities": [(18, 48, "MESSAGE"), (53, 66, "RECIPIENT_PHONE")]}),
    ("Send the message: 'Do you have any plans tonight?' to +336987654321.", {"entities": [(19, 49, "MESSAGE"), (54, 67, "RECIPIENT_PHONE")]}),
    ("Text +819876543210: 'Remember to bring your ID.'", {"entities": [(5, 18, "RECIPIENT_PHONE"), (21, 47, "MESSAGE")]}),
    ("Text +819876543210 'Remember to bring your ID.'", {"entities": [(5, 18, "RECIPIENT_PHONE"), (20, 46, "MESSAGE")]}),
    ("Send a message to +4987654321098 with the content: 'Don't forget to buy milk.'", {"entities": [(18, 32, "RECIPIENT_PHONE"), (52, 77, "MESSAGE")]}),
    ("Send a message to +4987654321098 with the content 'Don't forget to buy milk.'", {"entities": [(18, 32, "RECIPIENT_PHONE"), (51, 76, "MESSAGE")]}),
    ("Send a message to +5551987654321: 'I need your help.'", {"entities": [(18, 32, "RECIPIENT_PHONE"), (35, 52, "MESSAGE")]}),
    ("Send a message to +5551987654321 'I need your help.'", {"entities": [(18, 34, "RECIPIENT_PHONE"), (34, 51, "MESSAGE")]}),
    ("Text +447612345678 and ask: 'Have you received the package?", {"entities": [(5, 18, "RECIPIENT_PHONE"), (29, 59, "MESSAGE")]}),
    ("Send a message to +6176543210987 saying 'Congratulations!'", {"entities": [(18, 32, "RECIPIENT_PHONE"), (41, 57, "MESSAGE")]}),
    ("Send the message 'Can we meet tomorrow?' to +639987654321.", {"entities": [(18, 39, "MESSAGE"), (44, 57, "RECIPIENT_PHONE")]}),
    ("Please text +27123456789", {"entities": [(12, 24, "RECIPIENT_PHONE")]}),
    ("Text +441234567890: 'Let's grab dinner tonight.'", {"entities": [(5, 18, "RECIPIENT_PHONE"), (21, 47, "MESSAGE")]}),
    ("Send a message to +4912345678901 with the text 'Sorry for the delay.'", {"entities": [(18, 32, "RECIPIENT_PHONE"), (48, 68, "MESSAGE")]}),

Execute the above two code cells. Next, add the following code to the Google Colab file:

import pandas as pd
import os
from tqdm import tqdm
from spacy.tokens import DocBin
db = DocBin() # create a DocBin object
for text, annot in tqdm(train): # data in previous format
    doc = nlp.make_doc(text) # create doc object from text
    ents = []
    for start, end, label in annot["entities"]: # add character indexes
        span = doc.char_span(start, end, label=label, alignment_mode="contract")
        if span is None:
            print("Skipping entity")
    doc.ents = ents # label the text with the ents
db.to_disk("./train.spacy") # save the docbin object

Next, type in this command and execute it:

!python -m spacy init fill-config base_config.cfg config.cfg

Afterwards, execute this command:

!python -m spacy train config.cfg --output ./output --paths.train ./train.spacy ./train.spacy

Once you have trained this new model to identify named entities in text, you’ll want to test it yourself, with an input sentence. To infer the trained model, execute the below code. You can also replace the sentence in the below code with any other sentence you want to test the model with.

nlp1 = spacy.load(r"./output/model-best") # load the best model
doc = nlp1("Can you send a message to 8769786868 with the text 'Hello, how are you?'") # input sample text

Zip and download your trained model to use in the chatbot application. Be sure to name the zip file as :

!zip -r /content/ /content/output/model-best
from google.colab import files"/content/")

Once you run this cell, the best model will be downloaded on your system. Awesome, you did it!

Next, we will connect this model to our backend Flask API in order to run input sentences and identify named entities. Then use the recipient number and the message to send a message from the Twilio number to the recipient with the corresponding message.

Build the backend REST API with Flask and Python

Inside the Python_Twilio_AI_Messaging_Chatbot directory, create a folder named backend :

mkdir backend
cd backend

Inside this folder, create and activate a new Python virtual environment. Install the dependencies you will need for this project. If you are on a Unix or macOS system, run the following commands to do this:

python3 -m venv venv
source venv/bin/activate
pip install twilio flask flask_apscheduler flask_cors spacy

If you following this tutorial on a Windows machine, run the following commands instead:

python -m venv venv
pip install twilio flask flask_apscheduler flask_cors

Set the following environment variables, replacing the placeholder text with your values for Account SID, Auth Token, and Twilio phone number. You can find these in the Twilio Console :


You previously downloaded your model as into your system at the end of the Train your custom named entity recognition model section. Unzip this file inside the backend folder. This model zipfile is can be referenced from this GitHub link or download it from this Google Drive link.

Create a file called and add the following code . Replace `TWILIO_ACCOUNT_SID`, `TWILIO_AUTH_TOKEN` and `TWILIO_PHONE_NUMBER` with your Twilio account credentials in the code.

import spacy
from flask import Flask, request, jsonify
from twilio.twiml.messaging_response import MessagingResponse
import spacy.cli"en_core_web_lg")
from flask_cors import CORS
app = Flask(__name__)
nlp1 = spacy.load("./content/output/model-best") # load the best model
def generate_bot_response(user_message):
    doc = nlp1(user_message) # input sample text
    # Find named entities, phrases and concepts
    recipient_number = None
    message = None
    for entity in doc.ents:
        print(entity.text, entity.label_)
        if entity.label_== "RECIPIENT_PHONE":
            recipient_number = entity.text
        if entity.label_== "MESSAGE":
            message = entity.text
    if not recipient_number:
        return "Recipient Number not present in message. Please resend."
    if not message:
        return "Couldn't understand what message you want to send. Please resend."
    send_message_return = send_message(recipient_number, message)
    return send_message_return
def send_message(recipient, message):
    # Code for sending the message using Twilio
    # Replace this with your own Twilio implementation
    print("In send message")
    print("Recipient phone number received as", recipient)
    # Here's a sample implementation using the Twilio Python SDK
    from import Client
    # Twilio account credentials
    account_sid = 'TWILIO_ACCOUNT_SID'
    auth_token = 'TWILIO_AUTH_TOKEN'
    twilio_phone_number = 'TWILIO_PHONE_NUMBER'
    # Create a Twilio client
    client = Client(account_sid, auth_token)
    # Send the message using Twilio
        message = client.messages.create(
            body = message,
            from_= twilio_phone_number,
        return f"Message sent to {recipient} successfully!"
    except Exception as e:
        print("in exception")
        print(f"Failed to send message: {str(e)}")
        return "Couldn't send message, please check your Twilio credentials."
@app.route("/sms", methods=['POST'])
def sms_reply():
    data = request.get_json()
    user_message = data.get('Body')
    bot_response = generate_bot_response(user_message)
    response = MessagingResponse()
    return {"message":bot_response}
if __name__ == "__main__":

This code imports necessary libraries and modules including spacy, Flask, request, jsonify, and MessagingResponse from twilio.twiml.messaging_response. The spacy.cli.CORS (Cross-Origin Resource Sharing) is enabled for the Flask app using the flask_cors extension.

The Python Flask script loads a pre-trained spaCy model using the spacy.load function and assigns it to the variable nlp1. The generate_bot_response function takes a user message as input, processes it using the nlp1 model, and extracts relevant information from the named entities detected in the message. It then calls the send_message function with the recipient number and message extracted from the user's input. If an exception occurs while sending the message, it returns a default response.

The send_message function sends a message using the Twilio service. It uses the Twilio Python SDK to send an SMS message to the specified recipient number. It returns a success message if the message is sent successfully, or an error message if there's an exception.

The /sms route is defined for the Flask app, which handles incoming POST requests. It expects a JSON payload with a "Body" field containing the user's message. It calls the generate_bot_response function with the user's message and constructs a Twilio MessagingResponse object. It then returns a JSON response with the bot's response.

The __name__ == "__main__" condition ensures that the Flask app is only run when the script is executed directly, not when it is imported as a module. The app is run in debug mode for development purposes.

Build the chatbot UI with React

Now it's time to create the React frontend for the user to interact with.

Open a new terminal window. In the root directory of the Python_Twilio_AI_Messaging_Chatbot project, create a new React app called chatbot-frontend:

npx create-react-app chatbot-frontend

Once the app has been created, change into the directory, install `react-bootstrap`, and run the app with the following commands:

cd chatbot-frontend
npm install react-bootstrap
npm start

You can download an image from any site such as Rename it to chatbot-image.png  and store it inside the frontend directory. This is for the logo of the chatbot for the UI.

Verify the app is running by visiting http://localhost:3000 in the browser. The default React app will be visible. Next, update the code in the subdirectory  src/App.js  to the following code:

import React, { useState } from "react";
import "./App.css";

function App() {
  const [userInput, setUserInput] = useState("");
  const [chatHistory, setChatHistory] = useState([]);

  const sendMessage = async () => {
    // Send the user's message to the backend API
    const response = await fetch("", {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
      body: JSON.stringify({ Body: userInput }),

    const data = await response.json();

    // Update the chat history with the user's message and bot's response
    setChatHistory((prevHistory) => [
      { sender: "user", message: userInput },
      { sender: "bot", message: data.message },

    // Clear the input field

  return (
    <div className="chat-container">
      <div className="chat-header">
        <img className="chatbot-image" src="chatbot-image.png" alt="Chatbot" />
        <h1>Twilio AI Messenger</h1>
      <div className="chat-body">
        {, index) => (
            className={`message-container ${
              message.sender === "user" ? "user-message" : "bot-message"
      <div className="chat-input">
          onChange={(e) => setUserInput(}
          placeholder="Type your message here..."
        <button onClick={sendMessage}>Send</button>

export default App;

This React component represents a chat interface. The component uses the useState hook from React to define two state variables: userInput and chatHistory.

The sendMessage function is an asynchronous function that sends the user's message to a backend API using the fetch function. It then updates the chat history with the user's message and the bot's response. Finally, it clears the input field.

The component returns JSX, which represents the structure and content of the chat interface. The JSX is wrapped in a div with the class name "chat-container" and includes a chat header section with an image and a heading. The chat body section renders the chat history by mapping over the chatHistory state variable and creating a message container for each message. These containers have a dynamic class name based on the sender.

The JSX object includes a chat input section with an input field and a send button. This is bound to the userInput state variable and updates when the user types in the input field. The send button triggers the sendMessage function when clicked.

Now update the App.css file with the following:

/* App.css */

.chat-container {
  width: 400px;
  margin: 0 auto;
  padding: 20px;
  border: 1px solid #ccc;
  border-radius: 5px;
  box-shadow: 0 2px 5px rgba(0, 0, 0, 0.1);

.chat-header {
  display: flex;
  align-items: center;

.chatbot-image {
  width: 50px;
  height: 50px;
  border-radius: 50%;
  margin-right: 10px;

.chat-body {
  margin-top: 20px;
  display: flex;
  flex-direction: column;
  align-items: flex-start;

.message-container {
  margin-bottom: 10px;

.user-message {
  background-color: #eaf6ff;
  padding: 10px;
  border-radius: 5px;
  align-self: flex-end;

.bot-message {
  background-color: #f6f6f6;
  padding: 10px;
  border-radius: 5px;
  align-self: flex-start;

.chat-input {
  margin-top: 20px;
  display: flex;
  align-items: center;

.chat-input input {
  flex-grow: 1;
  padding: 10px;
  border: 1px solid #ccc;
  border-radius: 5px;
  margin-right: 10px;

.chat-input button {
  padding: 10px 20px;
  background-color: #007bff;
  color: #fff;
  border: none;
  border-radius: 5px;
  cursor: pointer;

.chat-input button:hover {
  background-color: #0069d9;
  • The .chat-container class sets the width, margin, padding, border, border-radius, and box shadow properties for the main container of the chat interface.
  • The .chat-header class sets the display and align-items properties for the header section of the chat interface.
  • The .chatbot-image class sets the width, height, border-radius, and margin properties for the chatbot's image.
  • The .chat-body class sets the margin-top, display, flex-direction, and align-items properties for the body section of the chat interface.
  • The .message-container class sets the margin-bottom property for each message container in the chat interface.
  • The .user-message class sets the background-color, padding, border-radius, and align-self properties for user messages in the chat interface.
  • The .bot-message class sets the background-color, padding, border-radius, and align-self properties for bot messages in the chat interface.
  • The .chat-input class sets the margin-top, display, and align-items properties for the input section of the chat interface.
  • The .chat-input input class sets the flex-grow, padding, border, border-radius, and margin-right properties for the input field in the chat interface.
  • The .chat-input button class sets the padding, background-color, color, border, border-radius, and cursor properties for the send button in the chat interface.
  • The .chat-input button:hover class sets the background-color when the send button is being hovered over.

Now it's time to run both the backend and frontend simultaneously and then test the application!

Test the AI powered messaging app

Run both the backend server and frontend app simultaneously, if they are not still already running from earlier. 

Once you have unzipped the model, open a new terminal. Change into the  backend folder and type:

flask run

The server will run on port 5000:

In another terminal, change into the chatbot-frontend directory and type npm start.

Running react app from terminal
Running react app from terminal

Navigate to http://localhost:3000/ to see the Twilio AI Messenger application.

The image displays the Twilio AI Messenger chatbox. There is a bar to type in your message and a Send button to send it.
The image displays the Twilio AI Messenger chatbox. There is a bar to type in your message and a Send button to send it.

You will first send your message, something like “Can you send the message “Hello, how are you?” to +9197895817457”. You must put your message within quotation marks, or you will run into errors. The chatbot will reply with the status saying “Message sent successfully to recipient +9197895817457 with the message Hello, how are you?”.

To test the application, enter your message in the text box and send it. Make sure the phone number is in E.164 format. Once you send the text to the chatbot, the chatbot will respond if the message was successfully sent or if there was an error. 

You will receive an SMS notification with the text you sent to the chatbot. If the message was successful, the chatbot will respond with "successfully sent the message to [GIVEN_PHONE_NUMBER]" along with the message "Hello, how are you?".

This is the message printed in the backend console:

 the terminal where the backend of the application is running. It is running on port 5000
 the terminal where the backend of the application is running. It is running on port 5000

Whenever we hit the endpoint with some data to send a message, it prints the message text and the recipient's phone number. Once the message is successfully delivered, it prints the reply text that will be sent to the chatbot all well. You can see what calls were made to the API endpoint from these server logs.

Try with different prompts and sentences to test out the Twilio AI messenger:

twilio ai messenger screenshot
twilio ai messenger screenshot

The above image shows a chat with the Twilio AI messenger. It continues from the previous chat where we asked the chatbot to send a message to a certain number. Adding another example, we ask it to send another message to a different number, which it successfully does. The message is “Send the test “Hey what is up” to +919670034500. The reply from the chatbot is “Message sent successfully to recipient +919670034500 with the message Hey what is up? This shows that our model can correctly detect the named entities, message and phone number and send the message successfully!

This is the message printed in the backend console:

displays the server logs for that API call where the message and phone number are printed
displays the server logs for that API call where the message and phone number are printed

A 200 response is displayed along with the server logs for that API call where the message and phone number are printed. The reply that is sent to the chatbot is also logged here on the terminal.

Here are a few more prompt ideas:

displays the server logs for that API call where the message and phone number are printed
displays the server logs for that API call where the message and phone number are printed

The above image displays the Twilio AI Messenger UI and a few prompts that you can test it out with. They include “Send a text to +918795678345 with the message “How are you doing?”. This is just to reaffirm that the model is able to correctly detect the named entities even if we have input sentences which are structured differently.

SMS sent from Twillio Messaging API
SMS sent from Twillio Messaging API

This shows the messages received on my phone screen from the Twilio AI Messenger. I tried with a set of prompts and the screen shows the messages I received. Every message starts with “Send from your Twilio trial account - “ and then the actual message. I sent messages saying “Hello, how are you?”, “Hello, how are you doing today?”.

The above screenshot is some of the messages received with the Twilio AI messenger.

In case you send any messages without the recipient number, this will be the error screenshot:

SMS sent from Twillio Messaging API
SMS sent from Twillio Messaging API

If your input is missing a message to send, the chatbot responds with “Couldn’t understand what message you want to send. Please resend.”

In case there are some issues with the Twilio credentials, this is the response:

screenshot of invalid Twilio credentials and error message
screenshot of invalid Twilio credentials and error message

The Twilio AI Messenger displays a response when the Twilio credentials in the backend are invalid.

What's next for AI messaging apps?

The integration of Spacy NLP, Twilio, and Flask in creating our messaging system has empowered us to forge meaningful connections and deliver a seamless user experience. By leveraging Spacy NLP's entity recognition, Twilio's messaging capabilities, and Flask's backend framework, we have built a powerful application that understands and responds to user input.

Looking forward, there are numerous avenues for enhancing this project. Expanding the application's NLP capabilities by integrating sentiment analysis, intent detection, and language translation can lead to a deeper understanding of user messages and more sophisticated responses. Improving the user interface with features like message threading, emojis, and user profiles, along with real-time updates using web sockets, can create a more dynamic and engaging chat experience for users.