Build a Vocabulary Bot for WhatsApp with Python and Twilio

Build a Vocabulary Bot for WhatsApp with Python and Twilio

My mother teaches English to visually impaired students. Due to the current situation with the Coronavirus, in-person classes had to be suspended, and now she is teaching through phone conference calls, with all learning material being exchanged on WhatsApp. I saw an opportunity to solve a problem here, so I decided to build a Vocabulary chatbot to assist her in her teaching.

This tutorial will show you how to create a WhatsApp chatbot that can help you in improving your vocabulary using the Twilio API for WhatsApp, Python and Flask. Below, you can see the conversation I had with the bot.

Tutorial Requirements

To follow this you need the following components:

• Python 3.6 or newer. If your operating system does not provide a Python interpreter, you can go to python.org to download an installer.
• Flask. We will create a web application that responds to incoming WhatsApp messages with it.
• ngrok. We will use this handy utility to connect the Flask application running on your system to a public URL that Twilio can connect to. This is necessary for the development version of the application because your computer is likely behind a router or firewall, so it isn’t directly reachable on the Internet. If you don’t have ngrok installed, you can download a copy for Windows, MacOS or Linux.
• A smartphone with an active phone number and WhatsApp installed.
• A Twilio account. If you are new to Twilio create a free account now. You can review the features and limitations of a free Twilio account.

Configure the Twilio WhatsApp Sandbox

Twilio provides a WhatsApp sandbox where you can easily develop and test your application. Once your application is complete you can request production access for your Twilio phone number, which requires approval by WhatsApp.

Let’s connect your smartphone to the sandbox. From your Twilio Console, select Programmable SMS and then click on WhatsApp. The WhatsApp sandbox page will show you the sandbox number assigned to your account, and a join code.

To enable the WhatsApp sandbox for your smartphone send a WhatsApp message with the given code to the number assigned to your account. The code is going to begin with the word join, followed by a randomly generated two-word phrase. Shortly after you send the message you should receive a reply from Twilio indicating that your mobile number is connected to the sandbox and can start sending and receiving messages.

Note that this step needs to be repeated for any additional phones you’d like to have connected to your sandbox.

Create a Python Virtual Environment

Following Python best practices, we are going to make a separate directory for our chatbot project, and inside it we are going to create a virtual environment. We then are going to install the Python packages that we need for our chatbot on it.

If you are using a Unix or Mac OS system, open a terminal and enter the following commands to do the tasks described above:

For those of you following the tutorial on Windows, enter the following commands in a command prompt window:

venv creates a virtual environment. vocab-bot is the name of the virtual environment. The last command uses pip, the Python package installer, to install the three packages that we are going to use in this project, which are:

• The Flask framework, to create the web application
• The Twilio Python Helper library, to work with the Twilio APIs
• The Requests package, to access third party APIs
• The Dotenv package, to load environment variables(secret keys) from a .env file.

For your reference, at the time this tutorial was released these were the versions of the above packages and their dependencies tested:

Create a Flask chatbot service

Let’s convert the idea into a working application: VocabBot.

The primary aim of VocabBot is to help you improve your English vocabulary. You can ask the bot word definitions, synonyms, antonyms and examples related to how a word is used.

Webhook

The Twilio API for WhatsApp makes use of webhooks to notify our application whenever our bot receives a message. Let us create a simple function that will respond to our webhook.

We’ll define a webhook using the Python micro-framework Flask.  Create a new file app.py inside of our project directory, with the following code:

Notes about this code example:

1. In the code snippet above, first we import the Flask class.

2. We then create an instance of this Flask class.

3. route() is a decorator to tell Flask what endpoint should trigger our function. Later, we will set the same endpoint as our webhook in the Twilio console.

4. We are receiving the data at the /vocabulary endpoint, using the POST HTTP verb. Each time an incoming message from the end-user is received on the WhatsApp number Twilio in turn invokes the /vocabulary endpoint with the details.

5. We then define the function vocabulary. The main application logic for communication with the Twilio API for WhatsApp goes inside this function.

6. __name__ == “__main__” ensures that the development server is run only when app.py is executed as the main script.

7. To test if our setup is working fine and the development server is running, you can just execute the command python app.py.

Messages and Responses

The message sent by the user is included in the payload of the POST request with a key of Body. In Flask we can access it through the request object.

Notes about this code example:

1. We import the request object from the Flask package.

2. We fetch the message sent by the user using request.values.get(‘Body’,’’).

3. We’ll further be using the incoming message to fetch the results, so to ensure that there is no discrepancy, I am converting the incoming_message to lower case using the string built-in method lower().

The response that Twilio expects from the webhook needs to be given in TwiML or Twilio Markup Language, which is an XML-based language. The Twilio helper library for Python comes with classes that make it easy to create this response without having to create XML directly. Below you can see how to create a response that includes a text component only.

VocabBot Logic

The incoming message string from the user will be in one of the following formats:

1. help - This returns a list of all possible formats users can type the message in.
2. meaning - type the word - Replace type the word with the word whose definition you are searching for. Example: meaning -  excitement.
3. synonyms - type the word - Replace type the word with the word whose synonyms you are searching. Example: synonyms - happy.
4. antonyms - type the word - Replace type the word with the word whose antonyms you are searching. Example: antonyms - happy.
5. examples - type the word - Replace type the word with the word whose examples you are searching. Example: examples - happy.

To handle these inputs, VocabBot will be split into three functions:

• create_help_message - creates and returns a formatted help message when user types help.
• vocabulary - helps in communicating with the Twilio API for WhatsApp and the user.
• get_dictionary_response - queries the data source (API), parses the fetched response and returns synonyms, antonyms, usage examples and meaning of the word.  

Here is the create_help_message function. Add it to app.py.

Now let’s code the complete version of the vocabulary function that we started earlier:

We split the incoming message string based on the dash character (-). If the length of the list created after splitting the incoming message is equal to 1 and the incoming message is help, we return the help message. If the length of the list created after splitting the incoming message is 2, then the word at the 0th index of the list is our query type, which can be message, examples, synonyms or antonyms, while the word at index 1 is the word typed by the user.

If an incoming message is not in the requested format then a generic error message is returned. For fetching and parsing the data returned by the data source we call the function get_dictionary_response, shown in the next section. The response returned by this function is added in the Twilio response message body, which will be delivered to the user on WhatsApp.

Data Source

We will be using Merriam-Webster's Collegiate Thesaurus API to get the definitions, synonyms, antonyms and usage examples for the requested word. You’ll have to sign up to get an API key.

Once you have the API key, you’ll be able to query the Merriam-Webster’s Thesaurus API. To have a look at the sample request URL and sample API response data structure, you can visit the API overview. This API is free for non-commercial use. Usage should not exceed 1000 queries per day per API key.

The API key needs to be sent in the request URL as a query parameter. Create a .env file and add a variable called “KEY_THESAURUS” with your API key.  

In the below code snippet we will define the get_dictionary_response function.

Add the imports and the function from the above snippet to your existing app.py file.

The new imports are:

• os - Part of the Python standard library. We use it to read environment variables. The os.getenv("KEY_THESAURUS") expression returns the API key stored in the .env file.
•  json - Part of the Python standard library. The json.loads(api_response.text) expression converts JSON data in string format to a Python dictionary.
• requests - A Python HTTP library. The requests.get(url) expression sends a GET request to the Merriam-Webster’s API.
• load_dotenv - Loads the variables defined in the .env file into the environment.

When the call to the Merriam-Webster’s Thesaurus API is successful we update the values of the definition, example, synonyms and antonyms variables with the contents of the API response. The API also returns the data of words similar to the queried word, however, we only use the data corresponding to the exact word.

In the API data structure syn stands for synonyms, def stands for definition, ants stands for antonyms, usage examples are present in dt.  For detailed meaning of JSON fields in API response data structure you can read the Merriam-Webster’s Thesaurus API documentation.

Our function get_dictionary_response returns a dictionary containing synonyms, antonyms, usage, and meaning of the requested word. Below is an example usage of this function from a Python shell:

Everything Together

Now you have seen all the aspects of the chatbot implementation, so we are ready to integrate all the pieces into the complete chatbot service. If you haven’t built your application yet, you can copy the code below into the *app.py* file. You can also use the code listing as a reference if you have been building the application incrementally.

Running the application

To start the application, run the following command from the terminal:

Since our application is currently local, there’s no way for Twilio to be able to send POST requests to the endpoint we just created. We will use Ngrok to set up a temporary public URL so that our app is accessible over the web.

On a second terminal go to the location where ngrok is installed and start ngrok at the same port on which app.py is listening, which is 5000.

Note the lines beginning with “Forwarding”. These show the public URL that ngrok uses to redirect requests into our service. What we need to do now is tell Twilio to use this URL to send incoming message notifications.

Go back to the Twilio Console, click on Programmable SMS, then on WhatsApp, and finally on Sandbox. Copy the https:// URL from the ngrok output and then paste it on the “When a message comes in” field. Since our chatbot is exposed under the /vocabulary URL, append that at the end of the root ngrok URL. Make sure the request method is set to HTTP Post. Don’t forget to click the red Save button at the bottom of the page to record these changes.

Testing VocabBot

Now you can start sending messages to the chatbot from the smartphone that you connected to the sandbox. Based on the request format you can start asking the bot word definitions, usage, synonyms, antonyms. Here, is the example session:

Keep in mind that when using ngrok for free there are some limitations. In particular, you cannot hold on to an ngrok URL for more than 8 hours, and the domain name that is assigned to you will be different every time you start the ngrok command. You will need to update the URL in the Twilio Console every time you restart ngrok.

Conclusion

In this tutorial, we have created a WhatsApp chatbot that helps you improve your English vocabulary. This was implemented using Flask and the Twilio API. The data related to dictionary words was fetched from Merriam-Webster’s Thesaurus API. The GitHub repository with the complete code for this project can be found here. 

Mridu Bhatnagar
Blogs: https://dev.to/mridubhatnagar
GitHub: https://github.com/mridubhatnagar
Twitter: https://twitter.com/Mridu__