Using the Twilio Python Helper Library in your Async Applications

September 15, 2020
Written by
Reviewed by
Diane Phan

Using the Twilio Python Helper Library in your Async Applications

If you are using an asynchronous web server with your Python application, you are bound by the first rule of async development, which is to never call functions that block. So what do you do when you need to use packages such as the Twilio Python Helper Library, which has no asynchronous version?

In this article I’m going to show you some of the options that you have to safely integrate Twilio and similar clients with your async application.

What is the problem?

Before I go into the solutions, I thought it would be good to give a quick review of the issues with blocking in asynchronous applications.

At the core of every async application there is the loop, an efficient task manager and scheduler that ensures that the CPU is shared as fairly as possible among all the running tasks. The type of multitasking used by async applications is cooperative, which means that tasks have to get a little bit of work done and then voluntarily suspend and return control to the loop.

The most common place where tasks can suspend is when they need to wait for I/O. Good examples of this are querying a database or sending an HTTP request to a third-party service. Right after these operations are issued, the task has nothing to do other than wait for a response from the other side, so at this point the task tells the loop what it is waiting for and returns control to it, so that it can find another task that can use the CPU in the meantime.

If you get a task that runs for too long without suspending and returning control to the loop, then all the other tasks starve. In a web application, this would mean that the web server would be completely blocked and unable to accept new requests, or even make progress on current ones, until the rogue task releases the CPU to the loop.

Often developers of async applications do not realize that they are calling blocking functions, and inadvertently introduce periods of task starvation, which greatly affect the performance and responsiveness of the application. This would happen if you use the Twilio client for Python, because this client uses the Python requests library to send HTTP requests, and this library is not compatible with asynchronous applications.

Below you can learn about four different ways to use Twilio services from your async application in Python without blocking.

Solution #1: Sending raw HTTP requests

A solution to this problem is to not use the blocking Twilio client, and instead send raw HTTP requests using an async HTTP client library.

From the Twilio documentation, we learn that to send an SMS we have to send the following request:

curl -X POST<TWILIO_ACCOUNT_SID>/Messages.json \
--data-urlencode "Body=Hi there" \
--data-urlencode "From=<TWILIO_PHONE_NUMBER>" \
--data-urlencode "To=<YOUR_PHONE_NUMBER>" \

We can take this curl example and translate it to Python, using one of the asynchronous HTTP clients. Using aiohttp, we can write an module as follows:

import os
import aiohttp

account_sid = os.environ.get('TWILIO_ACCOUNT_SID')
auth_token = os.environ.get('TWILIO_AUTH_TOKEN')

async def send_sms(from_, to, body):
    auth = aiohttp.BasicAuth(login=account_sid, password=auth_token)
    async with aiohttp.ClientSession(auth=aiohttp.BasicAuth(
            login=account_sid, password=auth_token)) as session:
        return await
            data={'From': from_, 'To': to, 'Body': body})

And now we are ready to integrate this function with an asyncio application:

import asyncio
from async_sms import send_sms

async def my_app():
    await send_sms(from_='+<TWILIO_PHONE_NUMBER>, to='+<YOUR_PHONE_NUMBER>, body='Hi there')

If you plan on testing the above example, remember that you have to set the TWILIO_ACCOUNT_SID and TWILIO_AUTH_TOKEN variables in the environment. You can find the values that apply to your Twilio account in the Twilio Console.

This solution can be extended to support any functionality supported by the Twilio client, but of course this is somewhat laborious, as you will need to browse through the documentation to find the correct HTTP requests that map to the functions that you want to implement.

Solution #2: Monkey-patching

This solution only applies if your application is based on the Gevent or Eventlet async frameworks. If your application is based on asyncio or a derived framework, then this one is not for you so skip ahead.

Monkey-patching is a feature of the Gevent and Eventlet frameworks that replaces the blocking functions in the Python standard library with equivalent asynchronous versions. The low-level socket and networking functions are patched so that whenever a call is made that requires waiting for a response the task is suspended and other tasks get the chance to run.

Consider the following standard Python module that sends an SMS with the Twilio client:

import os
from import Client

account_sid = os.environ.get('TWILIO_ACCOUNT_SID')
auth_token = os.environ.get('TWILIO_AUTH_TOKEN')

client = Client(account_sid, auth_token)

def send_sms(from_, to, body):
    return client.messages.create(from_=from_, to=to, body=body)

In a traditional Python application you would use this function as follows:

from sms import send_sms

def my_app():
    send_sms(from_='+<TWILIO_PHONE_NUMBER>', to='+<YOUR_PHONE_NUMBER>', body='Hi there')


If you want to make this application work under Gevent, you could do this:

from gevent import monkey

from sms import send_sms

def my_app():
    send_sms(from_='+<TWILIO_PHONE_NUMBER>', to='+<YOUR_PHONE_NUMBER>', body='Hi there')


The following version is similar, but it applies to Eventlet:

import eventlet

from sms import send_sms

def my_app():
    send_sms(from_='+<TWILIO_PHONE_NUMBER>', to='+<YOUR_PHONE_NUMBER>', body='Hi there')


As you can see, the only difference is that at the very beginning we call the appropriate monkey-patching function that installs a “fix” in the networking code that enables it to work with the async loop.

Solution #3: Using a thread executor

The asyncio package provides a cool trick to be able to run blocking code safely. The idea is to execute the blocking function in a separate thread, so that it does not affect the loop.

For this solution you can use the original send_sms() function from I have shown above, but instead of executing it directly we use the run_in_executor() function to send it to run in a thread pool:

import asyncio
from sms import send_sms

async def my_app():
    await asyncio.get_event_loop().run_in_executor(
        None, send_sms, '+<TWILIO_PHONE_NUMBER>', '+<YOUR_PHONE_NUMBER>', 'Hi there')

The first argument to run_in_executor() is a concurrent.futures.ThreadPoolExecutor instance that can be configured to your liking. If None is provided, then an executor with default options is created the first time the function is called and used from then on.

Solution #4: Using greenletio

The final solution is somewhat similar to the monkey-patching option available to Gevent and Eventlet users, but for the asyncio package. This is based on the greenletio package, which allows some blocking functions to be used in an asynchronous environment through monkey-patching of low-level blocking code in the Python standard library.

The implementation uses the patch_blocking() function to wrap the import of our SMS sending function. This makes sure that any I/O accesses issued by this function are patched with asyncio friendly equivalent functions. To complete the conversion, the async_() wrapper is applied to send_sms() so that it becomes awaitable.

import asyncio
from greenletio import patch_blocking, async_
with patch_blocking():
    from sms import send_sms

async def my_app():
    return await async_(send_sms)(
        from_='+<YOUR_TWILIO_NUMBER>', to='+<YOUR_PHONE_NUMBER>', body='Hi there')

Note that the greenletio package is fairly new. It is perfectly fine to use in your personal projects, but not something I would recommend as a production-ready solution as of yet.


So there you have it, four different ways to use a blocking Python library in an async application!

Once again I’d like to emphasize that these four solutions apply not only to the Twilio client, but also to any other packages that issue network requests to other services such as databases or APIs.

I hope you enjoyed this article and learned a few new ways to unblock your async applications!

Miguel Grinberg is a Python Developer for Technical Content at Twilio. Reach out to him at mgrinberg [at] twilio [dot] com if you have a cool Python project you’d like to share on this blog!