Python Error Alerting with Twilio and SendGrid

Python Error Alerting with Twilio and SendGrid

Error detection is a key part of any application deployed in the cloud. No matter how much care we give to testing and software quality, there will always be factors - sometimes outside our control - that can make our applications fail.

In this article, we will build a native Python solution that extends the standard logging library to send application failure alerts through Twilio Programmable SMS and/or SendGrid Email APIs.

Requirements

We will be using Python 3.9, but any version of Python from 3.6 should work as well. You can download Python from the official python.org website.

Since our code will use Twilio and SendGrid services to dispatch error alerts, you will need an account in at least one of these services (ideally both). We provide instructions below on how to sign-up or log-in and also collect the data you will need from each account.

Twilio Account

Login to your Twilio account (or sign up for free if you don’t have one) and take note of your Account SID and Auth Token:

Make sure you have a phone number listed in Phone Numbers > Manage Numbers and that the number has messaging enabled. If you don’t have a number, click on the “Buy a number” button at the upper-right corner of the screen to get one.

SendGrid Account

Login to your SendGrid account (or sign up if you don’t have one) and go to Settings > API Keys in the left menu. Click “Create API Key” in the top-right corner. Give the key a name and hit “Create & View”. Take note of the key, since SendGrid won’t display it to you again.

Project set up

Create a directory for the project:

Creating a virtual environment is often good practice, so let’s get this done now:

On a Windows computer, replace the source command in the last line above with:

You should now see a (.env) prefix added to your prompt, which confirms that your virtual environment is fully set up.

Install the following libraries on your virtual environment:

Optionally, pin the installed libraries to a local dependencies file:

The http-logging library is compatible with the native logging library from the Python standard library. It allows us to implement a custom backend to receive error messages, which in our case will send the errors to the Twilio and SendGrid APIs. The async logging handler in this library is similar to Python’s HTTP Handler class, but instead of generating blocking requests, it runs in a background thread to avoid blocking the main program, and also sends logs in batches, can keep a local cache in SQLite, and handles retries in case of remote API failure.

Environment Variables

We will use environment variables to retrieve secret API keys needed by the Async HTTP Logger to communicate with the Twilio and SendGrid backends.

Again, if you would like to use only one of the two services, skip the environment variables related to the other. For example: if you only want to use SMS, skip SENDGRID_SENDER_EMAIL, SENDGRID_API_KEY and ALERT_EMAIL.

Linux and MacOS should support the export command. On Windows, if you use a command prompt, you should use set instead of export. In PowerShell console, use $Env as follows:

When setting phone numbers, make sure to enter the complete number in E.164 format, which includes the plus sign and the country code.

Twilio HTTP Transport

In this section we are going to write a custom HTTP Transport class that will be responsible for communicating with the Twilio and SendGrid APIs.

Create a new Python file called logging_twilio.py. In this file, our custom class will inherit from http_logging.transport.AsyncHttpTransport. Import all required libraries and declare a custom class as indicated below:

We will use some custom attributes that are not part of the parent class implementation. For that, we need to override the constructor method in TwilioHttpTransport:

To customize the class behavior, we need to override its parent send method. It takes an events argument, which will be a list of events logged by our Python apps.

Now let’s implement the send_sms_alert and send_email_alert methods, which will use the Twilio and SendGrid APIs, respectively:

SMS stands for Short Message Service, so we obviously want to keep our alerts short on this channel. Additionally, the purpose is to only raise an alert, and not to provide all the details about the issue. Because of that, we concatenate multiple events (if there is more than one) in a single message, providing only the type of error and summary.

In the email channel, we get more verbose and include the error stack trace and some context information that could help to identify the route cause and fix the issue.

The logger name is sent in both channels. This way we can identify from which application the alerts are coming from.

Now that we have an HTTP Transport class integrated with Twilio and SendGrid, the next requirement is the logic to instantiate a Logger object that relies on the new TwilioHttpTransport facility.

Twilio HTTP Handler

The HTTP Transport class is ready, but it requires a handler class to work properly with the native Python logging machinery.  This should be an instance of the http_logging.AsyncHttpHandler class.

Create a new file called sample_app.py and enter the following code to Instantiate the Twilio HTTP transport and handler classes:

After that, we instantiate a logging.Logger object and add the twilio_handler as its handler:

If you already have a logger object coming from a different package (such as app.logger from the Flask framework, for example), skip the first line above and just use logger.addHandler(twilio_handler), where logger is the Logger object you already have in your application.

Notice that the secrets, phone numbers and email addresses are being retrieved from the environment variables we set at the beginning of this tutorial. This provides flexibility in case we want to use this code in multiple projects. It also avoids hardcoding API secrets, which is not a good idea.

Multiple handlers

The Python logging package is very powerful. The logging.Logger class is flexible enough to be extended with multiple handlers.

As explained above, the TwilioHttpTransport class will send minimal information about logs due to SMS limitations. Nonetheless, in the event of an error that requires further debugging, we certainly will want to grab the entire stack trace, information about which line of code failed, exact timestamps, etc. Although this will be sent in email messages, it is also advisable to keep the information in the local logs.

We can accomplish that by using the Logger.addHandler and adding one (or more) handlers to the Logger object.

For example, to send logs not only to our phone and email address, but also to the console, we may use the logging.StreamHandler, as demonstrated below:

Anything logged with the above logger object will be printed to the console and sent to our phone and email address through the Twilio and SendGrid APIs.

A logging.FileHandler may be used to store logs in the local filesystem if that makes sense. You could also use the same http_logging.AsyncHttpHandler again, but in this case, sending logs to a different backend host apart from Twilio and SendGrid.

Testing With a Sample Application

To test our new async error alerting system, add the following lines at the end of the sample_app.py script to trigger some log messages and an intentional error:

In the console, run this script with:

The following output should be printed to the console:

If you have everything set up correctly, you should receive SMS and email messages similar to the screenshots below. If you did not set up the options for one of the services (Twilio SMS or SendGrid Email) it will be ignored by the error alerting code.

Notice that the debug message 'Debugging...' was not printed to the console nor concatenated into the SMS and Email messages. That is because the default log level in the Python logging library is WARNING. The DEBUG level is lower than WARNING and, thus, discarded.

If you want DEBUG level messages to also be captured, set the level accordingly as shown below:

Despite our logger reliance on a custom handler (http_logging.AsyncHttpHandler) and a custom Transport (logging_twilio.TwilioHttpTransport) class, it behaves just like any other Python Logger object.

This makes it fully compatible as a drop-in replacement for any Python project you currently have, in case you’d like to integrate the SMS and Email alerting mechanism we’ve just developed throughout your stack and in any future project.

Wrapping Up

We’re done with our simple, yet powerful, Python alerting tool using SMS and Email, powered by the Twilio and SendGrid APIs. Since it is based on the native Python logging facility, we can use the same Python API we’re used to and easily integrate it in any project. One advantage is that it has no fixed costs, we are only charged when an SMS or Email message is actually sent.

A cache of logs is stored locally by the http-logging library. In the event that the external APIs or the cell phone carrier experience a downtime or network instability, our logger will retry sending the SMS and Email alerts later.

I’m a backend software developer and a father of two amazing kids who won’t let me sleep so that I can enjoy spending nights connecting APIs around. Keep track of other projects fresh from dawn on my Github profile. My direct contact and other social links in byrro.dev.