Receive and Reply to SMS and MMS Messages in Python with Amazon Lambda
Today we're going to get to "Hello, World!" on a serverless Twilio application using Amazon API Gateway, Amazon Lambda, and Python. By the end of this guide, you'll have the roots of your next world-changing Twilio Application. By the end of this paragraph you'll have a promise - you won't need to spin up a single VPS.
Sound exciting? Indubitably. Let's get started!
Oh, and if you haven't yet, please log into or create an AWS Account, and log into or create a Twilio Account. Don't worry, we'll be here when you're ready.
Tutorial Requirements
- An AWS account. If you don’t have an AWS account yet, sign up for an account for free and enjoy a generous allowance during your first year.
- A Twilio account. If you are new to Twilio you can create a trial account and start developing for free. (Please review the features and limitations of a free Twilio account.)
Twilio can send your web application an HTTP request when certain events happen, such as an incoming text message to one of your Twilio phone numbers. These requests are called webhooks, or status callbacks. For more, check out our guide to Getting Started with Twilio Webhooks. Find other webhook pages, such as a security guide and an FAQ in the Webhooks section of the docs.
What is a Webhook?
Webhooks are user-defined HTTP callbacks. They are usually triggered by some event, such as receiving an SMS message or an incoming phone call. When that event occurs, Twilio makes an HTTP request (usually a POST or a GET) to the URL configured for the webhook.
To handle a webhook, you only need to build a small web application that can accept the HTTP requests. Almost all server-side programming languages offer some framework for you to do this. Examples across languages include ASP.NET MVC for C#, Servlets and Spark for Java, Express for Node.js, Django and Flask for Python, and Rails and Sinatra for Ruby. PHP has its own web app framework built in, although frameworks like Laravel, Symfony and Yii are also popular.
Whichever framework and language you choose, webhooks function the same for every Twilio application. They will make an HTTP request to a URI that you provide to Twilio. Your application performs whatever logic you feel necessary - read/write from a database, integrate with another API or perform some computation - then replies to Twilio with a TwiML response with the instructions you want Twilio to perform.
Using Amazon Lambda with Twilio and Amazon API Gateway
Although the piece that will eventually be exposed to the world will belong to API Gateway, it's conceptually easier to start with code on Lambda. Since this may feel backwards to you, be prepared for some forward references in this section. This will all become clear when we expose everything to the world from API Gateway.
Create an Amazon Lambda Function
- From your choice of Amazon region, create a new Lambda function from your Lambda Console using the "microservice-http-endpoint-python" blueprint.
Make sure that the console is set to your preferred region. This is important because Lambda functions are only available within the context of the selected region. Most people choose a region that is geographically close to where they live.
This "microservice-http-endpoint-python" blueprint gets us to starting position with our lambda_function set up with the proper function signature to take input from API Gateway (eventually).
- Name your function replyMessages
- Select the basic Lambda permissions execution role (or use an existing role that you have set up already in the AWS IAM console)
- Remove the 'API Gateway' trigger by hitting the Remove button. We will be adding our own trigger when we are done with the code.
- Click the Create function button to create your function. We'll edit the code in the next step.
- In the code editor, replace the template code with the following lines of code.
Enter this code in its entirety into the inline editor:
Our entire "Hello, World!" Application fits in 6 lines of code. Simply: we're going to import Python 3's print, print out all of the headers and HTTP parameters we've received, then return TwiML.
Accessing Request Headers and Body Parameters
All of the POSTed HTTP parameters and any Headers we choose to include will be inside the event dictionary.
Although it appears to be magic at this point, there is a two-step process in API Gateway to prepare event for our consumption. API Gateway expects JSON input, so we'll be using a Body Mapping and splitting on ampersand characters to prepare the dictionary for Python. (This comes in the next section.)
print("Received event: " + str(event))There is some more magic in this line - print is redirected automatically to Amazon Cloudwatch. When our role and policies are properly setup, invocations of this function and any output will be found in CloudWatch.
Generating TwiML Manually with Lambda and Python
TwiML, Twilio's XML Markup Language, is incredibly powerful and lets you easily instruct Twilio on how to handle incoming actions. We're only using a tiny subset of TwiML today in order to respond to a message with a message of our own.
return '<?xml version=\"1.0\" encoding=\"UTF-8\"?>'\
'<Response><Message><Body>Hello world! -Lambda</Body></Message></Response>'With this string, we are first informing Twilio that we are responding with XML encoded in UTF-8. We then set up a Message tag nested inside a Response tag; the Message tag informs Twilio that we would like to reply with the contents of the <Body> tag (in this case, by SMS).
Responding with Media (MMS Messages)
TwiML has the flexibility to return media in your replies to incoming messages. Add a <Media> noun inside of the <Message>, and you'll be peppering your message with pngs in no time (yes, other image formats work as well).
Is your picture worth 1,000 words but you need 5,000? No problem - Twilio allows up to 10 media items per response.
Basic Lambda Settings
Under Basic settings, feel free to set your Memory (MB) to the minimum of 128 MB. We've personally used a timeout of 10 seconds without issue.
In production, you'll want to revisit these limits; note how Lambda's pricing changes based upon your selections.
- Hit the Save button to save any changes you've made.
Test Our Lambda Function
At this point feel free to 'Test' the function using the blue button at top; regardless of input you should see this result (including double quotes):
"<?xml version=\"1.0\" encoding=\"UTF-8\"?><Response><Message>Hello world! -Lambda</Message></Response>"
Setting Up Amazon API Gateway
Your lambda function now has resources allocated, policies and roles defined, and responds beautifully - our next step is to find a way to let the world speak with it. That's where Amazon API Gateway comes into play. We're going to use the Lambda/API Gateway integration to set up a path from Twilio to the Lambda function we just defined.
- In the AWS API Gateway's Console and your choice of region (It doesn't need to match the Lambda function's region), select the Create API button.
- Select the REST API option
(If you have no APIs in the current region, click on the Build button in under the REST API option.)
- Fill out an API name and Description. We're naming our API 'twilio_simple_responses' with a Description of 'Respond to incoming Twilio messages' and 'Regional' Endpoint Type.
- Hit the Create API button.
Create and configure your Resources
Now that we've created our new twilio_simple_responses REST API, we can create and configure it to play nicely with the replyMessages Lambda function that we created in the previous section.
- In the 'Actions' pull-down, you'll want to select Create Resource from the dropdown menu:
- Name your resource something like 'Message Responder', and use a Resource Path of
/message. Then hit the Create Resource button.
- From the dropdown, select Create Method, select POST, and hit the check mark (✓) to save.
- On the set-up screen, select 'Lambda Function' integration without Proxy integration, and select the region and name of your Lambda Function (We used replyMessages.).Note that the 'Lambda Function' field will automatically search as you type - you likely only need to enter the first few characters to find your function:
After saving, click 'Okay' on the granting permissions pop-up and you'll be ready to integrate Lambda!
We're going to take the setup one step at a time to properly integrate our function with API Gateway. Note that we have have to make several configuration changes in the API Gateway console because API Gateway is designed for JSON input and output. We need to work around that in order for Twilio, which posts a form and expects XML, to understand our response.
Integrating Our Request with our Lambda Function
Although our function is bare-bones now, most applications will eventually need some logic based upon incoming messages. We need to map a content type of application/x-www-form-urlencoded into JSON to work properly with API Gateway and Lambda.
- Click on Integration Request, then expand Body Mapping Techniques.
- Click Add mapping template and inserting application/x-www-form-urlencoded (without quotation marks)
- Make sure to hit the check mark (✓) to save! There will be a pop-up asking you to secure this mapping; confirm you would like to secure it.
API Gateway will change your 'Request Body Passthrough' to 'When there are no templates defined (recommended)' (if it does not or you cancel the request move it now)
- Enter this code into the editor which appears under the Content-Type box:
#set($httpPost = $input.path('$').split("&"))
{
#foreach( $kvPair in $httpPost )
#set($kvTokenised = $kvPair.split("="))
#if( $kvTokenised.size() > 1 )
"$kvTokenised[0]" : "$kvTokenised[1]"#if( $foreach.hasNext ),#end
#else
"$kvTokenised[0]" : ""#if( $foreach.hasNext ),#end
#end
#end
}This code, which was first suggested by avilewin in the AWS Developer Forums., splits our HTTP parameters into JSON key/value pairs.
- Finally, go back to 'Method Execution' by following the link at the top of the frame. (You may need to scroll up.)
Integrating Our Lambda Response with API Gateway
We need to do some plumbing from the output of our Lambda function. In a traditional API you'd want to properly use HTTP status codes, you'll generally only want to respond with a 200 and sometimes a 401 to Twilio's request. Additionally, API Gateway is set up to respond with a 'Content-Type' of application/json, while Twilio expects application/xml. Let's fix that now for the 200 case.
- Click on the Integration Response link.
- Expand the 200 Response, then expand the Body Mapping Templates.
- If there is an 'application/json' entry, remove that now.
- Select Add mapping template and add
application/xml. - In the text box, enter this mapping:
$input.path('$')Essentially, we are only echoing the return value of the Lambda function. This will also take care of the surrounding double quotes (") in the return string.
- Save the mapping, and return to Method Execution page with the link at the top of the screen.
Configuring our API Gateway Response
Next, we'll update the response that our twilio_simple_responses API sends back to Twilio. Instead of JSON (the default), we want to send back XML.
- Click the Method Response link.
- Under Response Body for 200, if
application/jsonis defined, remove that now. - Then add
application/xmlwith an 'Empty' model. - Don't forget to click the check mark (✓) to save your changes.
- Return to Method Execution page with the link at the top of the screen.
Deploying Our API and Choosing a Stage
We're almost there now. The last thing we'll want to do with our API Gateway is deply it.
- From the Actions menu, select Deploy API:
You'll then be asked which stage to deploy to. If you don't yet have one created, you can create a new stage from the menu.
- Name your stage 'prod':
- Expand the 'prod' stage which appears (or surf to Stages in the sidebar), and click on 'POST' under '/' and '/message'.
- At the top, you'll see an Invoke URL.
- Copy that URL.
With a full clipboard we're 2/3 of the way to a serverless Twilio application! Let's push on.
Configure Your Webhook URL
On the Twilio side of things, we need to tell Twilio where to make a request when it receives an SMS at your Twilio phone number.
- From the Twilio Console, navigate to the Numbers Section in the sidebar (#).
- Select the number you'd like to route to our new Lambda function.
- Under Messaging and in A Message Comes In, select the Webhook option, and paste the API Gateway URL into the text box (highlighted below). Ensure HTTP POST is selected.
Backup Webhook URL
You'll also notice the 'Primary Handler Fails' box. In production, you may want to have a secondary handler for incoming messages. Twilio will automatically fail-over to the secondary handler if it can't reach the primary handler in 15 seconds or if there is some other problem. See our Availability and Reliability guide for more details.
Protect your webhooks
Twilio supports HTTP Basic and Digest Authentication. Authentication allows you to password protect your TwiML URLs on your web server so that only you and Twilio can access them.
Learn more about HTTP authentication and validating incoming requests here. You can also check out our full guide on securing your Python Amazon Lambda app.
Text Your Twilio Number
And with that, all of the plumbing is complete! You now have Twilio watching for incoming messages, API Gateway listening for requests from Twilio, and Python logic in Lambda listening for API Gateway. Try texting your Twilio number to verify that the world still wants to say hello to you.
What's Next? Making a Large Delta with Twilio and Lambda
We highly suggest that you next visit our guide on validating incoming Twilio requests with Python on Lambda. In that article we'll explore some more advanced features of Lambda such as loading our Python Helper library and defining Environmental Variables, along with demonstrating some simple phone number checks and validating Twilio requests.
If you're ready to press on, Add-Ons will often simplify your next steps. Add-ons make the Twilio platform even more powerful by allowing you to integrate some amazing tools and services from our partners into your application.
Whatever you build, we're here to help - and to celebrate when you deploy. Let us know when you hit the deploy button by messaging us on Twitter!
Need some help?
We all do sometimes; code is hard. Get help now from our support team, or lean on the wisdom of the crowd by visiting Twilio's Stack Overflow Collective or browsing the Twilio tag on Stack Overflow.
