Build Your Own IVR with AWS Lambda, Amazon API Gateway and Twilio

My stomach is filled with butterflies. My mind is racing:

“What could it be? I think I have an idea… but maybe I’m wrong. What am I waiting for?!?”

Nothing quite tops the feelings you experience when someone gives you a gift. Our good friends at Amazon Web Services (AWS) stopped by the greatest city in the world for AWS Summit New York in July and dropped some super rad gifts on the audience. Of course, their gifts came in the form of developer tools that help make building and shipping software easier.

Of all the things AWS launched I am most excited about the Amazon API Gateway. Using the API Gateway we can now invoke Lambda functions when phone calls or text messages come into Twilio. Today, to help you see how it all works, we’re going to build an IVR (or “phone tree” for those of you not up on your telecom jargon) using AWS Lambda, the Amazon API Gateway and Twilio.

Our Tools

Look, No (Managing) Infrastructure!
We’ll start off our application by writing a couple Lambda functions. AWS Lambda is code that runs without you needing to put server infrastructure in place. The code will spin up, run in response to an event and then disappear until you need it again. What kind of events can trigger Lambda functions? Things like an image uploaded to your S3 bucket, a new entry in Dynamo DB, or a click on your website. And now, thanks to the API Gateway, a function can be invoked via an incoming HTTP request.

Let’s start building our application. Go to the AWS Lambda dashboard and click “Get Started Now”. The first thing we do when creating a Lambda function is pick a blueprint, we’re going to keep it simple and select the hello-world blueprint.

We’ll then be given the option to configure triggers. For this function, we don’t need to configure any triggers so we can just press the next button.

Now we’re going to set up our first function. We’ll name it “prompt”, this is the function we’ll invoke when a phone call comes into Twilio. It will prompt the caller to pick one of the options on our IVR.

We can pick between Node.js and Java for the run time of this function. I’m a big fan of Node.js so let’s select that.

With everything else in place we can now write our Lambda function. We want this function to provide two things:

  1. The message we want Twilio to say to someone when they call our number
  2. The options that a caller will be able to select when they call our IVR

When someone calls our number they’re going to have 3 options. They can call me, they can join a conference line or they can “get jiggy with it”.

Before we can create our function we need to assign it a role. The suggested role of lambda_basic_execution will work for us. If this is your first time creating a Lambda function then selecting this role will open a new window asking for some extra permissions. Just click “Allow” and you’ll be good to go. We’ll keep the Amazon defaults for all the other options for our Lambda function.

With our code in place, and our role properly set, we can click “Next” and then “Create Function”.

Now that our function has been created we can invoke it once and see that it returns our data by clicking the “Test” button. Clicking this button will prompt for some sample data to pass, we can leave the default data for this test:

We need to create one other Lambda function to take whatever option the user selects when prompted and return the information Twilio needs about that selection. Just like our prompt function we’ll select the hello-world blueprint with no trigger. Name your new function, takeAction. We’ll need to look at the input a caller entered and take the appropriate action:

Our options here contain the TwiML required to take each action the user could have selected. If you haven’t worked with TwiML before it’s how you give Twilio instructions of what to do with an incoming phone call or text message. It’s really just XML but TwiML is a much more fun word to say.

Whenever a phone call comes into our Twilio phone number Twilio will make an HTTP POST request to our server with some data about the phone call. This data comes through in the POST body as a query string so we also add a function (parseQuery) to parse out the data from our query string.

In this code we’re using the digits property of the events object. Any parameters passed to our Lambda function will be passed in the events object. We can use this to tell what digit our user pressed when prompted by the previous function.

With our code in place you can set the role to lambda_basic_execution, click next, and create our function.

This time when we invoke this function we want to invoke it with some test data. After you click “Test” update the “Input sample event” to pass digits:

With that sample input data in place we can test our function and see that it returns the correct action to take.

There’s Always Money in the Banana Stand
Right now, we can’t invoke our Lambda function via HTTP. That’s where Amazon’s API Gateway comes into play. The API Gateway let’s us create an endpoint that we can make a request to will invoke our function.

On the API Gateway dashboard click “Get Started”, select the option for “New API” and then give your new API a name. “My IVR” will work if you’re not feeling inspired.

We now have the world’s most basic API. So basic that it doesn’t actually do anything. We should probably fix that. Luckily for us, it won’t take too long.

To kick things off, we want to add a new resource. A resource represents an endpoint on our API. The location that we’re going to make a request to when we want to invoke our Lambda function. We can give our resource a name of prompt and then click “Create Resource”.

A resource by itself isn’t super useful, we need to add a method to it so the magic can really happen. A method is the HTTP method we want to be able to interact with this resource. Now we’ll add a POST method. We want the Integration Type to be Lambda Function. Pick the region you created your Lambda function in and then pick your prompt function. When prompted to add permissions to your Lambda Function click “Allow”.

Before we go any further we want to test our function. Click the “Test” button:

Looking good! One last thing, I mentioned before, that Twilio gets basic instructions of what to do with a phone call by receiving TwiML from your server. You’ll notice our request is currently returning JSON. It’s ok, we can change that! Click on the Integration Response (it’s the box in the bottom left). We need to edit the Mapping Template for our Default mapping. There should only be one row in the table showing our integration responses, click the arrow next to that row. Next click application/json under “Body Mapping Templates”:


Now that we’ve indicated we’ll be using a template we can define the format:

This code is a bit dense so let’s step through it:

  1. On the first line we’re getting the data returned from our Lambda function and storing it in the $inputRoot variable. You can read a bit more about interacting with the input and how this works in the AWS API Gateway documentation.
  2. Next we’re creating a new variable called $i and setting it’s value to 1. We’ll use this variable when we prompt users to select an option.
  3. On line 3 our TwiML beginsby identifying that it is a <Response> to Twilio’s HTTP request.
  4. The fourth line introduces the TwiML verb <Gather> which allows us to capture any digits the caller presses. We pass <Gather> two arguments:
  1. numDigits of 1 to indicate we only need to capture the first digit they enter
  2. action of “/takeAction”. action tells Twilio where to POST to after the <Gather> is over.
  1. Nested within our <Gather> verb, the <Say> verb joins the party. <Say> gives us access to a text to speech engine. We’re passing our <Say> verb the message returned by our Lambda function.
  2. On line 6 we kick off a loop over all the options we got back from our Lambda function.
  3. Our friend <Say> returns on line 7 as we tell the caller what digit to press for each specific option.
  4. We then increment our $i variable so caller is given the correct digit to press.
  5. We close out our loop and the XML elements we’ve opened.

Once you’ve added this code save the Template Mapping by clicking the “Save” button for both the template we just added and Integration Response. as whole. Test the function again and you’ll see our response is now TwiML. Beautiful!

We’re getting close to a working application. We just need to do a couple more things to make sure the API Gateway works with Twilio. First, we need to tell our endpoint how to handle an incoming request with the Content-Type of “application/x-www-form-urlencoded”. Let’s go to the Integration Request for this resource and add a Mapping Template for this Content-Type:
lambda-prompt-setup

Our template is going to be very basic:

The other thing we need to do is switch the Content-Type on our Method Response from application/json to application/xml:

Now we need to repeat the same steps for our takeAction function. Create a new resource at the top level of the API, a new POST method for that resource, and set that method to invoke takeAction. Also update the Integration Request and Method Response in the same way. If you forgot how to do all of this just jump back up to the instructions you followed for the prompt resource.

The one thing different between takeAction and prompt is the template for our Integration Response:

Since our Lambda function is returning the appropriate TwiML based on the number entered by a user we just need to output the result of our lambda function which is stored in inputRoot.

Now that our API is in place we need to deploy it. Click “Action” button and select “Deploy API” button. It’ll ask you to create a new deployment stage. Let’s call it dev. Make note of the URL that Amazon gives your API. It should look something like this:

Hi, May I Help You?
We’re almost done, we just need to set up a Twilio phone number. Head over to the “Buy a Number” page. Get a new number that looks nice to you.

Once you’ve got your number set the Voice Request URL to the URL that Amazon gave your API. Then make sure to append your resource (/prompt):

And then?
We’ve now built an IVR that scales with you thanks to Lambda, the Amazon API Gateway and Twilio. Take a look at the other TwiML verbs and see what other things you could when people call in.

Have any questions or want to show me something sick you’ve hacked together with Lambda and Twilio? Find me on twitter (@rickyrobinett) or say “annyong” in the comments below.

  • mipo

    endless loop in parseQuery

    • mipo

      also in template is missing ‘+’on row 8

      • rg33

        I think you meant missing ++ on row 18 of the takeAction function?

        for (var i = 0; i < a.length; i++) {

  • Himanshu Sachdeva

    Another error: Integration request content type should be only “application/x-www-form-urlencoded

    • Pierre Jodouin

      Thank you!

  • obiefernandez

    AWS Lambda includes standard NodeJS packages. Save yourself some trouble using the querystring package instead of rolling your own parseQuery function.

  • If anyone is having trouble with the Integration Request & receiving ‘Could not parse request body into json’ responses from API Gateway, check out this post – https://forums.aws.amazon.com/thread.jspa?messageID=663593 – it has code that will convert the posted data to json.

  • Alex Welch

    When setting up the mapping template I am getting a 500 error. Looking at the stack trace it looks like my messages get processed by prompt correctly.. but i won’t return it. I am using the exact code you have in the example. Is the error in that I have to change the string in the action call?

    • Alex Welch

      Found the error. There is a random 1 in line 8. #set($i = $i 1)

  • Gman

    Straight from a blog on Twilio’s site:

    https://www.twilio.com/blog/2013/08/best-practices-for-securing-your-twilio-app.html

    … which states, “Twilio’s API supports SSL for all communications, and we strongly recommend that you do not send your account credentials via HTTP to port 80. Instead, use SSL and send credentials via HTTPS on port 443. To use SSL, simply use HTTPS to connect to Twilio.”

    … at the same time Twilio says on another blog …

    https://www.twilio.com/blog/2015/09/build-your-own-ivr-with-aws-lambda-amazon-api-gateway-and-twilio.html

    … which states, “we’re going to build an IVR … using AWS Lambda, the Amazon API Gateway and Twilio … And now, thanks to the API Gateway, a function can be invoked via an incoming HTTP request.”

    My question is, which is it? Should we be concerned with exposing our credentials with every PUT/GET through HTTP as suggested on the first blog, or NOT be too concerned as implied in the second blog?

    Just curious.

    • Hey Gman, we absolutely recommend using a secure connection between your application and Twilio.

      You’ll notice that the URL we use from AWS at the end of this article is an HTTPS one too.

      When we talk about using HTTP requests we do mean a secure request. On an application level, an HTTP and an HTTPS request are effectively the same, so we write applications that respond to HTTP requests, however in production we always use and recommend HTTPS connections. You should make sure to keep your credentials secure, otherwise they can be intercepted and abused, so do be concerned! Sorry this has seemed contradictory, but HTTPS is the recommendation.

      • Gman

        Thank you for the speedy clarification!

  • Dan Bryan

    Hi, thanks for the post, this has helped me get started using Lambda. Is there any reason why you’ve chosen to use mapping templates rather than use the Twilio helper library to generate the TwiML and pass it straight through the API Gateway? (it looks as though this would make the logic for the generation ofTwiML more flexible).

    It seems to be good practise to check that requests are being sent from the Twilio servers (https://www.twilio.com/docs/api/security), do you have an example of how this can be done on Lambda?

    Thank you
    Dan

    • Doug Harling

      Dan, did you get this to work? This is how I did it, using the Twilio helper library. I can see the valid TwiML when performing a test from my AWS API. However, I can’t ever get it to work when trying from my phone. I keep getting an error in my Twilio logs that says it can’t parse the request body into json.

      I’m embarrassed to say how long this has taken me. Functionally this is as simple as I can get and it’s taken me days to figure this out.

      Here is the log to my test from my AWS endpoint test. I hope it rings a bell and you (or anyone) can help me out. Thanks!

      Execution log for request test-request
      Sun Jul 17 02:55:16 UTC 2016 : Starting execution for request: test-invoke-request
      Sun Jul 17 02:55:16 UTC 2016 : HTTP Method: GET, Resource Path: /8ball
      Sun Jul 17 02:55:16 UTC 2016 : Method request path: {}
      Sun Jul 17 02:55:16 UTC 2016 : Method request query string: {From=11111111111, Body=hey}
      Sun Jul 17 02:55:16 UTC 2016 : Method request headers: {}
      Sun Jul 17 02:55:16 UTC 2016 : Method request body before transformations: null
      Sun Jul 17 02:55:16 UTC 2016 : Endpoint request URI: https://lambda.us-west-2.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-west-2:319618807536:function:magic8ball/invocations
      Sun Jul 17 02:55:16 UTC 2016 : Endpoint request headers: {x-amzn-lambda-integration-tag=test-request, Authorization=************************************************************************************************************************************************************************************************************************************************************************************************************************f03697, X-Amz-Date=20160717T025516Z, x-amzn-apigateway-api-id=qehb430znk, X-Amz-Source-Arn=arn:aws:execute-api:us-west-2:319618807536:qehb430znk/null/GET/8ball, Accept=application/json, User-Agent=AmazonAPIGateway_qehb430znk, Host=lambda.us-west-2.amazonaws.com, X-Amz-Content-Sha256=1b77d163dbaae68d26a5d4b5a208299ee80c69e8322bdd908ccd4c3de1cc9fb6, Content-Type=application/json}
      Sun Jul 17 02:55:16 UTC 2016 : Endpoint request body after transformations: {
      “body” : “hey”,
      “fromNumber” : “11111111111”
      }
      Sun Jul 17 02:55:16 UTC 2016 : Endpoint response body before transformations: “Are you making up these questions yourself?”
      Sun Jul 17 02:55:16 UTC 2016 : Endpoint response headers: {x-amzn-Remapped-Content-Length=0, x-amzn-RequestId=e3ca1535-4bc9-11e6-8f7d-03cf265082fd, Connection=keep-alive, Content-Length=140, Date=Sun, 17 Jul 2016 02:55:15 GMT, Content-Type=application/json}
      Sun Jul 17 02:55:16 UTC 2016 : Method response body after transformations: “Are you making up these questions yourself?”
      Sun Jul 17 02:55:16 UTC 2016 : Method response headers: {Content-Type=application/xml}
      Sun Jul 17 02:55:16 UTC 2016 : Successfully completed execution
      Sun Jul 17 02:55:16 UTC 2016 : Method completed with status: 200

  • Nate

    When I try to edit the Mapping Templates for application/json and switch from “Output passthrough” to “Mapping Template” there is no option for Mapping template. If i just enter the above code the Integration response changes from a Lambda status regex to a HTTP status regex

    • Mark

      If you follow the steps in this article you get excited and it fails. I could not get this to work and spent three days on it. One of our dev. team looked at it with me and said he’d rather run this on rails. IMO twillio should delete this article or fix it!

  • KD

    (Not sure if anyone is still monitoring this post since it is 2 years old, but trying…)

    I followed the complete instructions and when I call the Twilio number I am able to hear the IVR prompts defined in the /prompts function. But if I click 1,2 or 3, I get a message “an error has occurred” and the call disconnects.
    Can the author or someone who has this working please help me what could be missing?

    • rickyrobinett

      Hey KD! Wanna drop me an e-mail and I can help debug? ricky@twilio.com

      • KD

        Hey thanks for offering to help. I have emailed you some details.

    • youcantignoremytechno

      care to share here if you got it to work? I’m having the same trouble and and can’t sort it out.

    • Abdulmujeeb Jamiu

      @disqus_AMxbydT8un:disqus @youcantignoremytechno
      I encountered the same issue and I was able to fix it.

      Cause:
      A base url was automatically added to action=”/dev/takeaction” causing it to make request to another API other than your AWS Api-Gateway.

      Solution:
      Remove the based url added e.g: action=”https://www.twilio.com/blog/dev/takeaction” to action=”/dev/takeaction”
      OR
      change the base url to match your Api-Gateway such as action=”https://asdfghjkl.execute-api.us-west-2.amazonaws.com/dev/takeaction”

  • activeliftoff

    Thanks much for the tutorial! I was able to follow it and get up and running without any issues. Keep creating more tutorials!

  • CanuteTheGreat

    After deploying the API Gateway I get the error: {“message”:”Missing Authentication Token”} when trying to test. My URL looks like https://XXXX.execute-api.us-west-2.amazonaws.com/dev/prompt Testing in the API Gateway console does work.

    • Hayden Jacob Baldwin

      Did you figure this out? I’;m getting the same error

      • CanuteTheGreat

        No and I would love to know how to fix it.

        • Hayden Jacob Baldwin

          Twilio now has their own invocation functionality. I just went through that. There’s a simple menu option that’s essentially the same IVR. Also I think it winds up being cheaper.

  • abinash bastola

    @rickyrobinett:disqus tried changing the “Integration Request” to application/x-www-form-urlencoded for “/prompt” API, but when trying to test it, its giving me the status 415 as below:
    {
    “message”: “Unsupported Media Type”
    }

    Can you help me if i am missing anything?

    • Mark Oehler

      I was getting the same thing until I changed the “Request body passthrough” to “When no template matches the request Content-Type header” Then it worked on the tester and from twilio. Not sure if something had changed on AWS between when the article was published and now. Also not sure how that option affects the operation (there is a warning triangle when selecting it).