Receive and Reply to SMS and MMS Messages on the ESP8266 with C++ and ngrok
Today we'll use Twilio's Programmable SMS to respond to incoming messages directly from a C++ application running on a ESP8266. When a SMS or MMS is sent to your Twilio number, we'll use ngrok to route a secure connection from Twilio to a route on your ESP8266. We'll then apply some logic on Twilio's request, possibly blink some LEDs, and send a reply using Twilio's markup language TwiML.
Sound difficult? Not a bit, byte, or even a nibble! We'll have you hooked on our webhooks in no time.
Twilio can send your web application an HTTP request when certain events happen, such as an incoming text message to one of your Twilio phone numbers. These requests are called webhooks, or status callbacks. For more, check out our guide to Getting Started with Twilio Webhooks. Find other webhook pages, such as a security guide and an FAQ in the Webhooks section of the docs.
What is a Webhook?
Webhooks are user-defined HTTP callbacks. They are usually triggered by some event, such as receiving an SMS message or an incoming phone call. When that event occurs, Twilio makes an HTTP request (usually a POST or a GET) to the URL configured for the webhook.
To handle a webhook, you only need to build a small web application that can accept the HTTP requests. Almost all server-side programming languages offer some framework for you to do this. Examples across languages include ASP.NET MVC for C#, Servlets and Spark for Java, Express for Node.js, Django and Flask for Python, and Rails and Sinatra for Ruby. PHP has its own web app framework built in, although frameworks like Laravel, Symfony and Yii are also popular.
Whichever framework and language you choose, webhooks function the same for every Twilio application. They will make an HTTP request to a URI that you provide to Twilio. Your application performs whatever logic you feel necessary - read/write from a database, integrate with another API or perform some computation - then replies to Twilio with a TwiML response with the instructions you want Twilio to perform.
What is TwiML?
TwiML is the Twilio Markup Language, which is just to say that it's an XML document with special tags defined by Twilio to help you build your SMS and voice applications. TwiML is easier shown than explained. Here's some TwiML you might use to respond to an incoming phone call:
<?xml version="1.0" encoding="UTF-8"?> <Response> <Say>Thanks for calling!</Say> </Response>
And here's some TwiML you might use to respond to an incoming SMS message:
<?xml version="1.0" encoding="UTF-8"?> <Response> <Message>We got your message, thank you!</Message> </Response>
Every TwiML document will have the root <Response> element and within that can contain one or more verbs. Verbs are actions you'd like Twilio to take, such as <Say> a greeting to a caller, or send an SMS <Message> in reply to an incoming message. For a full reference on everything you can do with TwiML, refer to our TwiML API Reference.
Prerequisites for Receiving and Responding to SMS Messages on the ESP8266
There are a number of prerequisites you'll need to keep in mind in order to properly follow this guide. At the heart (or maybe the brain) is the Espressif ESP8266, found on modules as inexpensive as $2. Boasting TLS 1.2 support, it's one of the most budget-friendly parts that can exercise Twilio's APIs securely.
At a minimum for integrating Twilio webhooks, you'll need an ESP8266, a method of programming it, and a way to securely route to the ESP. To follow this particular article, you'll additionally need to ensure you can program your ESP8266 from the Arduino IDE.
ESP8266 Board Selection
If you haven't yet selected a development board, we'd suggest checking the tested board list from the Arduino on ESP8266 Github repository. They don't cover every ESP8266 variation or roll-your-own PCBs, but picking a tested variant is the quickest way to get started. The ESP-8266 12-E variants such as this one work well.
When I first built this, I used a Sparkfun Thing and Sparkfun's Basic FTDI breakout for programming. The Thing overloads the DTR pin and won't work when programming and connected to the serial monitor. For most boards you don't need to touch the serial
#define s in the code.
But if you do: you can disable or enable software serial by changing the
USE_SOFTWARE_SERIAL in Twilio_ESP8266_Example.ino.
ESP8266 core for Arduino conveniently provides the Xtensa gcc toolchain, the very useful Arduino libraries, and an easy way to get started programming the ESP8266.
Although it is beyond the scope of the guide, please let us know any success you have receiving and replying to SMS and MMS messages on the ESP8266 using other toolchains. We'd love to hear your methodology!
Generating TwiML Manually on the ESP8266
In our Twilio helper libraries we've included code to assist in formulating correct TwiML. Alas, with a part supporting 16 megabytes maximum flash memory with RAM measured in kilobytes, we have to generate our own TwiML from scratch.
Don't worry, though - Twilio's markup language is very simple and powerful.
For this example application, one path will respond to incoming SMS and MMSes with a canned, generic message. In another, we demonstrate very simple (insecure) authentication against a master number, allowing that number to turn on and off the onboard LED on many ESP8266 development boards.
In this code:
- We receive an incoming text message and immediately prep a response starting with
<?xml version=\"1.0\" encoding=\"UTF-8\"?>.
- We then decide if a message is from an 'authorized' number, which we defined as matching our master_number variable.
- If we are authorized and receive a body only containing '0' or '1', we turn off or on the onboard LED and use the
<Message>tags. (Note that some boards have an active low LED and will have inverted logic - we'll leave it to you to switch the messages!)
An authorized incoming SMS with '?' as the body will respond with instructions and the current status of the LED, again in the
<Message> tags. Unauthorized requests with any of those three commands will be warned off.
Finally, we respond with
application/xml as the MIME type and a status code of 200.
Setting Up a Route on the ESP8266
Actually setting up a route on the ESP8266 web server is very simple with the
ESP8266WebServer library. At a minimum, you will need to instantiate a ESP8266WebServer object, set up a single route and function to handle requests, and start the server listening. Twilio expects TwiML in response - so once you get the route working properly, you'll want to return TwiML to direct Twilio on how to respond to the incoming message.
In our code, this line:
sets up a route at /message, and
handle_message is a function pointer to a void function (our implementation is in the Github Repo). When a request is made to /message, that function is called and the ESP8266 goes through the previously dicussed logic to pick a response (and possibly blink the LED!).
All of the plumbing to make this work is highlighted in this code sample.
Configuring Your Webhook URL
If all went well, you now have a webserver running on the ESP8266 on a friendly network! If you have made it this far, the remainder of your plumbing will be spent outside of the ESP8266.
First, however, you're going to need to find the webserver. Use whatever means is easiest to find the IP address of the ESP8266 on your local network. If you are hooked up to serial on the board, this might be as easy as looking for the assigned IP address in a message similar to this successful run:
..... Connected to WiFi, IP address: 192.168.1.155 Connecting to host api.twilio.com Certificate fingerprints match.
While your ESP8266 almost certainly won't be found at 192.168.1.155, when you do find the IP you can test by visiting it (in a browser) at:
<YOUR ESP8266'S IP ADDRESS>:8000/message
and verifying you get an xml response.
If it works, you're ready to share it with the world and let Twilio know where to find your ESP. But first, we need to find a way to expose it.
Using ngrok to Expose the ESP8266
A simple way to test our new ESP8266 server is with ngrok. ngrok makes it very easy to tunnel secure requests to your local network, ensuring that encryption isn't broken until inside your (hopefully) more controllable internal network.
On a machine which can see the ESP8266 and has ngrok installed, simply run the following command in your choice of terminal (replacing 192.168.1.155 with the IP address of your ESP8266):
ngrok http 192.168.1.155:8000
... and that's really all you need. You should see ngrok take over your terminal with a nice summary:
Configuring a Messaging Webhook with Twilio
Now that our ESP8266 is exposed to the world with ngrok, there is only one more step before we can text our $2 piece of silicon for fun and profit.
Login to the Twilio Console, and navigate to a number (preferably, the number set in the from_number variable) you'd like to hook to your board.
Scroll down, and you will see a section to configure webhook URLs for messaging. Simply paste in the secure ngrok URL followed by
/message and you'll be good to go:
After that, you're cooking with Twilio. Try texting '0', '1', '?' and whatever else you desire from the
master_number and other numbers and see what you get. That's really all it takes to hook up your ESP8266 device with Twilio!
Backup Webhook URL
You'll note there is an optional 'Primary Handler Fails' field. In your final application, you will likely want to add a secondary endpoint for Twilio to fall back on if the ESP8266 is unreachable, so you can handle the logic for when it comes back online.
Simply add a URL to that field and Twilio will failover automatically on an error code or if the primary webhook doesn't respond within 15 seconds. Refer to our Availability and Reliability guide for more details on the fallback URL.
Protect your webhooks
Twilio supports HTTP Basic and Digest Authentication. Authentication allows you to password protect your TwiML URLs on your web server so that only you and Twilio can access them.
Learn more about HTTP authentication here.
Respond with Media (MMS Message)
To send an MMS, it's as simple as adding a new tag to the TwiML with a
<Media> tag. You can include up to 10 individual media tags in a response.
Although it's beyond the scope of this piece, that means you could send back images hosted (or generated dynamically) on the ESP8266 in response to incoming messages - nifty!
Enhance Messages with Add-ons
Need more information about the phone number that sent the message? Need to analyze the message itself for sentiment or other data? Add-ons are available in the Add-ons Marketplace to accomplish these tasks and more.
To learn how to enable Add-ons for your incoming SMS messages, refer to our Add-ons quickstart.
Who Needs More than 64 kB of RAM, Let Alone 640 kB?
Between this article and our article on how to send SMS and MMS messages from an ESP8266, we've covered how to handle incoming and outgoing messages with the Espressif ESP8266 and the Twilio API. Now you can deploy your web apps to environments servers couldn't previously go.
The next step is up to you. Use Twilio and your imagination to set up a security, home automation, or remote monitoring app - we'd love to hear what you build! Contact us on Twitter with a link to your next big Thing.
Need some help?
We all do sometimes; code is hard. Get help now from our support team, or lean on the wisdom of the crowd by visiting Twilio's Stack Overflow Collective or browsing the Twilio tag on Stack Overflow.