Automated Survey with Ruby and Sinatra

This Sinatra web application handles automated surveys using the Twilio API and TwiML.

For the sake of simplicity, this application is set up to handle questions and answers for a single survey. Pre-defined survey questions are loaded when the application starts for the first time.

Instacart uses Twilio to power their customer service surveys and integrate that feedback into their customer database. Read more here.

Preparing the Survey

In order to perform automated surveys we first need to have some questions we want to ask. If your application is configured properly, then our sample survey (found in the application repository) will automatically load into the database for your convenience.

You can modify the survey questions by cleaning your database, editing the config/questions.yml file, and then re-running the app.

Now it's time to get set up with Twilio so that a user can take our survey.

Respond to Twilio's Voice Request

Whenever one of your Twilio phone numbers receives a call, Twilio will make an HTTP request to the voice request URL configured with the HTTP method specified by your webhook (either GET or POST).

If you don't already have a server configured to use as your webhook, ngrok is a great tool for testing webhooks locally.

For this application, Twilio should be configured to make a GET request to the application's /surveys/voice endpoint. The controller then delegates the TwiML generation to an abstraction called TwimlGenerator.

The TwimlGenerator looks for our survey. It will then state its name and welcome the user using TwiML's Say verb. The call is then redirected to the first survey question with the Redirect verb.

Ask the Caller a Question

In the previous step we redirected our client's call to another endpoint that handles the request and returns further instructions for delivering a survey question.

This endpoint checks to see if the incoming request is from an SMS or a voice call and builds our survey question as a TwiML response. Each type of question and interaction (Call or SMS) will produce different instructions for proceeding. For instance, we can record a voice message or gather a key press during a call, but we can't do the same for text messages. In both cases (call or SMS), our code uses the Say verb.

Finally, the app records the caller's answer to the question. If we expect a free-form voice response, we need to use the Record verb. However, if we expect dialpad input, we should use the Gather verb. Both verbs take an action attribute and a method attribute. Twilio will use both attributes to make a request that we can use to store the caller's answer to the question.

Let's see how we can handle and maintain the state of our survey.

Persist a Question's Answer

Twilio has made a POST request to the /questions/:question_id/answers URL. The request includes everything we need to record the caller's answer to our question. Aside from the question_id parameter, Twilio includes a wealth of information with every request. For this sample application we will store the RecordingUrl or Digits parameter for voice and yes-no/numeric answers accordingly. The app also saves the CallSid so we can uniquely identify the origin of the call.

As most surveys include more than a single question, the app should redirect the user to the next question using the Redirect verb again.

If there are no more unanswered questions in the survey, the user will be notified that the survey is complete using the Say verb and dropping the call using the Hangup verb.

Now that we're set up for voice call surveys, let's see how to handle surveys taken via SMS.

Identifying Survey SMS Threads

When the user interacts with our survey over SMS we don't have something like an ongoing call session with a well defined state. Since all SMS requests will be sent to the /surveys/sms main endpoint, it becomes harder to know if an SMS is answering question 2 or 20.

In order to keep track of the survey questions and their associated answers, we'll use Twilio cookies. This way, we can add a unique identifier to the conversation sent with every Twilio SMS request with the same to/from phone numbers. This allows us to save each answer's origin properly.

This approach can also be used to deliver the next question_id in subsequent requests.

Respond to Twilio's SMS Request

Once we receive an SMS sent to our Twilio phone number, Twilio makes an HTTP GET request to the application's SMS endpoint /surveys/sms.

If there is no record of a previous SMS conversation, we add cookies to identify the SMS conversation (origin) and the next question (question_id). Using the TwimlGenerator generate_for_sms_question method, the application then delivers TwiML instructions to send the first question to the survey taker.

If the SMS arrives at our server with origin and question_id cookies attached, it means that the SMS is part of an existing conversation and we can refer to the question_id. The application then processes the body content, uses it to persist a new answer, and finally delivers TwiML instructions to send the next question to the survey taker.

This is how you create SMS responses. Let's dive into sending SMS messages with Twilio.

Delivering Questions using SMS

Here the application creates the TwiML necessary to respond to every incoming SMS from a survey taker. For this purpose we'll use the Message verb every single time.

If the method receives a nil value for the question argument, it means the survey has ended. Our application will then respond with a grateful farewell message.

If it's the first_time the hash argument is true, then the user is just beginning our survey. We will include a welcome message and the text of the first question. For subsequent messages, we will respond with the text of the next question in our survey.

We've seen how to ask questions, receive and store responses. Now lets have a look at the results.

Display the Survey Results

For this endpoint we simply query the database using data_mapper finder methods (note: the data_mapper is now deprecated. Consider using the Ruby Object Mapper for your production applications) and then display the information within an ERB template. We display a panel for every question in the survey, and inside each panel we list the responses from different calls.

You can access this page in the application's root route, /surveys/results.

That's it!

If you have configured one of your Twilio numbers to work with the application built in this tutorial, you should be able to take the survey and see the results under the root route of the application. We hope you found this sample application useful.

Where to Next?

If you're a Ruby developer working with Twilio, you might also enjoy these tutorials:

Ruby Quickstart for Programmable Voice

Learn how to use Twilio Client to make browser-to-phone and browser-to-browser calls with ease.

ETA Notifications with Ruby and Rails

Learn how to implement ETA Notifications using Ruby on Rails and Twilio.

Did this help?

Thanks for checking out this tutorial! If you have any feedback to share with us, we'd love to hear it. Connect with us on Twitter and let us know what you build!