By
This Sinatra web application handles automated surveys using the Twilio API and TwiML.
For the sake of simplicity, this application is set up to handle questions and answers for a single survey. Pre-defined survey questions are loaded when the application starts for the first time.
Preparing the Survey
In order to perform automated surveys we first need to have some questions we want to ask. If your application is configured properly, then our sample survey (found in the application repository) will automatically load into the database for your convenience.
You can modify the survey questions by cleaning your database, editing the config/questions.yml file, and then re-running the app.
Editor: this is a migrated tutorial. Clone the code from https://github.com/TwilioDevEd/automated-survey-sinatra/
require 'data_mapper'
require_relative '../models/survey'
require_relative '../models/question'
require_relative '../models/answer'
module DataMapperHelper
def self.setup(database_url)
DataMapper.setup(:default, database_url)
DataMapper.finalize
# this section automatically creates the tables
Survey.auto_upgrade!
Question.auto_upgrade!
Answer.auto_upgrade!
end
def self.seed_if_empty
if Survey.all.count == 0
survey = Survey.create(title: 'Default')
questions = YAML.load_file('config/questions.yml')
questions.each do |q|
survey.questions.create(body: q['body'], question_type: q['question_type'])
end
survey.questions.save
end
end
end
Now it's time to get set up with Twilio so that a user can take our survey.
Respond to Twilio's Voice Request
Whenever one of your Twilio phone numbers receives a call, Twilio will make an HTTP request to the voice request URL configured with the HTTP method specified by your webhook (either GET or POST).
If you don't already have a server configured to use as your webhook, ngrok is a great tool for testing webhooks locally.
For this application, Twilio should be configured to make a GET request to the application's /surveys/voice endpoint. The controller then delegates the TwiML generation to an abstraction called TwimlGenerator.
The TwimlGenerator looks for our survey. It will then state its name and welcome the user using TwiML's Say verb. The call is then redirected to the first survey question with the Redirect verb.
module TwimlGenerator
def self.generate_for_incoming_call(survey, base_url)
welcome_message = "Thank you for taking the #{survey.title} survey"
redirect_url = "#{base_url}/questions/1"
response = Twilio::TwiML::VoiceResponse.new
response.say welcome_message
response.redirect redirect_url, method: 'get'
response.to_s
end
def self.generate_for_voice_question(question)
action_url = "/questions/#{question.id}/answers"
response = Twilio::TwiML::VoiceResponse.new
response.say question.body
response.say QuestionMessages.message_for(question.question_type)
if question.voice?
response.record action: action_url, method: 'post'
else
response.gather action: action_url, method: 'post'
end
response.to_s
end
def self.generate_for_exit
response = Twilio::TwiML::VoiceResponse.new
response.say 'Thanks for your time. Good bye'
response.hangup
response.to_s
end
def self.generate_for_sms_question(question, hash = {})
if question.nil?
return respond_sms('Thank you for taking this survey. Goodbye!')
end
msg = ''
msg << 'Thank you for taking our survey!' if hash[:first_time]
msg << question.body
msg << QuestionMessages.message_for(:yesno_sms) if question.yes_no?
respond_sms(msg)
end
def self.respond_sms(message)
response = Twilio::TwiML::MessagingResponse.new
response.message message
response.to_s
end
private_class_method :respond_sms
module QuestionMessages
def self.message_for(type)
{
yesno: 'Please press the 1 for yes and the 0 for no and then hit the pound sign',
voice: 'Please record your answer after the beep and then hit the pound sign',
numeric: 'Please press a number between 0 and 9 and then hit the pound sign',
yesno_sms: " Type '1' or '0'."
}.fetch(type)
end
end
end
Ask the Caller a Question
In the previous step we redirected our client's call to another endpoint that handles the request and returns further instructions for delivering a survey question.
This endpoint checks to see if the incoming request is from an SMS or a voice call and builds our survey question as a TwiML response. Each type of question and interaction (Call or SMS) will produce different instructions for proceeding. For instance, we can record a voice message or gather a key press during a call, but we can't do the same for text messages. In both cases (call or SMS), our code uses the Say verb.
Finally, the app records the caller's answer to the question. If we expect a free-form voice response, we need to use the Record verb. However, if we expect dialpad input, we should use the Gather verb. Both verbs take an action attribute and a method attribute. Twilio will use both attributes to make a request that we can use to store the caller's answer to the question.
module TwimlGenerator
def self.generate_for_incoming_call(survey, base_url)
welcome_message = "Thank you for taking the #{survey.title} survey"
redirect_url = "#{base_url}/questions/1"
response = Twilio::TwiML::VoiceResponse.new
response.say welcome_message
response.redirect redirect_url, method: 'get'
response.to_s
end
def self.generate_for_voice_question(question)
action_url = "/questions/#{question.id}/answers"
response = Twilio::TwiML::VoiceResponse.new
response.say question.body
response.say QuestionMessages.message_for(question.question_type)
if question.voice?
response.record action: action_url, method: 'post'
else
response.gather action: action_url, method: 'post'
end
response.to_s
end
def self.generate_for_exit
response = Twilio::TwiML::VoiceResponse.new
response.say 'Thanks for your time. Good bye'
response.hangup
response.to_s
end
def self.generate_for_sms_question(question, hash = {})
if question.nil?
return respond_sms('Thank you for taking this survey. Goodbye!')
end
msg = ''
msg << 'Thank you for taking our survey!' if hash[:first_time]
msg << question.body
msg << QuestionMessages.message_for(:yesno_sms) if question.yes_no?
respond_sms(msg)
end
def self.respond_sms(message)
response = Twilio::TwiML::MessagingResponse.new
response.message message
response.to_s
end
private_class_method :respond_sms
module QuestionMessages
def self.message_for(type)
{
yesno: 'Please press the 1 for yes and the 0 for no and then hit the pound sign',
voice: 'Please record your answer after the beep and then hit the pound sign',
numeric: 'Please press a number between 0 and 9 and then hit the pound sign',
yesno_sms: " Type '1' or '0'."
}.fetch(type)
end
end
end
Let's see how we can handle and maintain the state of our survey.
Persist a Question's Answer
Twilio has made a POST request to the /questions/:question_id/answers URL. The request includes everything we need to record the caller's answer to our question. Aside from the question_id parameter, Twilio includes a wealth of information with every request. For this sample application we will store the RecordingUrl or Digits parameter for voice and yes-no/numeric answers accordingly. The app also saves the CallSid so we can uniquely identify the origin of the call.
As most surveys include more than a single question, the app should redirect the user to the next question using the Redirect verb again.
If there are no more unanswered questions in the survey, the user will be notified that the survey is complete using the Say verb and dropping the call using the Hangup verb.
require 'sinatra/base'
require 'sinatra/config_file'
require 'sinatra/cookies'
require 'tilt/erb'
require 'digest'
require_relative './helpers/datamapper_helper'
require_relative './helpers/request_helper'
require_relative './lib/twiml_generator'
ENV['RACK_ENV'] ||= 'development'
require 'bundler'
Bundler.require :default, ENV['RACK_ENV'].to_sym
module AutomatedSurvey
class App < Sinatra::Base
set :show_exceptions, false
set :raise_errors, false
set :root, File.dirname(__FILE__)
register Sinatra::ConfigFile
config_file 'config/app.yml'
DataMapperHelper.setup(settings.database_url)
DataMapperHelper.seed_if_empty
# home
get '/' do
erb :index
end
# surveys
get '/surveys/voice' do
survey = Survey.first
twiml = TwimlGenerator
.generate_for_incoming_call(survey, RequestHelper.base_url(request))
content_type 'text/xml'
twiml
end
get '/surveys/sms' do
twiml = ''
origin = request.cookies['origin']
question_id = request.cookies['question_id']
if first_user_sms?
question = Question.get(1)
add_question_id_to_cookie(response, question.id)
add_origin_id_to_cookie(params[:SmsSid])
twiml = TwimlGenerator.generate_for_sms_question(question, first_time: true)
else
Answer.create(
user_input: params[:Body],
origin: origin,
from: params[:From],
question_id: question_id.to_i
)
question = Question.find_next(question_id.to_i)
new_question_id = question.nil? ? nil : question.id
add_question_id_to_cookie(response, new_question_id)
twiml = TwimlGenerator.generate_for_sms_question(question, first_time: false)
end
content_type 'text/xml'
twiml
end
get '/surveys/results' do
survey = Survey.first
calls = Answer
.all(fields: [:id, :origin], unique: true, order: nil)
.map(&:origin)
.uniq
answers_per_call = {}
calls.each do |origin|
answers_per_call[origin] = Answer.all(origin: origin)
end
erb :results, locals: { answers_per_call: answers_per_call, survey: survey }
end
# questions
get '/questions/:question_id' do
question = Question.get(params[:question_id])
twiml = TwimlGenerator.generate_for_voice_question(question)
content_type 'text/xml'
twiml
end
# answers
post '/questions/:question_id/answers' do
Answer.create(
recording_url: params[:RecordingUrl],
user_input: params[:Digits],
origin: params[:CallSid],
from: params[:From],
question_id: params[:question_id].to_i
)
next_question = Question.find_next(params[:question_id].to_i)
if next_question.nil?
twiml = TwimlGenerator.generate_for_exit
else
twiml = TwimlGenerator.generate_for_voice_question(next_question)
end
content_type 'text/xml'
twiml
end
error do
'An application error has ocurred'
end
private
def first_user_sms?
request.cookies['origin'].to_s.empty? ||
request.cookies['question_id'].to_s.empty?
end
def add_question_id_to_cookie(response, question_id)
response.set_cookie 'question_id', value: question_id
end
def add_origin_id_to_cookie(sms_sid)
origin = Digest::SHA1.hexdigest(sms_sid)
response.set_cookie 'origin', value: origin
end
end
end
Now that we're set up for voice call surveys, let's see how to handle surveys taken via SMS.
Identifying Survey SMS Threads
When the user interacts with our survey over SMS we don't have something like an ongoing call session with a well defined state. Since all SMS requests will be sent to the /surveys/sms main endpoint, it becomes harder to know if an SMS is answering question 2 or 20.
In order to keep track of the survey questions and their associated answers, we'll use Twilio cookies. This way, we can add a unique identifier to the conversation sent with every Twilio SMS request with the same to/from phone numbers. This allows us to save each answer's origin properly.
require 'sinatra/base'
require 'sinatra/config_file'
require 'sinatra/cookies'
require 'tilt/erb'
require 'digest'
require_relative './helpers/datamapper_helper'
require_relative './helpers/request_helper'
require_relative './lib/twiml_generator'
ENV['RACK_ENV'] ||= 'development'
require 'bundler'
Bundler.require :default, ENV['RACK_ENV'].to_sym
module AutomatedSurvey
class App < Sinatra::Base
set :show_exceptions, false
set :raise_errors, false
set :root, File.dirname(__FILE__)
register Sinatra::ConfigFile
config_file 'config/app.yml'
DataMapperHelper.setup(settings.database_url)
DataMapperHelper.seed_if_empty
# home
get '/' do
erb :index
end
# surveys
get '/surveys/voice' do
survey = Survey.first
twiml = TwimlGenerator
.generate_for_incoming_call(survey, RequestHelper.base_url(request))
content_type 'text/xml'
twiml
end
get '/surveys/sms' do
twiml = ''
origin = request.cookies['origin']
question_id = request.cookies['question_id']
if first_user_sms?
question = Question.get(1)
add_question_id_to_cookie(response, question.id)
add_origin_id_to_cookie(params[:SmsSid])
twiml = TwimlGenerator.generate_for_sms_question(question, first_time: true)
else
Answer.create(
user_input: params[:Body],
origin: origin,
from: params[:From],
question_id: question_id.to_i
)
question = Question.find_next(question_id.to_i)
new_question_id = question.nil? ? nil : question.id
add_question_id_to_cookie(response, new_question_id)
twiml = TwimlGenerator.generate_for_sms_question(question, first_time: false)
end
content_type 'text/xml'
twiml
end
get '/surveys/results' do
survey = Survey.first
calls = Answer
.all(fields: [:id, :origin], unique: true, order: nil)
.map(&:origin)
.uniq
answers_per_call = {}
calls.each do |origin|
answers_per_call[origin] = Answer.all(origin: origin)
end
erb :results, locals: { answers_per_call: answers_per_call, survey: survey }
end
# questions
get '/questions/:question_id' do
question = Question.get(params[:question_id])
twiml = TwimlGenerator.generate_for_voice_question(question)
content_type 'text/xml'
twiml
end
# answers
post '/questions/:question_id/answers' do
Answer.create(
recording_url: params[:RecordingUrl],
user_input: params[:Digits],
origin: params[:CallSid],
from: params[:From],
question_id: params[:question_id].to_i
)
next_question = Question.find_next(params[:question_id].to_i)
if next_question.nil?
twiml = TwimlGenerator.generate_for_exit
else
twiml = TwimlGenerator.generate_for_voice_question(next_question)
end
content_type 'text/xml'
twiml
end
error do
'An application error has ocurred'
end
private
def first_user_sms?
request.cookies['origin'].to_s.empty? ||
request.cookies['question_id'].to_s.empty?
end
def add_question_id_to_cookie(response, question_id)
response.set_cookie 'question_id', value: question_id
end
def add_origin_id_to_cookie(sms_sid)
origin = Digest::SHA1.hexdigest(sms_sid)
response.set_cookie 'origin', value: origin
end
end
end
This approach can also be used to deliver the next question_id in subsequent requests.
Respond to Twilio's SMS Request
Once we receive an SMS sent to our Twilio phone number, Twilio makes an HTTP GET request to the application's SMS endpoint /surveys/sms.
If there is no record of a previous SMS conversation, we add cookies to identify the SMS conversation (origin) and the next question (question_id). Using the TwimlGenerator generate_for_sms_question method, the application then delivers TwiML instructions to send the first question to the survey taker.
If the SMS arrives at our server with origin and question_id cookies attached, it means that the SMS is part of an existing conversation and we can refer to the question_id. The application then processes the body content, uses it to persist a new answer, and finally delivers TwiML instructions to send the next question to the survey taker.
require 'sinatra/base'
require 'sinatra/config_file'
require 'sinatra/cookies'
require 'tilt/erb'
require 'digest'
require_relative './helpers/datamapper_helper'
require_relative './helpers/request_helper'
require_relative './lib/twiml_generator'
ENV['RACK_ENV'] ||= 'development'
require 'bundler'
Bundler.require :default, ENV['RACK_ENV'].to_sym
module AutomatedSurvey
class App < Sinatra::Base
set :show_exceptions, false
set :raise_errors, false
set :root, File.dirname(__FILE__)
register Sinatra::ConfigFile
config_file 'config/app.yml'
DataMapperHelper.setup(settings.database_url)
DataMapperHelper.seed_if_empty
# home
get '/' do
erb :index
end
# surveys
get '/surveys/voice' do
survey = Survey.first
twiml = TwimlGenerator
.generate_for_incoming_call(survey, RequestHelper.base_url(request))
content_type 'text/xml'
twiml
end
get '/surveys/sms' do
twiml = ''
origin = request.cookies['origin']
question_id = request.cookies['question_id']
if first_user_sms?
question = Question.get(1)
add_question_id_to_cookie(response, question.id)
add_origin_id_to_cookie(params[:SmsSid])
twiml = TwimlGenerator.generate_for_sms_question(question, first_time: true)
else
Answer.create(
user_input: params[:Body],
origin: origin,
from: params[:From],
question_id: question_id.to_i
)
question = Question.find_next(question_id.to_i)
new_question_id = question.nil? ? nil : question.id
add_question_id_to_cookie(response, new_question_id)
twiml = TwimlGenerator.generate_for_sms_question(question, first_time: false)
end
content_type 'text/xml'
twiml
end
get '/surveys/results' do
survey = Survey.first
calls = Answer
.all(fields: [:id, :origin], unique: true, order: nil)
.map(&:origin)
.uniq
answers_per_call = {}
calls.each do |origin|
answers_per_call[origin] = Answer.all(origin: origin)
end
erb :results, locals: { answers_per_call: answers_per_call, survey: survey }
end
# questions
get '/questions/:question_id' do
question = Question.get(params[:question_id])
twiml = TwimlGenerator.generate_for_voice_question(question)
content_type 'text/xml'
twiml
end
# answers
post '/questions/:question_id/answers' do
Answer.create(
recording_url: params[:RecordingUrl],
user_input: params[:Digits],
origin: params[:CallSid],
from: params[:From],
question_id: params[:question_id].to_i
)
next_question = Question.find_next(params[:question_id].to_i)
if next_question.nil?
twiml = TwimlGenerator.generate_for_exit
else
twiml = TwimlGenerator.generate_for_voice_question(next_question)
end
content_type 'text/xml'
twiml
end
error do
'An application error has ocurred'
end
private
def first_user_sms?
request.cookies['origin'].to_s.empty? ||
request.cookies['question_id'].to_s.empty?
end
def add_question_id_to_cookie(response, question_id)
response.set_cookie 'question_id', value: question_id
end
def add_origin_id_to_cookie(sms_sid)
origin = Digest::SHA1.hexdigest(sms_sid)
response.set_cookie 'origin', value: origin
end
end
end
This is how you create SMS responses. Let's dive into sending SMS messages with Twilio.
Delivering Questions using SMS
Here the application creates the TwiML necessary to respond to every incoming SMS from a survey taker. For this purpose we'll use the Message verb every single time.
If the method receives a nil value for the question argument, it means the survey has ended. Our application will then respond with a grateful farewell message.
If it's the first_time the hash argument is true, then the user is just beginning our survey. We will include a welcome message and the text of the first question. For subsequent messages, we will respond with the text of the next question in our survey.
module TwimlGenerator
def self.generate_for_incoming_call(survey, base_url)
welcome_message = "Thank you for taking the #{survey.title} survey"
redirect_url = "#{base_url}/questions/1"
response = Twilio::TwiML::VoiceResponse.new
response.say welcome_message
response.redirect redirect_url, method: 'get'
response.to_s
end
def self.generate_for_voice_question(question)
action_url = "/questions/#{question.id}/answers"
response = Twilio::TwiML::VoiceResponse.new
response.say question.body
response.say QuestionMessages.message_for(question.question_type)
if question.voice?
response.record action: action_url, method: 'post'
else
response.gather action: action_url, method: 'post'
end
response.to_s
end
def self.generate_for_exit
response = Twilio::TwiML::VoiceResponse.new
response.say 'Thanks for your time. Good bye'
response.hangup
response.to_s
end
def self.generate_for_sms_question(question, hash = {})
if question.nil?
return respond_sms('Thank you for taking this survey. Goodbye!')
end
msg = ''
msg << 'Thank you for taking our survey!' if hash[:first_time]
msg << question.body
msg << QuestionMessages.message_for(:yesno_sms) if question.yes_no?
respond_sms(msg)
end
def self.respond_sms(message)
response = Twilio::TwiML::MessagingResponse.new
response.message message
response.to_s
end
private_class_method :respond_sms
module QuestionMessages
def self.message_for(type)
{
yesno: 'Please press the 1 for yes and the 0 for no and then hit the pound sign',
voice: 'Please record your answer after the beep and then hit the pound sign',
numeric: 'Please press a number between 0 and 9 and then hit the pound sign',
yesno_sms: " Type '1' or '0'."
}.fetch(type)
end
end
end
We've seen how to ask questions, receive and store responses. Now lets have a look at the results.
Display the Survey Results
For this endpoint we simply query the database using data_mapper finder methods (note: the data_mapper is now deprecated. Consider using the Ruby Object Mapper for your production applications) and then display the information within an ERB template. We display a panel for every question in the survey, and inside each panel we list the responses from different calls.
You can access this page in the application's root route, /surveys/results.
require 'sinatra/base'
require 'sinatra/config_file'
require 'sinatra/cookies'
require 'tilt/erb'
require 'digest'
require_relative './helpers/datamapper_helper'
require_relative './helpers/request_helper'
require_relative './lib/twiml_generator'
ENV['RACK_ENV'] ||= 'development'
require 'bundler'
Bundler.require :default, ENV['RACK_ENV'].to_sym
module AutomatedSurvey
class App < Sinatra::Base
set :show_exceptions, false
set :raise_errors, false
set :root, File.dirname(__FILE__)
register Sinatra::ConfigFile
config_file 'config/app.yml'
DataMapperHelper.setup(settings.database_url)
DataMapperHelper.seed_if_empty
# home
get '/' do
erb :index
end
# surveys
get '/surveys/voice' do
survey = Survey.first
twiml = TwimlGenerator
.generate_for_incoming_call(survey, RequestHelper.base_url(request))
content_type 'text/xml'
twiml
end
get '/surveys/sms' do
twiml = ''
origin = request.cookies['origin']
question_id = request.cookies['question_id']
if first_user_sms?
question = Question.get(1)
add_question_id_to_cookie(response, question.id)
add_origin_id_to_cookie(params[:SmsSid])
twiml = TwimlGenerator.generate_for_sms_question(question, first_time: true)
else
Answer.create(
user_input: params[:Body],
origin: origin,
from: params[:From],
question_id: question_id.to_i
)
question = Question.find_next(question_id.to_i)
new_question_id = question.nil? ? nil : question.id
add_question_id_to_cookie(response, new_question_id)
twiml = TwimlGenerator.generate_for_sms_question(question, first_time: false)
end
content_type 'text/xml'
twiml
end
get '/surveys/results' do
survey = Survey.first
calls = Answer
.all(fields: [:id, :origin], unique: true, order: nil)
.map(&:origin)
.uniq
answers_per_call = {}
calls.each do |origin|
answers_per_call[origin] = Answer.all(origin: origin)
end
erb :results, locals: { answers_per_call: answers_per_call, survey: survey }
end
# questions
get '/questions/:question_id' do
question = Question.get(params[:question_id])
twiml = TwimlGenerator.generate_for_voice_question(question)
content_type 'text/xml'
twiml
end
# answers
post '/questions/:question_id/answers' do
Answer.create(
recording_url: params[:RecordingUrl],
user_input: params[:Digits],
origin: params[:CallSid],
from: params[:From],
question_id: params[:question_id].to_i
)
next_question = Question.find_next(params[:question_id].to_i)
if next_question.nil?
twiml = TwimlGenerator.generate_for_exit
else
twiml = TwimlGenerator.generate_for_voice_question(next_question)
end
content_type 'text/xml'
twiml
end
error do
'An application error has ocurred'
end
private
def first_user_sms?
request.cookies['origin'].to_s.empty? ||
request.cookies['question_id'].to_s.empty?
end
def add_question_id_to_cookie(response, question_id)
response.set_cookie 'question_id', value: question_id
end
def add_origin_id_to_cookie(sms_sid)
origin = Digest::SHA1.hexdigest(sms_sid)
response.set_cookie 'origin', value: origin
end
end
end
That's it!
If you have configured one of your Twilio numbers to work with the application built in this tutorial, you should be able to take the survey and see the results under the root route of the application. We hope you found this sample application useful.
Where to Next?
If you're a Ruby developer working with Twilio, you might also enjoy these tutorials:
Ruby Quickstart for Programmable Voice
Learn how to use Twilio Client to make browser-to-phone and browser-to-browser calls with ease.
ETA Notifications with Ruby and Rails
Learn how to implement ETA Notifications using Ruby on Rails and Twilio.
Did this help?
Thanks for checking out this tutorial! If you have any feedback to share with us, we'd love to hear it. Connect with us on Twitter and let us know what you build!
