Receive Twitter Notifications by SMS with EventMachine and the Twitter Streaming API

I’ve been having fun with the Twitter and Twilio APIs recently. First was the release of a tool to send and receive Tweets using SMS and Twilio. In part 2 I started to dig into how the app was built, starting with how to send a Tweet with SMS. Now it’s time to look into how the other side works: how to send SMS notifications when particular events happen on Twitter. This is going to get a bit more complicated than part 2 so strap in, grab the code so far from GitHub and let’s get started.

Real Time Events from Twitter

For this feature, we want to be able to send an SMS to our phone when events we want to know about happen on Twitter. For the purposes of this application those events are direct messages and mentions.

To achieve this we could repeatedly poll the Twitter API but we might find the time difference between notifications and when they actually happened frustrating. This may be ok when you’re interested in beer updates from your local shop but Twitter is a bit more real time than that. Thankfully we have the Twitter streaming API which gives us a stream of events as they happen on the platform. Check out an example of the streaming API below:

An example of viewing the twitter streaming sample in curl.

That’s a lot of data and it’s not even the firehose. The streaming API’s user streams feature gives us all the tweets and activities relevant to a user account. It’s what your Twitter client uses to show you your stream and this is what we will use to listen for the notifications for this app.

So, the plan is to listen to the user stream for our account and send an SMS when a direct message or mention happens. You can use the Twitter gem for the streaming API, but like most Ruby code the gem blocks the process until the stream is stopped. Were we to run this within the Sinatra app we built in the last post we would not be able to simultaneously wait for webhooks at our /messages endpoint and collect and respond to events from the streaming API. Instead, let’s use the TweetStream gem which uses the magic of EventMachine to avoid blocking.

Exploiting the EventMachine

EventMachine is evented I/O for Ruby based on the reactor pattern. You may have heard of other event based systems, like Python’s Twisted or young upstart JavaScript platform Node.js. EventMachine uses an event loop to process incoming events and dispatch them to event handlers.

TweetStream uses EventMachine to process events from the Twitter streaming API and dispatch Tweets to handlers that we define. This is why we are using Thin as the web server for the application. Thin is also based on EventMachine so when you start the Thin server TweetStream is able to use the same EventMachine reactor to process events. Non EventMachine based Ruby web servers, like the default WEBrick or Unicorn, don’t provide a reactor for TweetStream, so the gem becomes unable to listen for incoming Tweets. With Thin we can listen to the Twitter streaming API and to incoming web requests within the same process.

Finishing the App

Let’s start with the app we had at the end of part 2. If you didn’t write the code from part 2, you can clone the repo from that point with this command:

$ git clone -b how-to-send-a-tweet-with-sms  https://github.com/philnash/twitter-sms.git

1	$ git clone -b how-to-send-a-tweet-with-sms https://github.com/philnash/twitter-sms.git

We need to add TweetStream to our Gemfile:

source "https://rubygems.org"

ruby "2.2.2"

gem "sinatra"
gem "rack"
gem "twitter"
gem "thin"
gem "tweetstream"
gem "twilio-ruby"

group :development do
  gem "envyable"
end

source "https://rubygems.org"

ruby "2.2.2"

gem "sinatra"

gem "rack"

gem "twitter"

gem "thin"

gem "tweetstream"

gem "twilio-ruby"

group :development do

gem "envyable"

end

Then run:

$ bundle install

1	$ bundle install

Once that is complete, we can get into app.rb and implement TweetStream.

When we start the application, Thin will start the EventMachine reactor. For TweetStream to hook into the same reactor we call it within a block that we pass to EM.schedule:

# app.rb
EM.schedule do
 # Do TweetStream processing here
end

# app.rb

EM.schedule do

# Do TweetStream processing here

end

That is all we need to do to run our streaming API listener and the web application at the same time. Let’s take a look at what to do when we receive a tweet.

TweetStream to SMS

First we need to start with some config. Check out part 1 for how to find or create all the correct credentials. Make sure your permissions for your Twitter application allow read, write and direct messages, otherwise only parts of the following will work.

# app.rb
EM.schedule do
  TweetStream.configure do |config|
    config.consumer_key        = ENV["TWITTER_CONSUMER_KEY"]
    config.consumer_secret     = ENV["TWITTER_CONSUMER_SECRET"]
    config.oauth_token         = ENV["TWITTER_ACCESS_TOKEN"]
    config.oauth_token_secret  = ENV["TWITTER_ACCESS_SECRET"]
  end

client = TweetStream::Client.new

 # Handle the tweets
end

# app.rb

EM.schedule do

TweetStream.configure do |config|

config.consumer_key = ENV["TWITTER_CONSUMER_KEY"]

config.consumer_secret = ENV["TWITTER_CONSUMER_SECRET"]

config.oauth_token = ENV["TWITTER_ACCESS_TOKEN"]

config.oauth_token_secret = ENV["TWITTER_ACCESS_SECRET"]

end

client = TweetStream::Client.new

# Handle the tweets

end

We also need to add one more bit of config to our config/env.yml file. We didn’t need it before but since we’re now sending SMS messages we need to know the Twilio number we are sending from. Grab the number from your Twilio dashboard and add it like:

# config/env.yml
development:
  TWILIO_ACCOUNT_SID: YOUR_ACCOUNT_SID
  TWILIO_AUTH_TOKEN: YOUR_AUTH_TOKEN
  TWITTER_CONSUMER_KEY: YOUR_CONSUMER_KEY
  TWITTER_CONSUMER_SECRET: YOUR_CONSUMER_SECRET
  TWITTER_ACCESS_TOKEN: YOUR_ACCESS_TOKEN
  TWITTER_ACCESS_SECRET: YOUR_ACCESS_SECRET
  TWITTER_USERNAME: YOUR_USERNAME
  MY_PHONE_NUMBER: YOUR_PHONE_NUMBER
  MY_TWILIO_NUMBER: "+442033898457"

# config/env.yml

development:

TWILIO_ACCOUNT_SID: YOUR_ACCOUNT_SID

TWILIO_AUTH_TOKEN: YOUR_AUTH_TOKEN

TWITTER_CONSUMER_KEY: YOUR_CONSUMER_KEY

TWITTER_CONSUMER_SECRET: YOUR_CONSUMER_SECRET

TWITTER_ACCESS_TOKEN: YOUR_ACCESS_TOKEN

TWITTER_ACCESS_SECRET: YOUR_ACCESS_SECRET

TWITTER_USERNAME: YOUR_USERNAME

MY_PHONE_NUMBER: YOUR_PHONE_NUMBER

MY_TWILIO_NUMBER: "+442033898457"

Once that is sorted we set up a block to receive a direct message and send us an SMS. We need to instantiate a new Twilio client and use it to send the message.

# app.rb
EM.schedule do
  # Config (removed for clarity)

  client = TweetStream::Client.new

  client.on_direct_message do |direct_message|
    twilio = Twilio::REST::Client.new(ENV["TWILIO_ACCOUNT_SID"], ENV["TWILIO_AUTH_TOKEN"])
    twilio.messages.create(
      from: ENV["MY_TWILIO_NUMBER"],
      to:   ENV["MY_PHONE_NUMBER"],
      body: "DM from #{direct_message.sender.screen_name}: #{direct_message.text}"
    )
  end
end

# app.rb

EM.schedule do

# Config (removed for clarity)

client = TweetStream::Client.new

client.on_direct_message do |direct_message|

twilio = Twilio::REST::Client.new(ENV["TWILIO_ACCOUNT_SID"], ENV["TWILIO_AUTH_TOKEN"])

twilio.messages.create(

from: ENV["MY_TWILIO_NUMBER"],

to: ENV["MY_PHONE_NUMBER"],

body: "DM from #{direct_message.sender.screen_name}: #{direct_message.text}"

)

end

The streaming API sends us all events, even ones we initiated, so we have to check that the direct message wasn’t sent by our account.

# app.rb
EM.schedule do
# Config

  client.on_direct_message do |direct_message|
    if direct_message.sender.screen_name != ENV["TWITTER_USERNAME"]
      twilio = Twilio::REST::Client.new(ENV["TWILIO_ACCOUNT_SID"], ENV["TWILIO_AUTH_TOKEN"])
      twilio.messages.create(
        from: ENV["MY_TWILIO_NUMBER"],
        to:   ENV["MY_PHONE_NUMBER"],
        body: "DM from #{direct_message.sender.screen_name}: #{direct_message.text}"
      )
    end
  end
end

# app.rb

EM.schedule do

# Config

client.on_direct_message do |direct_message|

if direct_message.sender.screen_name != ENV["TWITTER_USERNAME"]

twilio = Twilio::REST::Client.new(ENV["TWILIO_ACCOUNT_SID"], ENV["TWILIO_AUTH_TOKEN"])

twilio.messages.create(

from: ENV["MY_TWILIO_NUMBER"],

to: ENV["MY_PHONE_NUMBER"],

body: "DM from #{direct_message.sender.screen_name}: #{direct_message.text}"

)

end

We can also set up a block for handling mentions:

# app.rb
EM.schedule do
 # Config and direct messages

 client.on_timeline_status do |status|
    if status.user_mentions.any? { |mention| mention.screen_name == ENV["TWITTER_USERNAME"] }
      twilio = Twilio::REST::Client.new(ENV["TWILIO_ACCOUNT_SID"], ENV["TWILIO_AUTH_TOKEN"])
      twilio.messages.create(
        from: ENV["MY_TWILIO_NUMBER"],
        to:   ENV["MY_PHONE_NUMBER"],
        body: "@mention from #{status.user.screen_name}: #{status.text}"
      )
    end
  end
end

# app.rb

EM.schedule do

# Config and direct messages

client.on_timeline_status do |status|

if status.user_mentions.any? { |mention| mention.screen_name == ENV["TWITTER_USERNAME"] }

twilio = Twilio::REST::Client.new(ENV["TWILIO_ACCOUNT_SID"], ENV["TWILIO_AUTH_TOKEN"])

twilio.messages.create(

from: ENV["MY_TWILIO_NUMBER"],

to: ENV["MY_PHONE_NUMBER"],

body: "@mention from #{status.user.screen_name}: #{status.text}"

)

end

In this case, we have to listen to all timeline status messages and pick out the ones that include a mention of our account. You’ve probably noticed we have a bit of repetition here, so let’s tidy that up. Firstly, a method to return a Twilio client will be useful:

# app.rb
def twilio
  @twilio ||= Twilio::REST::Client.new(ENV["TWILIO_ACCOUNT_SID"], ENV["TWILIO_AUTH_TOKEN"])
end

# app.rb

def twilio

@twilio ||= Twilio::REST::Client.new(ENV["TWILIO_ACCOUNT_SID"], ENV["TWILIO_AUTH_TOKEN"])

end

Then something to send an SMS message, only the body of the message needs to change each time so we can write:

# app.rb
def send_sms(message)
  twilio.messages.create(
    to: ENV["MY_PHONE_NUMBER"],
    from: ENV["MY_TWILIO_NUMBER"],
    body: message
  )
end

# app.rb

def send_sms(message)

twilio.messages.create(

to: ENV["MY_PHONE_NUMBER"],

from: ENV["MY_TWILIO_NUMBER"],

body: message

)

end

Now we can use this method to clean up the notification events:

# app.rb
EM.schedule do
  # Config

  client.on_direct_message do |direct_message|
    if direct_message.sender.screen_name != ENV["TWITTER_USERNAME"]
      send_sms("DM from #{direct_message.sender.screen_name}: #{direct_message.text}")
    end
  end

  client.on_timeline_status do |status|
    if status.user_mentions.any? { |mention| mention.screen_name == ENV["TWITTER_USERNAME"] }
      send_sms("@mention from #{status.user.screen_name}: #{status.text}")
    end
  end
end

# app.rb

EM.schedule do

# Config

client.on_direct_message do |direct_message|

if direct_message.sender.screen_name != ENV["TWITTER_USERNAME"]

send_sms("DM from #{direct_message.sender.screen_name}: #{direct_message.text}")

end

client.on_timeline_status do |status|

if status.user_mentions.any? { |mention| mention.screen_name == ENV["TWITTER_USERNAME"] }

send_sms("@mention from #{status.user.screen_name}: #{status.text}")

end

Finally in this block we start the stream with one line:

# app.rb
EM.schedule do
 # All the above setup

 client.userstream
end

# app.rb

EM.schedule do

# All the above setup

client.userstream

end

We can run the application, using Thin, with the following command:

$ bundle exec thin start -R config.ru

1	$ bundle exec thin start -R config.ru

Now, get your friends to send you a DM or @mention on Twitter and watch the SMS messages roll in!

Once the app is running, you should receive @mentions by SMS

Streaming and web serving in one

There we go: in two blog posts we’ve gone from SMS to Twitter and back again in 62 lines of Ruby. This shows the huge power available to us Rubyists. With a few gems we can run an event based server that listens to a streaming API and responds to webhooks that act as a bridge between our Twitter account and SMS.

If you have any feedback on how I built this or there’s a feature you wish this app could perform, it’s all up on GitHub, so raise an issue or, even better, a pull request.

Any other questions, feel free to drop me a comment on this post or email me at philnash@twilio.com. Or shoot me a Tweet at @philnash, I’m waiting for your SMS!