How to send SMS at scale with Twilio

Tags

Java

Products

Messaging

Time to read: 8 minutes

April 16, 2021

Written by

Matthew Gilliard

Twilion

Reviewed by

Adam King

Twilion

Liz Moy

Twilion

August Allard

Twilion

How to send SMS at scale with Twilio

If you want to send a single SMS using Twilio you'll need a Twilio account, a phone number to send them from, and of course you'll need to know the destination phone number and the content of the message. Once you have these you can send your message with a single API call either with a direct HTTP request, or using any of our helper libraries. Check out our Programmable SMS Quickstarts for more details.

What if you need to send a hundred messages? Or a thousand? Or even more? In this post I'll talk through the challenges you may face as you scale up and how to get past them. Code samples will be in Java but the same principles apply with any programming language.

Sending a single SMS

Sending one SMS with Java is covered in this blog post. Feel free to go and read it, but to summarize: set up a Twilio account (if you don't have one yet, sign up for a free account here and receive $10 credit when you upgrade), buy a phone number then initialize the Twilio client and send the message with code like this:

Twilio.init(
    System.getenv("TWILIO_ACCOUNT_SID"),
    System.getenv("TWILIO_AUTH_TOKEN"));

Message message = Message.creator(
    new PhoneNumber("<TO number - ie your cellphone>"),
    new PhoneNumber("<FROM number - ie your Twilio number"),
    "Hello from Twilio 📞")
  .create();

System.out.println(message.getSid());

[this code on GitHub]

What happened to my message?

When you make an API request to send a message, a successful API response means that Twilio has accepted and enqueued the message. There are a few states for a message after that - typically "sending" and then "sent", but there are other possible states, detailed on this page.

The last line of code above prints out the message's SID (String Identifier), which starts SM. You can use this SID to uniquely identify the message and check its status in the Twilio console Programmable Message Logs page. Message log retention is configurable in a number of ways and message logs can be downloaded in bulk. See this post for more information.

Screenshot: the console page showing the states that a message goes through, from created to delivered.

For a single message it's probably OK for you to check that page manually, but this will get to be tedious once you're sending a lot of messages.

Another option is to set a Status Callback URL when you create the message. This will instruct Twilio to make an HTTP request to the URL you provide on any status change. The payload for that request will include the new status and the message SID so you can correlate and keep track of message statuses programmatically.


Message message = Message.creator(
        new PhoneNumber("<TO number - ie your cellphone>"),
        new PhoneNumber("<FROM number - ie your Twilio number"),
        "Hello from Twilio 📞")
    .setStatusCallback("https://<YOUR STATUS CALLBACK URL>")
    .create();

[link to this code on GitHub]

Notice the call to .setStatusCallback(url) on line 5.

This approach of using a single "from" number, and making one API call per SMS is fine for testing and small volumes of messages.

Challenges of scaling up

As you scale up from sending one message to hundreds or thousands of messages, you will find that the approach outlined above doesn't scale. You will need to pay attention to at least some of the following:

Carrier rate-limits. Carriers limit SMS to 1 Message Segment per "from" phone number per second in the USA, or 10/sec in most other countries. An SMS will be at least one segment, longer messages might be split into several. Still confused? We have you covered:
- What the heck is a Segment?
- Twilio Messaging Segment Calculator.
Customer preferences. When you are sending lots of messages, it's likely that the recipients aren't all in the same place. People prefer to receive messages from numbers that are local to them. This is especially true if your recipient list is international.
Anti-spam. You will want a way to handle "STOP" or "UNSUBSCRIBE" responses automatically. You also may want a mechanism that captures data about recipients who unsubscribe.
Twilio API rate limits. The limits are high, but if you make requests too quickly, or have too many concurrent requests you can get HTTP/429 responses from Twilio's API. To ensure delivery you will need to catch that error, slow down and retry.
Monitoring and re-delivering. Messages that are successfully enqueued may later fail at the delivery stage. There are various reasons why this can happen. Monitoring and reacting to this becomes harder as you scale up.
Carrier filtering. Carriers can filter messages to prevent spam, fraud or abuse, and there may be regional legal restrictions on what kinds of messages are allowed at certain times of day

Lets go over the tools you can use to address these challenges before they become problems.

Messaging Services

Your first friend on this journey will be the Messaging Service. These are pools of one or more phone numbers that can be used in place of a "from" number when you send your SMS, and they have some clever behaviour which kicks in when you add more than one number to them.

If you are building a proof-of-concept for a production service, have any ambition to scale up or there's even a hint of a chance to have your SMS service go viral, we absolutely recommend using a Messaging Service from the start. This will become especially important if you are sending messages in the USA as A2P 10DLC changes come into effect.

Once the Messaging Service is set up, the code to send SMS is almost exactly the same as before, and you can add or remove numbers from your sender pool at any time. You will have solved a lot of the problems that future-you might face and unlocked a raft of cool new options too.

When you send an SMS from a Message Service, the "from" number for a particular recipient's message can be configured in a number of ways, collectively known as Copilot. This includes:

GeoMatching: using the number closest (measured by area or national code) to the recipient
Scaler: distributing outbound SMS evenly over multiple "from" numbers. If you send several messages to the same recipient over time, then Sticky Sender can make sure that they see the same "from" number each time.

Another couple of cool features that Messaging Services give you are:

Advanced Opt-Out: Customize the messages you use for compliance with "STOP" and "UNSUBSCRIBE" responses
Smart Encoding: cutting the size of your SMS by replacing smart quotes and other tricky characters with their GSM-7 equivalents
Alpha Sender: show an alphanumeric sender ID as well as the "from" number in most countries

That's not all: Check our documentation on Messaging Services for a full list of what is possible.

Creating and using your Messaging Service

You can create a Messaging Service using the API, the Twilio CLI, or in your Twilio console:

Screenshot: create message service dialog. I've called it "Owl Airlines Messaging Service"

The use case question sets some defaults for the Messaging Service, such as the ability to process responses, but you can change any of the settings later so don't spend too long thinking about this for now.

After creating the Messaging Service, add one or more numbers to the sender pool. If you aren't sure how to work out how many numbers you will need, check this post on Messaging Queue Insights. You can also add an Alphanumeric Sender ID:

Screenshot: configuring the sender pool for a messaging service. Added 2 phone numbers and an Alpha Sender

Last, to refer to this Messaging Service from code you will need its SID which starts with MG:

Screenshot: configuring the name of the Messaging Service and seeing the SID on the "properties" section of the console

You can see the sections on the left where you can configure other things like smart encoding and customised opt-out. For now, head back to the code and use the Messaging Service in place of a hard-coded "from" number:


Message message = Message.creator(
        new PhoneNumber("<TO number - ie your cellphone>"),
        System.getenv("MESSAGING_SERVICE_SID"),
        "Hello from Twilio 📞")
    .setStatusCallback("https://<YOUR-STATUS-CALLBACK-URL>")
    .create();

[This code on GitHub]

The only difference between this and the previous code sample is that you pass the Messaging Service SID (as a String) instead of a PhoneNumber object as the from parameter. I recommend keeping this SID as an environment variable, as with all sensitive IDs.

Using a Messaging Service can help with carrier rate-limiting, recipient-number locality and customising opt-out. Alpha Sender is a great feature to help people know who's messaging them too:

Mobile screenshot: an SMS app showing "Hello from Twilio". The sender is "Owl Airline"

However, you're still making one API call per message. At scale, making lots of API requests one after another is slow — perhaps you'll try to parallelize them but that's harder to coordinate, and failures become harder to track and recover from. You will still have the risk of API rate limits, too. If you're worried about these kinds of things, then read on.

Notify Services

The next tool in your belt should be a Notify Service. These let you send a message to multiple recipients in one API call. It is also possible to send mobile push notifications and integrate with Facebook Messenger and Alexa, but this example stays with SMS by creating a Notify Service which wraps the Messaging Service we used above. Being able to send SMS to multiple recipients in a single API call can be very convenient, with the caveat that it has to be the exact same message to each recipient.

Creating a Notify Service

Create a new Notify Service on the Notify Services page in your Twilio console. To configure it select the Messaging Service and save your changes. You will need to use the Service SID, starting IS, to use the Notify Service from code.

You can also create Notify Services using the API or the Twilio CLI.

To send messages using the Notify Service you need to provide a list of recipients. Because Notify can send messages over different channels this is behind an abstraction called a Binding.

You can either create Bindings through the API and reference them when sending a message, or provide the Bindings as you send your messages. Let’s use the latter approach here, which means creating a List<String> of Bindings where each entry is a short snippet of JSON looking like {"binding_type":"sms", "address": NUMBER}. There are many ways to generate these, but this code is sufficient:

String bindingFormat = "{\"binding_type\": \"sms\", \"address\": \"%s\"}";

List<String> bindings = Stream.of(
        "Recipient Phone Number #1",
        "Recipient Phone Number #2",
        "Recipient Phone Number #3" // etc
    )
    .map(phoneNumber -> String.format(bindingFormat, phoneNumber))
    .collect(Collectors.toList());

Notification notification = Notification.creator(
        System.getenv("TWILIO_NOTIFY_SERVICE_SID"))
    .setToBinding(bindings)
    .setBody("Multiple SMS sent in one API request using a Notify Service 🎉")
    .create();

[this code on GitHub]

The upper limit for the number of recipients is dictated by the 1MB size limit on HTTP requests to the Twilio API — if you exceed the limitthe API will return an HTTP/413 response. Typically 10,000 bindings is safe. If you need more than that, we recommend splitting the bindings across multiple requests.

Mobile Screenshot: the same SMS app as before.

Because the Notify Service uses the Messaging Service you still have the benefit of the sender-pool to help with carrier rate-limiting, recipient-number locality and customising opt-out, as well as Alpha Sender. With the ability to send a message to 10,000 recipients in one API call you have less work to do with coordination and error handling in your code, and less chance to reach the Twilio API rate limit.

What happened to my messages?

There are a few options for keeping track of messages that are sent through a Notify Service.

You can check the Notify Service logs page in your console for a summary of the results:

Screenshot: notification details showing one sent SMS and one failed because the number wasn't a real phone number.

Messages which pass phone number validation also appear in the Programmable Messaging Logs page, which can be fetched in code using Message.reader(). For example this code prints the status of all messages sent in the last 2 hours:

Message.reader()
    .setDateSentAfter(ZonedDateTime.now().minus(2, ChronoUnit.HOURS))
    .setDateSentBefore(ZonedDateTime.now())
    .read()
    .forEach(m -> System.out.println(m.getTo() + ": " + m.getStatus()));

Another option is to set a Delivery Callback URL when you create the notification. When delivery is complete an HTTP call will be made to that URL which includes the delivery state of all the messages you requested:


Notification notification = Notification.creator(
        System.getenv("TWILIO_NOTIFY_SERVICE_SID"))
    .setToBinding(bindings)
    .setBody("Multiple SMS sent in one API request using a Notify Service 🎉")
    .setDeliveryCallbackUrl("<YOUR DELIVERY CALLBACK URL>")
    .create();

[this code on GitHub]

A final option is to set a status callback URL which does the same as when you set it for a single SMS as shown in the first "what happened to my message?" section above. You will receive HTTP requests to that URL with details of any change of status for each message. This is typically at least two requests per SMS. Configure it like this:


Notification notification = Notification.creator(
        System.getenv("TWILIO_NOTIFY_SERVICE_SID"))
    .setToBinding(bindings)
    .setBody("Multiple SMS sent in one API request using a Notify Service 🎉")
    .setSms(Map.of("status_callback", "<YOUR STATUS CALLBACK URL>"))
    .create();

[this code on GitHub]

You can mix and match these as you like — set a status callback for each SMS, a delivery callback for the whole lot, and check it manually if you need to.

Short Codes

If you are regularly sending high volumes of SMS it might be worth considering using a short code — a 5 or 6 digit number which can send and receive SMS. These are more expensive than regular phone numbers but are pre-approved by carriers to have throughput of 100 Message Segments per Second and are not subject to carrier filtering. Short codes can be added to Messaging Services and will be used preferentially if possible. You can configure a fallback to long code if the short code message fails (for example due to carrier restrictions or handset preferences).

Wrapping up

You've seen how to use Messaging Services and Notify Services to scale up and keep track of up to tens of thousands of SMS messages. If you have any questions about this or anything else you're building with Twilio, I'd love to hear from you.