Texting emojis and non-Latin characters with C#, .NET and Twilio SMS

Time to read: 5 minutes

November 29, 2018
Written by
Daniel Phiri
Contributor
Opinions expressed by Twilio contributors are their own

texting-csharp.png

As technology expands the reach of our products and services it’s important for us to create content to suit users from all over the world. As we primarily use Latin characters for programming, creating content for languages like Japanese or Arabic that have a different alphabet is not as easy as typing them on a standard keyboard. This post will show you how to send non-Latin characters, including emoji, in text messages with C# and Twilio Programmable SMS.

Prerequisites

You should have the following tools installed to complete the program described in this post:

If you would like to see a full integration of Twilio APIs in a .NET Core application then checkout this free 5-part video series. It's separate from this blog post tutorial but will give you a full run down of many APIs at once.

Why UTF-8?

The problem started when people realized that the character encoding standard of the time, ASCII, needed to be extended to accommodate the alphabets of other languages. In the late 1980's more than 10 different character sets were developed to represent the alphabets of languages like Russian, Arabic, Hebrew, and Turkish. It became cumbersome to keep track of and manage the different character sets.

That’s why the Unicode standard was developed. It integrated all these character sets into a single encoding standard that boasts over 100,000 different characters from 146 languages. There happens to be a really great Smashing Magazine article explaining Unicode and other encoding methods.

UTF-8 is a variable width character encoding that represents all the Unicode characters in one to four 8-bit bytes. UTF-8 is the dominate character encoding standard for the web, in use on over 90 percent of web pages.

C# escape characters

A convenient way to include UTF-8 non-Latin characters in C# programs is by using  character escape sequences. Escape sequences let us represent characters that have a specific meaning to the compiler, like quotation marks, which are used to denote text strings, as literal characters. For example, to store a string value in a variable, we use quotation marks to show that the text is string.

To store the sentence “Scripts over everything!” as a string in a variable, we write the following line of code.

string message = “Scripts over everything!”;

When we run it, it works just fine and the message is printed.

However, if our text has quotation marks in it we will see a number of errors if we write the code below:


String message = “Drew said “Scripts over everything” ”;

To fix this we use escape characters.

For this case, we use the \” escape sequence, which lets us add quotation marks as part of the string. The code should look like this:

String message = "Drew said \"Scripts over everything\" ";

Similarly, to use non-Latin characters supported in UTF-8 we need to use the \u escape sequence followed by the hexadecimal notation for the specific character we want to use. We’ll get deeper into this in the next section.

Using UTF-8 in code

To get started with the case study solution we'll begin by creating a new project. Open Visual Studio 2017 and select File > New > Project. You'll see a dialog box asking what type of project we want to create. We need to select Console App under .NET Core. Let's name the app “textio”.

Click OK. You should see the following code in the Program.cs file:

using System;

namespace textio
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Hello World!");
        }
    }
}

We need to install the Twilio .NET package using the NuGet Package Manager GUI, Package Manager command line interface (CLI), or .NET command line interface. Twilio has an awesome walk through on how to do this. In this tutorial, we'll use the NuGet Package Manager CLI. Open the console from the menu bar by selecting Tools > NuGet Package Manager > Package Manager Console. After it opens, enter the command Install-Package Twilio to install the package.

Add types to the namespace

We need to make the compiler aware that we’re using certain types in our program. It’s also useful because it stops us from prepending namespaces when using their methods. For example, by including the line using System;, we can simply say Console.Write(message) instead of System.Console.Write(message). Add the following using directives:

using System.Diagnostics;
using System.Globalization;
using Twilio;
using Twilio.Rest.Api.V2010.Account;
using Twilio.Types;

Your Program.cs file should look like this now:

using System;
using System.Diagnostics;
using System.Globalization;
using Twilio;
using Twilio.Rest.Api.V2010.Account;
using Twilio.Types;

namespace textio
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Hello World!");
        }
    }
}

Add Twilio credentials

Before we continue we need to create a Twilio account. Head over to their website and sign up for a free account. Once the sign up process is completed, we need to add our Twilio Account SID and Auth Token to use the Twilio Programmable SMS service in our code. You can get these in your Dashboard once you’ve completed the sign-up process.

It’s important to remember that when working with production code we don’t enter secret keys or tokens as we are going to do here. Another post on the Twilio blog brilliantly shows how to handle developer secrets in production code.

We’ll also need a Twilio phone number. You can a free trial number from your Twilio Dashboard under Programmable SMS. Be sure to send yourself a trial message with the phone number you used to register with Twilio. You can only send text messages to registered phone numbers with a trial account. Of course, the phone number you use to register with Twilio must be able to receive SMS messages.

Now that we have our Account SID, Token, and Phone Number we delete the Console.WriteLine("Hello World!"); statement from Program.cs and replace it with the code below:

// Find your Account Sid and Auth Token at https://twilio.com/console and add them below
const string accountSid = "<Your Twilio Account SID>";
const string authToken = "<Your Twilio Auth Token>";
TwilioClient.Init(accountSid, authToken);

// Add the number you want to send a text to here
string rec = "<Your recipient's phone number>";

//Add your Twilio phone number here
string phone = "<Your Twilio Phone Number>";

Replace the placeholders in the code above with the information from your Twilio dashboard. Use a registered phone number as the recipient’s phone number so you can send messages to it from your trial account.

Send the first text message

In the last line of the Main method in Program.cs, right after var phone = "<Your Twilio Phone Number>";, paste the code below:

 // The message you want to send comes here
 Console.WriteLine("Enter your message:"); // Prompt
 string text = Console.In.ReadLine();
 text = DecodeEncodedNonAsciiCharacters(text);

var to = new PhoneNumber(rec);
// Putting everything together and sending the text
var message = MessageResource.Create(
    to,
    from: new PhoneNumber(phone), body: text);

Console.WriteLine("\nYou just sent a text with Twilio *Mic Drop* ");
Console.WriteLine(message.Sid);
Console.Write("\nSuccess! Press any key to exit...");
Console.ReadKey(true);

You’ll notice that this code includes a reference a new method, DecodeEncodedNonAsciiCharacters. We are going to use the Windows command line to input the SMS messages we will send. By default the Windows command line does not support UTF-8 characters or escape sequences. This new method will take the text input from the command line, including the text for the escape sequences, and convert it to escape sequences in C#.

private static string DecodeEncodedNonAsciiCharacters(string value)
{
    return System.Text.RegularExpressions.Regex.Replace(
        value,
        @"\\u(?<Value>[a-zA-Z0-9]{4})",
        m => {
            return ((char)int.Parse(m.Groups["Value"].Value, NumberStyles.HexNumber)).ToString();
        });
}

The code blocks above strap everything together, MessageResource.Create gets your number, the recipient, and the message, then sends the message using Twilio. The last few lines output a message to the command line containing the message ID (SID) returned by Twilio to confirm your message was sent.

Send a text message

Test the code with text. Run the application by pressing F5, enter the message “Hello! Is it memes you’re looking for?” at the command line prompt and press Enter. You should receive a text message like this:

Send a message with UTF-8 characters

We can send a message in Japanese using escape sequences. “Hello” in Japanese is こんにちは which can be represented by the escape sequence \u3053\u3093\u306B\u3061\u306F

Pretty cool! I used the r12a Unicode converter app to convert the Japanese text to its escape sequence.

Run the app with F5 and enter “\u3053\u3093\u306B\u3061\u306F” when prompted for your message.

We should get lovely greeting in Japanese as show below:

Lets go a step further and add some emoji ✨, I used the r12a app again to get some emoji escape sequences. Run the app, enter "\u3053\u3093\u306B\u3061\u306F \u2728" and press Enter when prompted for a message.

We get this:

I’m a huge movie buff. Replace this message with the following code and see what you get!

`"\uD83D\uDC00 \uD83C\uDF72 \uD83D\uDC68\u200D\uD83C\uDF73 \uD83D\uDC45";`

Can you guess what movie this is?

Summary

That’s it! We’re sending texts in any language supported by UTF-8. Today, we went through UTF-8, what it is and why it's important. We found out about escape sequences and how they help us manipulate strings. We particularly learned to use these escape sequences to add UTF-8 characters in our texts and sent some dope texts in Japanese. Hopefully this makes it easier for your products to reach a larger audience.

Additional resources

You can find the complete project code on GitHub.

Feel free to reach out to me with questions on Twitter @malgamves, Instagram @malgamves or GitHub.

Related Posts

Related Resources

A newspaper article

Twilio Docs

From APIs to SDKs to sample apps

API reference documentation, SDKs, helper libraries, quickstarts, and tutorials for your language and platform.
An Open book

Resource Center

The latest ebooks, industry reports, and webinars

Learn from customer engagement experts to improve your own communication.
User group reactions

Ahoy

Twilio's developer community hub

Best practices, code samples, and inspiration to build communications and digital engagement experiences.