Analyze Texts using IBM Watson and Twilio Functions with Node.js

Time to read: 4 minutes

June 18, 2020

Written by

Lizzie Siegle

Twilion

Dave Nugent

Contributor

Reviewed by

Diane Phan

Twilion

Phil Nash

Twilion

Analyze Texts using IBM Watson and Twilio Functions with Node.js

The IBM Watson supercomputer is perhaps best known for beating two quizmasters on Jeopardy, but it can do so much more, such as diagnose cancer, fight crime, and more. Watson also provides APIs and tools to help developers build intelligent models and apps. This post will use deep learning to extract relevant information such as texts, like entities, keywords, categories, sentiment, emotion, and more from a text message with Watson Natural Language Understanding (NLU) and Twilio Functions.

This post was written with IBM developer advocate Dave Nugent for a joint workshop.

updated text gouda baseball ibm sentiment

Prerequisites

To follow along with this post, you need three things:

A Twilio account - sign up for a free one here and receive an extra $10 if you upgrade through this link
A Twilio phone number with SMS capabilities - configure one here
IBM Cloud account - sign up for a free one here

Now let's get started with IBM Watson!

Make a Watson Natural Language Service

To make our Watson NLU service, go to your IBM Cloud dashboard and click the blue Create Resource button in the top right corner.

Next, type "natural language" in the search bar that says Search the catalog and click enter.

You should see quite a few options. Select Natural Language Understanding.

The lite plan should automatically be selected. Click the blue create button.

Whoop whoop! You've just made your free Watson Natural Language Understanding service!

To get our required credentials, click the Navigation Menu in the top left corner.

Select Resource list from the dropdown menu followed by your Natural Language Understanding resource.

Keep this page with your API key and URL (which are unique to you) open--we'll use those in the next step!

Call your Watson NLU Service from a Twilio Function

One underrated Twilio product (in my opinion) is Twilio Functions. Functions is a serverless environment for Node.js applications. It allows developers to get up-and-running quickly.

Make your way to the Configure section of your Twilio Functions console. To add your IBM Watson NLU apikey and url copied from the last section as environment variables, click the red plus button near Environment Variables and name the two values as "watson_apikey" and "watson_url". Add your corresponding values accordingly. With our environment variables set, it's time to import the node modules required for this project.

Click the red plus button near Dependencies. Add ibm-watson as a package. Feel free to use a * in the version field to indicate the latest version and then click Save at the bottom!

Nice! Now we can access our IBM Watson credentials in any of our Twilio Functions. Let's make one now.

Make a Twilio Function

Above the Configure button on the left, select Manage and make a new Function by clicking the red plus symbol and then select "+ Blank" to make a blank Function. Give your Function a name like "IBM Watson SMS analysis" and a path like "/ibm".

Nice! We have a Twilio Function.

Analyze Texts with Watson Natural Language Understanding

Replace all the code already in your Function with

const NaturalLanguageUnderstandingV1 = require('ibm-watson/natural-language-understanding/v1');
const { IamAuthenticator } = require('ibm-watson/auth'); 
exports.handler = function(context, event, callback) {
    let twiml = new Twilio.twiml.MessagingResponse();
    console.log(event.Body);
  const analyzeParams = {
    'text': event.Body,
    'features': {
      "sentiment": {},
      "categories": {},
      "concepts": {},
      "entities": {},
      "keywords": {}
    }
  };
  const naturalLanguageUnderstanding = new NaturalLanguageUnderstandingV1({
    version: '2020-06-12',
    authenticator: new IamAuthenticator({
      apikey: context.watson_apikey,
    }),
    url: context.watson_url,
  });
  naturalLanguageUnderstanding.analyze(analyzeParams)
    .then(analysisResults => {
        const score = analysisResults.result.sentiment.document.score;
        const label = analysisResults.result.sentiment.document.label;
        let categories = analysisResults.result.categories;
        let keywords = analysisResults.result.keywords;
        categories = categories.filter(item => item.score > 0.8).map(item => item.label).join(', ').replace(/\\|\//g,', ');
        categories = categories.substring(1).split(','); //remove initial comma
        keywords = keywords.reduce((a,v,i) => {return a + (i===0?"":", ") + v.text}, "");
        const msg = "We detected " +label + 
            " sentiments, and identified the categories " +
            categories + " relating to keywords " + keywords;
	    twiml.message(msg);
	    callback(null, twiml);
    })
    .catch(err => {
      callback(null, 'Error: ' + err);
    });
};

That's a lot of code! Let's walk through it.

We import Watson NLU and authentication at the top with

const NaturalLanguageUnderstandingV1 = require('ibm-watson/natural-language-understanding/v1');
const { IamAuthenticator } = require('ibm-watson/auth');

Then in exports.handler we make a Twilio Messaging Response and make the analyzeParams variable that tells Watson what to do. Here, we tell Watson to analyze the inbound SMS message providing us information about sentiment, categories, concepts, entities and keywords. They are initialized as empty objects to return the default options for those categories. You could add parameters if you want to limit the number of responses, for example.

 const twiml = new Twilio.twiml.MessagingResponse();
    const analyzeParams = {
        'text': event.Body,
        'features': {
        "sentiment": {},
        "categories": {}
        }
    };

You could receive less data by changing or removing these properties in features.

Next, we instantiate a natural language understanding API client and authenticate it with our API Key. The Watson library provides an authenticator object IamAuthenticator we can use to do that. This is where we access our environment variables by appending the variable names to context so "watson_apikey" is referenced with context.watson_apikey.

const naturalLanguageUnderstanding = new NaturalLanguageUnderstandingV1({
        version: '2020-06-12',
        authenticator: new IamAuthenticator({
            apikey: context.watson_apikey,
        }),
        url: context.watson_url,
    });

For more information on IAM authentication, check out the npm module page for Watson.

Lastly, we call the Watson NLU API with the parameters we built up. This returns a promise whose results, when resolved, helps build a response message containing a sentiment score, label, and categories.

naturalLanguageUnderstanding.analyze(analyzeParams)
    .then(analysisResults => {
        const score = analysisResults.result.sentiment.document.score;
        const label = analysisResults.result.sentiment.document.label;
        let categories = analysisResults.result.categories;
        let keywords = analysisResults.result.keywords;
        categories = categories.filter(item => item.score > 0.8).map(item => item.label).join(', ').replace(/\\|\//g,', ');
        console.log(`keywords: ${JSON.stringify(keywords)}`);
        keywords = keywords.reduce((a,v,i) => {return a + (i===0?"":", ") + v.text}, "");
        categories = categories.substring(1).split(','); //remove initial comma
        const msg = "We detected " +label + 
            " sentiments, and identified the categories " +
            categories + " relating to keywords " + keywords;
	    twiml.message(msg);
	    callback(null, twiml);
    })
    .catch(err => {
      callback(null, 'Error: ' + err);
    });
};

Tada! You can hit this Function with an HTTP request with a tool like Postman, but let's configure a Twilio phone number to analyze texts using IBM Watson.

Configure your Twilio Phone Number with a Twilio Function

If you don't have a Twilio number yet, go to the Phone Numbers section of your Twilio Console and search for a phone number in your country and region, making sure the SMS checkbox is ticked.

In the Messaging section of your purchased number, in the A Message Comes In section, set the dropdown menu to Function instead of Webhook and then on the right select your Function from the larger dropdown menu, as shown below. Hit Save.

Whip out your phone and text your Twilio number to see IBM Watson analyze your text! Feel free to play around with what Watson analyzes and send different text messages.

You can view this on Dave's GitHub.

What's Next for Analyzing Texts with Watson

With Watson, you can quickly analyze and infer different tidbits much better from the texts. How is someone feeling? Do they need an animal gif even if they don't outright say so? You can perform sentiment analysis with TensorFlow.js, in a phone call, and more. Let us know online or in the comments what you're building!