Using Bots To Route Customer Requests Based On Sentiment and Emotion

October 12, 2016
Written by
Al Cook
Twilion

Twilio Bug Logo

2016: the year where no strategy or vision pitch was complete without mentioning bots. 

You can’t watch a tech keynote, scroll through your newsfeed, or be anywhere online without reading how bots are replacing apps, or replacing humans.  

Assuming though for just a moment that we don’t turn our every human interaction, from wedding vows to childcare, into an AI driven chat based interaction… we have a question to answer: what is a realistic view of how companies could be using bots today? I’m particularly interested in the possibilities for using bots within a call center (But not as a replacement for humans – despite the hype we’re not a fully virtual society quite yet).

Sentiment driven routing

To explore these ideas, I built a call center prototype to look at ways to merge human and bot interaction together. I’ve been chewing on a few questions: Could you have customers chat with a bot first to better determine their intent, and even emotional state, and use that information to connect them to a better matched agent? Could you save the agent time by having the bot capture key information first and inform the agent when they take the interaction? What about handling self-service questions entirely automatically without ever passing the chat to a human agent?

My prototype handles inbound interactions coming in over both SMS and from Twilio’s new Facebook integration, all routed by TaskRouter. I also used Marketplace AddOns for details about the users texting in, along with Meya.ai for the bot platform, and Firebase. The code for this is all available in github, so you can follow along as we go through the architecture.

A customer messaging in ‘hey I could do with some help’ will get routed in a completely different way to someone messaging ‘You guys really suck I can’t believe you still haven’t fixed this’. And someone messaging ‘what are your opening hours?’ doesn’t need to be routed to an agent at all.

Customers messaging in first cause a task to be created in TaskRouter. The Task serves as the primary key for the entire lifecycle of the customer interaction. When the task is created, it sits in a queue waiting to be bot qualified, and my app server connects the messages back and forth with the bot platform. The customer first chats with a bot, which determines their intent and emotional state.

Sending messages related to unqualified tasks to the bot:

client.workspace.tasks(taskSid).get(function(err, task) {
   attr = JSON.parse(task.attributes);
   if (!attr.hasOwnProperty('bot_qualified')) {
     console.log("this task is not yet bot qualified");
     console.log("posting to meya with user id " + meyaUserID_string + " and text " + request.body['Body']);
     req
       .post('https://meya.ai/webhook/receive/BCvshMlsyFf').auth(meyaAPIKey).form({
         user_id: meyaUserID_string,
         text: request.body['Body']
       })
       .on('response', function(response) {
         console.log("got response from meya " + response);
       })
   } else {
     console.log("this task is already bot qualified");
   }
 });

The Meya bot platform uses an easy scripting interface to storyboard the interactions. It starts by gathering the intent of the first message, and then transitions between different states from there depending on what’s said – the sequence will flow through to the next state unless you specify a transition to a different state.

 

intents:
 misunderstood: help
 hi: hi
 how_are_you: how_are_you
 help: help
 whats_up: whats_up
 who_are_you: who_are_you
states:
 how_are_you:
   component: meya.text
   properties:
     text: I'm good! Thanks for asking!
   transitions:
     next: delay
 whats_up:
   component: meya.text
   properties:
     text: Oh we're just chilling.
   transitions:
     next: delay
 who_are_you:
   component: meya.text
   properties:
     text: I'm a bot, I will gather some information first and then pass you to an
       agent who can help
   transitions:
     next: delay
 hi:
   component: meya.random_text
   properties:
     responses:
     - Hi :)
     - Hello, there!
     - Howdy!
     - Bonjour.
 delay:
   component: al_delay
 help:
   component: meya.wit
   properties:
     text: How can I help you with British Exports?
     require_match: false
     token: <wit.ai token>
   transitions:
     angry: angry
     happy: happy
     needs_help: needs_help
     problem: problem
     service_question: service_question
     no_match: unsure_state
 angry:
   component: intent_checker
   properties:
     text: I'm really sorry. Would you mind if we chat a bit more and I can see if
       I can help make things better?
     emotion: angry
   return: true
 happy:
   component: intent_checker
   properties:
     text: I'm glad to hear it. Let me send you a free t-shirt to show our gratitude.
       :)
     emotion: happy
   return: true
 needs_help:
   component: intent_checker
   properties:
     text: I can definitely help you out. I'm going to need to ask you a few more
       questions.
     emotion: needs_help
   return: true
 problem:
   component: intent_checker
   properties:
     text: We'll get that fixed ASAP. One moment please.
     emotion: problem
   return: true
 service_question:
   component: intent_checker
   properties:
     text: You're asking the right person. Let me ask you a couple of questions so
       I can get you the answer you want.
     emotion: service_question
   return: true
 unsure_state:
   component: intent_checker
   properties:
     text: Sorry...let me pass you on to someone who can better help with that
     emotion: unsure
   return: true

When a bot gets an answer to the question ‘how can i help’, it uses Wit to determine sentiment. Wit is really easy to train from a data set of responses what the intent of the interaction is. The more you train it, the better is is at handling different variations of what the customer might say.

 

screenshot-2016-10-07-10-50-16

Once the bot has determined the intent, we’re ready to update the task attributes in TaskRouter to say that the task has been bot qualified, and mark their intent. In this case, it assigns each task to one of the following states: angry, happy, needs_help, problem, service_question, or unsure. To do this, I used the native Twilio integration available within the bot platform Meya.ai, so that my bot logic directly calls the update task API with the new attributes.

from meya import Component
import re
import json
from twilio.rest import TwilioTaskRouterClient

class IntentChecker(Component):
   def start(self):
       account_sid = <account sid>
       auth_token  = <auth token>
       client = TwilioTaskRouterClient(account_sid, auth_token)
       # read in the response text, and default to empty if invalid or missing
       text = self.properties.get('text') or ""
       # meyaUserID=JSON.loads(self.db.user.user_id)
       meyaUserID = self.db.user.user_id.split('@@')
       taskSid=meyaUserID[2]
       task = client.tasks("WS056355824815f89c7cc46e5d8cacaf20").get(taskSid)
       task_attributes= json.loads(task.attributes)
       task_attributes['bot_qualified']='true'
       task_attributes['bot_intent']=self.properties.get('emotion')       
       print task_attributes
       attribute_string=json.dumps(task_attributes)
       task = client.tasks("WS056355824815f89c7cc46e5d8cacaf20").update(taskSid,attributes=attribute_string)
       message = self.create_message(text=text)
       return self.respond(message=message)

With the task updated, TaskRouter then moves the task into the appropriate queue so the best agents can handle the query. The gif below shows a realtime visualization of a TaskRouter workspace (code for this is also in github).

 

7OgV6Pld8gXyR34ZRKI9uQKtjWQ8Xywd6bM9LYuh83Tj0pansfKi2eF0oWTFiraxhjJG-w4JL3-7obiFN0z94-6VYBaC5YGRo5sI5QC4KNWG5GakGEQDVLgh_UTGiXsIoVdOM08-

Once a task is bot qualified, and a suitable agent becomes available based on the determined intent, TaskRouter will push a reservation request to the agent. At this stage the agent gets to see all of the conversation history so they can get up to speed immediately and continue the conversation.

One of the key benefits of using messaging for customer service is that agents can handle multiple messaging interactions simultaneously. So I used the new multitasking feature of TaskRouter to be able to specify how many tasks the agent can handle concurrently. As you change capacity of the worker, they get additional task reservations:

app.get('/updateCapacity', function(request, response) {
 // This function uses the TaskRouter multi-tasking API to change concurrent task capacity
 var options = {
   method: 'POST',
   url: 'https://taskrouter.twilio.com/v1/Workspaces/' + workspaceSid + '/Workers/' + request.query.workerSid + '/Channels/default',
   auth: {
     username: accountSid,
     password: authToken
   },
   form: {
     Capacity: request.query.capacity
   }
 };
 console.log(options);
 req(options, function(error, response, body) {
// snip code that goes on to handle request…
});
 response.send('');
});

 

multitasking

And finally I also used Marketplace AddOns to get the name and address of folks texting in:

try {
             var addOnsData = JSON.parse(request.body.AddOns);
             friendlyName_first = addOnsData['results']['nextcaller_advanced_caller_id']['result']['records'][0]['first_name'];
             friendlyName_last = addOnsData['results']['nextcaller_advanced_caller_id']['result']['records'][0]['last_name'];
             address_street = addOnsData['results']['nextcaller_advanced_caller_id']['result']['records'][0]['address'][0]['line1'];
           } catch (err) {}
           myFirebase.child("profiles").child(newTaskResponse.sid).set({
             'first_name': friendlyName_first,
             'last_name': friendlyName_last,
             'address_street': address_street,
             'message_type': 'sms',
             'profile_pic': 'img/unknownavatar.jpeg'
           });

So now we have a customer service solution where users can message in through SMS or Facebook, and based on what they say they need help with (and how they say it), they will get routed to the best qualified agent. The agent will have all the context of the conversation so far, and details about the customer from the Marketplace AddOns lookup of their phone number. The agent can choose how many customers they want to handle concurrently using the new multi-tasking capability of TaskRouter.

You can hear me talk more about this demo in this video from SIGNAL. All my code is available in github. The back end is all Node, the front end uses Foundation, and the bot platform used was meya.ai (who are phenomenally helpful, nice people). Disclaimer: I hadn’t coded anything in ten years, and had never used Node or any of these tools before, so in no way is this production code. Pull Requests welcome!