Supercharge your contact centre experience with Flex and Veridas Voice Biometrics

Time to read: 9 minutes

November 19, 2021
Written by
Ana Cristina Andres Del Valle
Twilion
Reviewed by
Andrej Saweljew
Twilion

Supercharge your contact centre experience with Flex and Veridas Voice Biometrics

Wouldn't it be great to verify the identity of the person who is at the other side of a phone call just using their voice? Agents in contact centres spend precious time asking security questions to verify customers’ identities. This process is not friendly and often relies on information that can be phished – stolen by untrustworthy third parties. Furthermore, from the customer's point of view, it is also an arduous and bewildering process. Either they have to spend an average of 30 seconds answering personal questions, or they are forced to memorize passwords that are often forgotten. In the end, these processes do not actually verify an identity but simply check that a person has access to certain information.

Voice biometrics can help contact centres create friendly and natural ways to verify who is speaking on the phone. Veridas Voice Biometric Authentication solution services facilitate the registration of a user’s biometric credentials when calling a contact centre. Later, any agent will be able to verify or identify that person through their voice.

In this post, you will learn how to add voice biometrics into Twilio's contact centre platform, Flex. We will show you how to integrate the Veridas Voice Biometrics plugin for Flex.

Prerequisites

Before you can start the tutorial, you’ll need to set up a few accounts.

  1. A Twilio Flex Account. If you do not have one yet, follow this guide to set up your Flex account for free.
  2. The Twilio CLI and the Flex plugin for the CLI.
  3. Veridas client secrets. You can get them after filling this form for the Veridas Voice Biometric Authentication solution for Twilio.
  4. The Veridas Voice Biometrics npm packages, provided by Veridas.
  5. A secure way to store ids and biometric credentials. 

In our example, we will be using the Veridas Voice Biometric Authentication Plugin in testing mode. We will focus on integrating the voice-biometrics in the contact centre; therefore, we will be simulating the storage and retrieval of the biometric information using a cookie in the browser displaying the Flex agent interface. We will also indicate where the code to store and retrieve credentials from the DB would go when you develop a production solution.

How to Build the Integration

Install the Twilio Flex plugin CLI

First of all, you will need to set up a Flex Plugin development environment. To do so please follow this quickstart that will show you how to create a Flex plugin from scratch. You can also check this past blog post for inspiration on how to make the most of Flex Plugins.

In any case, you will need to work with the Twilio CLI and the Flex Plugins CLI.

npm install twilio-cli -g
twilio plugins:install @twilio-labs/plugin-flex

Connect your Twilio Flex account to the CLI.

twilio profiles:create

If you have already installed the Twilio CLI and plugins, check that you are using the right account:

twilio profiles:list

Create the Veridas Flex plugin

We will first create the caliope plugin using the Twilio CLI

twilio flex:plugins:create veridas-plugin --install

Or, if you prefer a typescript project, run 

twilio flex:plugins:create veridas-plugin --install --typescript

To see the Plugin running locally visit http://localhost:3000

cd veridas-plugin 
twilio flex:plugins:start

Create the Veridas component in your plugin

Create the Webcomponent

Part of the Veridas integration will be run as a WebComponent on the Flex UI.

Install WebComponent from NPM:

npm install --save webcomponent

Where ${PACKAGES_PATH} contains the location where you have saved the files you were sent by Veridas as part of the prerequisites.

Replace the X.Y.Z for the version released by Veridas.

Create the VeridasComponent class

To create the Veridas Component to be inserted in Flex:

  • Create the folder  src/components/veridas/ 
  • Create a VeridasComponent.js file inside it, with the following React code:
import React from 'react';
import { TaskHelper, withTaskContext } from '@twilio/flex-ui';
import '@veridas/caliope-web-agent-twilio';

import { developmentConfig, urlsConfig, parametersConfig } from './Config.js';
import { externalMethods } from './Utils.js';

class VeridasComponentAux extends React.Component {
    component = null
    constructor(props) {
        super(props);
        this.referenceComponent = React.createRef();
    }

    componentDidMount(){
        const component = this.referenceComponent.current;
        component.developmentConfig = developmentConfig;
        component.urlsConfig = urlsConfig;
        component.parametersConfig = parametersConfig;
        component.externalMethods = externalMethods;
    }

    render(){
        const { task } = this.props;
        let isLiveCall = TaskHelper.isLiveCall(task).toString();

        let conferenceSid = null;
        if (typeof task.conference !== 'undefined') {
            if("conferenceSid" in task.conference){
                conferenceSid = task.conference.conferenceSid;
            }
        }
        
        return (
            <caliope-web-agent-twilio 
                ref={this.referenceComponent}
                securityToken={this.props.token}
                callSid={task.attributes.call_sid}
                callerId={task.attributes.caller} 
                taskSid={task.taskSid}
                conferenceSid={conferenceSid}
                callInProgress={isLiveCall}/>
        );
    }
}
    
export default withTaskContext(VeridasComponentAux);

The Veridas plugin uses several Flex parameters to work. DO NOT EDIT THEM or the plugin will not work as expected. The following parameters should not change:

  • securityToken: It is a security measure recommended by Twilio documentation for calling Twilio functions from Flex plugins.
  • callSid: Current call identifier.
  • callerId: Customer telephone number.
  • taskSid: Current flex task identifier.
  • conferenceSid: Unique identifier of the current conference.
  • callInProgress: bool value, true if call is active.

 

Include VeridasComponent in your code

Import the created component inside the Flex project (VeridasPlugin.js) like this:

import VeridasComponent from './components/veridas/VeridasComponent.js';

Create the config and utils files

In the same Veridas folder, the following two files need to be created:

1. Config.js:

 

export const developmentConfig = {
    mockApi: false,
    mockResponseType: "",
    automaticVerification: false,
    deleteOnlyAfterVerification: false
};

export const urlsConfig = {
    urlSio: '',
    urlAudioConsumer: '',
    urlGetTemporaryToken: '',
    urlStartStreaming: '',
    urlStopStreaming: ''
};

export const parametersConfig = {
    scoreThreshold: '0.8',
    authenticityThreshold: '0.0',
    audioTotalDurationRegistration : '3.0',
    audioTotalDurationVerification : '3.0'
};

To use this plugin, get the following parameters from Veridas:

  • urlSio: URL of the server API.
  • urlAudioConsumer: URL of the audio consumer service.

Also, you'll need to configure following parameters after the section Create Veridas Service with streaming functions

  • urlStartStreaming: URL to the startStream Twilio Function you will develop later
  • urlStopStreaming: URL to the stopStream Twilio Function you will develop later
  • urlGetTemporaryToken: URL to the getTemporaryToken Twilio Function you will develop later
2. Utils.js:

 

function storeCredential(id, credential) {
 /*
  * ***SAMPLE IMPLEMENTATION***
  *
  * This method is used to store the credentials generated
  * inside the webcomponents. It uses the browser cookies
  * for this purpose.
  *
  * Here can be implemented an integration with YOUR OWN DATABASE
  * to store the credentials.
  *
  * Input:
  *   - id: identifier for the current client (phone number by default)
  *   - credential: credential generated
  */
 console.log(`storeCredential id = ${id}`);
 const d = new Date();
 d.setTime(d.getTime() + 30 * 24 * 60 * 60 * 1000);
 const expires = `expires=${d.toUTCString()}`;
 document.cookie = `${id}=${credential};${expires};path=/`;
}

function deleteCredential(id) {
 /*
  * ***SAMPLE IMPLEMENTATION***
  *
  * This method is used to delete a credential already stored.
  * It deletes the browser cookie previously generated.
  *
  * Here can be implemented an integration with YOUR OWN DATABASE
  * to delete the credentials.
  *
  * Input:
  *   - id: identifier for the current client (phone number by default)
  */
 console.log(`deleteCredential id = ${id}`);
 const d = new Date();
 d.setTime(d.getTime() + -1 * 24 * 60 * 60 * 1000);
 const expires = `expires=${d.toUTCString()}`;
 document.cookie = `${id}=;${expires};path=/`;
}

function getCredential(id) {
 /*
  * ***SAMPLE IMPLEMENTATION***
  *
  * This method is used to obtain an already stored credential.
  * It retrieves the browser cookie previously generated.
  *
  * Here can be implemented an integration with YOUR OWN DATABASE
  * to retrieve the credential.
  *
  * Input:
  *   - id: identifier for the current client (phone number by default)
  *
  * Output:
  *   Credential stored for the given user id
  */
 console.log(`getCredential id = ${id}`);
 const name = `${id}=`;
 const ca = document.cookie.split(';');
 for (let i = 0; i < ca.length; i += 1) {
   let c = ca[i];
   while (c.charAt(0) === ' ') {
     c = c.substring(1);
   }
   if (c.indexOf(name) === 0) {
     return c.substring(name.length, c.length);
   }
 }
 return '';
}

function onResponse(result) {
 /*
  * This method is called whenever the webcomponents receive a result
  * from the Cloud service.
  *
  * Here can be implemented an integration to log or save these results.
  *
  * Input:
  *   - result: json message that contains the result obtained from cloud service
  */
 console.log(`onResponse result = ${JSON.stringify(result)}`);
}

function getTemporaryToken(urlGetTemporaryToken, securityToken) {
 /*
  * This method is in charge of obtaining a temporary token to be used
  * in calls to the cloud service.
  */
 //return '1234';
 const requestOptionsTwilioFunction = {
   method: 'POST',
   headers: {
     'Content-Type': 'application/x-www-form-urlencoded;charset=UTF-8',
   },
   body: new URLSearchParams({
     Token: securityToken,
   }),
 };

 const request = async () => {
   const response = await fetch(
     urlGetTemporaryToken,
     requestOptionsTwilioFunction
   );
   const json = await response.json();
   return json
 }

 return request();
}

export const externalMethods = {
 storeCredential,
 deleteCredential,
 getCredential,
 onResponse,
 getTemporaryToken
}

Then, insert this component in your Twilio Flex Project inside the init function. You can use the following snippet:

flex.CallCanvas.Content.add(
      <VeridasComponent key="veridas-component" token={manager.store.getState().flex.session.ssoTokenPayload.token}/>, { sortOrder: 1 });

Create a Veridas Service with streaming functions

We will use some Twilio Functions to link Flex with the Veridas Voice Biometric Authentication solution. You can develop the functions using the Twilio CLI Serverless Toolkit or, alternatively, you can also code the service directly on the console as follows:

  1. Go to your Twilio Console
  2. Click on All products and services on the left panel (the “three dots” button)
  3. Go to the Runtime section and click on Functions
  4. Click on Services > Create service
  5. Set the name field to VeridasService and click on Next
  6. Click on Add > Add Function and set /startStreamAudio as name
  7. Click on Add > Add Function and set /stopStreamAudio as name
  8. Click on Add > Add Function and set /getTemporaryToken as name
  9. Insert code snippets from the below sections
  10. Make sure your functions are set as public (from protected by default). These functions will be protected from unsolicited requests by validating tokens generated by Twilio. Only the Twilo environment will be able to execute them.
  11. Click on Settings > Dependencies on the bottom left of the page
  12. Add the twilio-flex-token-validator module (version 1.5.5)
  13. Update the twilio module version to 3.56.0
  14. Add the form-data module (version 4.0.0)
  15. Add the axios module (version 0.23.0)
  16. Add the https module (version 1.0.0)
  17. Save and deploy the new functions

An already existing Service can be used instead of a new one.

Start Stream function

Open the startStreamAudio function and paste the following block of code. Then, save the code.

const TokenValidator = require('twilio-flex-token-validator').functionValidator;

async function requestStartStream(context, event, callback, response) {
    const client = context.getTwilioClient();
    const call = client.calls(event.callSid);
    const conference = await client.conferences(event.conferenceSid);
    const participant = await conference.participants(event.callSid);

    participant.update({endConferenceOnExit: false, beepOnExit: false})
    .then(participant => {
        call.update({twiml: '<Response><Start><Stream name="'+event.callSid+'" url="'+event.url+'" track="inbound_track"/></Start><Dial><Conference endConferenceOnExit="true" beep="false">'+event.taskSid+'</Conference></Dial></Response>'})
        .then(call => {
            callback(null, response);
        }).catch(e => {
            console.log(e);
            response.setBody("Exception");
            callback(response, err);
        });
    });
}

exports.handler = TokenValidator(function(context, event, callback) {
    const response = new Twilio.Response();
    response.appendHeader('Access-Control-Allow-Origin', '*');
    response.appendHeader('Access-Control-Allow-Methods', 'OPTIONS, POST, GET');
    response.appendHeader('Access-Control-Allow-Headers', 'Content-Type');

    requestStartStream(context, event, callback, response);
});

Stop Stream function

Open the stopStreamAudio function and paste the following block of code. Then, save the code.

const TokenValidator = require('twilio-flex-token-validator').functionValidator;

async function requestStopStream(context, event, callback, response) {
    const client = context.getTwilioClient();
    const call = client.calls(event.callSid);
    const conference = await client.conferences(event.conferenceSid);
    const participant = await conference.participants(event.callSid);
    
    participant.update({endConferenceOnExit: false, beepOnExit: false})
    .then(participant => {
        call.update({twiml: '<Response><Stop><Stream name="'+event.callSid+'" /></Stop><Dial><Conference endConferenceOnExit="true" beep="false">'+event.taskSid+'</Conference></Dial></Response>'})
        .then(call => {
            callback(null, response);
        }).catch(e => {
            console.log(e);
            response.setBody("Exception");
            callback(response, null);
        });
    });
}

exports.handler = TokenValidator(function(context, event, callback) {
    const response = new Twilio.Response();
    response.appendHeader('Access-Control-Allow-Origin', '*');
    response.appendHeader('Access-Control-Allow-Methods', 'OPTIONS, POST, GET');
    response.appendHeader('Access-Control-Allow-Headers', 'Content-Type');

    requestStopStream(context, event, callback, response);
});

Get Temporary Token function

Open the getTemporaryToken function and paste the following block of code. Then, save the code.

const TokenValidator = require('twilio-flex-token-validator').functionValidator;

function generateCloudRequest(context) {
  const FormData = require('form-data');
  const https = require('https');

  const formdata = new FormData();
  formdata.append("grant_type", "client_credentials");
  formdata.append("client_id", context.client_id);
  formdata.append("client_secret", context.client_secret);

  const requestConfig = {
    method: 'POST',
    url: context.getTemporaryTokenCloudUrl,
    data: formdata,
    headers: { 
    ...formdata.getHeaders()
    },
    httpsAgent: new https.Agent({
      rejectUnauthorized: false
    })
  };
  
  return requestConfig
}

exports.handler = TokenValidator(function(context, event, callback) {
  const axios = require("axios");
  const response = new Twilio.Response();
  response.appendHeader('Access-Control-Allow-Origin', '*');
  response.appendHeader('Access-Control-Allow-Methods', 'OPTIONS, POST, GET');
  response.appendHeader('Access-Control-Allow-Headers', 'Content-Type');
  response.appendHeader('Content-Type', 'application/json')

  const requestConfig = generateCloudRequest(context);
  axios(requestConfig)
  .then(function (result) {
    response.setBody(`${result.data.token_type} ${result.data.access_token}`);
    return callback(null, response);
  })
  .catch(function (error) {
    console.error(error);
    return callback(null, error);
  });
});

Remember to redeploy the functions so the changes take effect.

Final setup for the Veridas Plugin

Veridas Temporary Token URL

Once Veridas provides you with valid credentials and configuration URLs, enter the Environment Variables in the Twilio Functions Service Settings section, or .env if using Twilio's CLI for deployment. Insert the following parameter and add the provided info:

  • getTemporaryTokenCloudUrl: URL to the Veridas endpoint to obtain a temporary token

Veridas credentials

In the same Environment Variables section, set the following credentials provided by Veridas:

  • client_secret
  • client_id

Cloud URLs

Once Veridas provides you with valid credentials and audio consumer service URLs, set them in the Config.js file.

  • urlSio: API URL for the service
  • urlAudioConsumer: Audio consumer service url

Twilio Streaming functions URLs

Set these urls in the Config.js file.

  • urlGetTemporaryToken: URL to your getTemporaryToken Twilio Function
  • urlStartStreaming: URL to your startStream Twilio Function
  • urlStopStreaming: URL to your stopStream Twilio Function

Request-parameters customization

The parameters used for the requests to the Veridas service can be customized in the Config.js file.

  • scoreThreshold: Number ranging from 0 to 1 representing the similarity between the incoming voice and the voiceprint. 0 stands for completely different vocal patterns, while 1 would mean completely identical. (The margin of error of 0.01 is due to rounding results.)
  • authenticityThreshold: Number ranging from 0 to 1 representing how likely the sample (incoming audio) is to be a bonafide voice. The closer the number is to 1, the higher the chance of being a genuine voice. The margin of error of 0.01 due to rounding results.
  • audioTotalDurationRegistration: Minimum number of seconds of speech required to process the audio on a registration.
  • audioTotalDurationVerification: Minimum number of seconds of speech required to process the audio during a verification.

The code presented here has a realistic starting point but you might want to tune it to your own use case.

Finally, do not forget to redeploy the functions (in the console interface) and the plugin with any change.

`twilio flex:plugins:deploy --changelog 'add veridas plugin'

Database integration and external methods

In the Utils.js file, there are functions that are designed for an integration with a client's database. The code provided explains how each function should work. These functions are called within the plugin’s internal logic:

  • storeCredential: Called whenever a credential needs to be saved in the database. For example: when a registration completes successfully and a credential is generated from the subject's voice.
  • deleteCredential: Called whenever a credential needs to be removed from the database. For example: when an Agent presses the Delete Registration button.
  • getCredential: Called to obtain the credential from a subject. For example: when a new call is started and the plugin needs to know if that subject has already been registered.
  • onResponse: Called whenever a result is retrieved from the Cloud service. This allows analyzing and storing results and metrics.

The code attached for the Utils.js contains a simple cookie database system to help start testing the solution and appreciate the possibilities of the integration. However, a production application should be using your own highly secured database.

Automatic verification

To activate the live verification in all calls, we can change the Config.js to tell the system that we are moving from manual to automatic:

  • automaticVerification: boolean value to represent whether the logic is going to automatically start verifications of previously registered users.

If set to true, when a new call is started, if the user was previously registered, a verification starts right away.

Delete only after verification

In the Config.js file you’ll find the following parameter:

  • deleteOnlyAfterVerification: boolean value to represent whether the logic is going to force a successful verification before enabling the deletion button.

If set to true, the delete button will not be enabled until a successful verification is completed.

Test the Veridas Flex plugin

Deploy and activate the plugin on your Twilio Flex instance:

$ twilio flex:plugins:deploy --changelog 'add veridas'
$ twilio flex:plugins:release --plugin caliope@0.0.1 --name "Autogenerated Release 1636552720686" --description "Veridas plugin version 1"

Make sure that the plugin is active in your Flex configuration console.

How to onboard and test the biometric credentials of a caller

Once the Veridas plugin is activated, you will see the Veridas component as soon as you pick up the call. At this stage, you will want to register a given caller.

Veridas Voice Biometrics Plugin in action

Request your caller to speak for a while, ask them questions, etc. and press register.

Veridas Voice Biometric Plugin registering

The plugin will gather the audio from the stream and perform the registration. When the registration finishes correctly, the plugin will test the authentication of the caller's voice. Remember that the default plugin stores the biometric credentials of the caller along with the caller's phone number in a cookie for demo purposes.

 

Veridas Voice Biometrics Plgun after registration

Now you can test the biometric authentication. Click on "Authenticate user" and let the caller speak. The plugin will gather the voice and biometric credentials and let Veridas check whether they are correct.

Veridas Voice Biometrics Plugin Authenticating

If Veridas authenticates the caller, the plugin will show a green tick and a message informing the agent.

Veridas Voice Biometrics Plugin successful athentication

You can also test when there is no correct match. Let another person call with that number, and you will see that it does not pass the test. Veridas even detects when a caller is a recorded message.

Veridas Voice Biometrics unsuccessful authentication

Watch the whole process in action

 

 

API mocking: cross-product integration testing

Calls to the API can be mocked to ease your development and testing. This mocking is set up within the Config.js file via the following parameters:

  • mockApi: boolean value to represent whether the API needs to be mocked or not.
  • mockResponseType: type of response that the internal logic is going to retrieve while mocking the API. Below are the possibilities for this value.

Types of responses available:

  • resultSuccess: applicable either for registrations and verifications. Used whenever a process is successful.
  • resultFailSpoof: applicable either for registrations and verifications. Used whenever a process results in a spoofing scenario.
  • resultFailScore: applicable only for verifications. Used whenever a process results in a low score scenario (not a match between the subject's voice and credential sent in the request).
  • errorConsumerNotReady: applicable either for registrations and verifications. Used whenever the audio consumer required is not up and running in Veridas servers.
  • errorParameters: applicable either for registrations and verifications. Used whenever a request contains wrong parameters or is missing some.
  • errorAudio: applicable either for registrations and verifications. Used whenever a problem with the audio occurs.
  • errorDasPeak: applicable either for registrations and verifications. Used whenever an internal problem in the biometrics process occurs.

Conclusion

Voice biometrics brings customer engagement for identity and authentication to the next level in a contact center scenario. With Flex and the Veridas Plugin you could be running world class voice authentication in your contact centre in a matter of hours.
You can check the technical details of the Veridas Flex Plugin in the Veridas doc site. 

Biography

Ana Andrés is a Telecommunications Engineer and Computer Sciencie PhD. She is part of the Solutions Engineering team in Twilio. She acts as a trusted advisor for customers in Europe in everything related to Twilio technology and products. You can reach to Ana at aandresdelvalle [at] twilio.com

We would like to thank the Veridas team for their contribution to this blog.

Related Posts

Related Resources

A newspaper article

Twilio Docs

From APIs to SDKs to sample apps

API reference documentation, SDKs, helper libraries, quickstarts, and tutorials for your language and platform.
An Open book

Resource Center

The latest ebooks, industry reports, and webinars

Learn from customer engagement experts to improve your own communication.
User group reactions

Ahoy

Twilio's developer community hub

Best practices, code samples, and inspiration to build communications and digital engagement experiences.