By
In this guide we’ll cover how to combine Twilio Add-ons with Programmable Voice to use data from Twilio's partners in your C# (ASP.NET MVC) web application. Let's get started!
What’s an Add-On?
Twilio Add-ons are pre-integrated third party services that you can use with your Twilio application. Add-ons enhance the responses your application receives from Twilio by including additional data from Twilio’s partners.
In this guide we’ll cover how to combine Twilio Add-ons with Programmable Voice to use data from Twilio's partners in your C# web application. The sample code in this guide will use the Twilio C# SDK and C# (ASP.NET MVC). Let's get started.
Sample data from the IBM Watson Speech to Text Add-on This is a sample of the payload your application would receive when using the IBM Watson Speech to Text add-on.
{
"status":"successful",
"message":null,
"code":null,
"results":{
"ibm_watson_speechtotext":{
"request_sid":"XRc82d4197bbc51d5982588ae367365a51",
"status":"successful",
"message":null,
"code":null,
"payload":[
{
"content_type":"application/json",
"url":"https://api.twilio.com/2010-04-01/Accounts/AC597d02bf69de79f8f411f289ac211a0d/Recordings/REab85c1c2860471abf040b545c4e5aa40/AddOnResults/XRc82d4197bbc51d5982588ae367365a51/Payloads/XH364bd4514b1377fd8844cd9793416131/Data"
}
],
"links":{
"add_on_result":"https://api.twilio.com/2010-04-01/Accounts/AC597d02bf69de79f8f411f289ac211a0d/Recordings/REab85c1c2860471abf040b545c4e5aa40/AddOnResults/XRc82d4197bbc51d5982588ae367365a51",
"payloads":"https://api.twilio.com/2010-04-01/Accounts/AC597d02bf69de79f8f411f289ac211a0d/Recordings/REab85c1c2860471abf040b545c4e5aa40/AddOnResults/XRc82d4197bbc51d5982588ae367365a51/Payloads",
"recording":"https://api.twilio.com/2010-04-01/Accounts/AC597d02bf69de79f8f411f289ac211a0d/Recordings/REab85c1c2860471abf040b545c4e5aa40"
}
}
}
}
Here’s an example that uses the IBM Watson Speech to Text Add-on. The generated transcription will be pushed to the application through a callback from Twilio.
The IBM Watson Speech add-on can convert Twilio audio recordings into text using machine intelligence to combine information about grammar and language structure.
Sample recording analysis from the IBM Watson Speech to Text Add-on This is the sample analysis that the Add-on partner provides which is included in the URL of the payload above.
{
"user_token": "XRf951c9b0bce23930750991099e553b0a",
"results":[
{
"results":[
{
"keywords_result":{},
"alternatives":[
{
"timestamps":[
["I", 0.04, 0.28],
["love", 0.28, 0.71],
["coding", 0.74, 1.25]
],
"confidence":0.782,
"transcript":"I love coding "
}
],
"final":true
}
],
"result_index":0
}
],
"id":"b0e7a660-bcb6-11e6-8b1b-192a5f5dc12f",
"event":"recognitions.completed_with_results"
}
Enabling Add-ons in the Twilio Console
The first stop in using an add-on is your Twilio Console. Log in and navigate to the Add-ons page within the Marketplace section.
Then choose on the add-on you want to enable and click the “Install” button. After accepting the Terms of Service, it will be installed on your account. In this example, we’ll use the IBM Watson Speech to Text Add-on.
Once installed, select the checkbox for the recording method you are using (see the different recording methods here). In our example to follow, we would only need the "Record Verb Recordings".
Set the "Callback URL" and "Callback Request Method" as appropriate for your public server URL (instructions below).
Pro Tip: you can quickly inspect what Twilio will send to your callback URL using tools like PostBin before you actually set up your server.
Legal implications of call recording
If you choose to record voice or video calls, you need to comply with certain laws and regulations, including those regarding obtaining consent to record (such as California’s Invasion of Privacy Act and similar laws in other jurisdictions). Additional information on the legal implications of call recording can be found in the "Legal Considerations with Recording Voice and Video Communications" Help Center article.
Notice: Twilio recommends that you consult with your legal counsel to make sure that you are complying with all applicable laws in connection with communications you record or store using Twilio.
Now we’re ready to use the extra data from our add-on in our application.
Using Add-on data in your application
Twilio will include data from all of your add-ons in an “AddOns” field on HTTP requests it makes to your application. The following example demonstrates how to access add-on data within your C# (ASP.NET MVC) application.
Use IBM Watson Speech to Text Add-on
using System;
using System.Web.Mvc;
using Twilio.AspNet.Mvc;
using Twilio.TwiML;
public class VoiceController : TwilioController
{
[HttpPost]
public ActionResult Index()
{
var response = new VoiceResponse();
response.Say("Hi! I want to know what do you think about coding.");
response.Record(maxLength: 10, action: new Uri("/recording", UriKind.Relative));
response.Hangup();
return TwiML(response);
}
}
The VoiceController will receive the call, record the next 10 seconds of the call and redirect the code execution to the RecordingController and hangup.
using System;
using System.Web.Mvc;
using Twilio.AspNet.Mvc;
using Twilio.TwiML;
public class RecordingController : TwilioController
{
public class BodyData
{
public string RecordingUrl { get; set; }
}
[HttpPost]
public ActionResult Index(BodyData bodyData)
{
string recordingUrl = bodyData.RecordingUrl;
Console.WriteLine(recordingUrl);
var response = new VoiceResponse();
response.Say("Thanks for howling... take a listen to what you howled.");
response.Play(new Uri(recordingUrl));
response.Say("Goodbye.");
return TwiML(response);
}
}
The RecodingController will receive a link to recorded data and execute it.
using System;
using System.Text;
using System.Collections.Generic;
using System.Configuration;
using System.Web.Mvc;
using Twilio.AspNet.Mvc;
using Newtonsoft.Json;
using System.Net;
using System.IO;
using recording_addons_watson_csharp.HttpResponseClasses;
public class CallbackController : TwilioController
{
[HttpPost]
public ActionResult Index(HttpResponseClasses.BodyData bodyData)
{
var data = bodyData.AddOns;
var addOns = JsonConvert.DeserializeObject<HttpResponseClasses.RootObject>(data);
if (addOns.results.ibm_watson_speechtotext.ToString() == "")
{
return Content("Add Watson Speech to Text add-on in your Twilio console");
}
var payloadUrl = addOns.results.ibm_watson_speechtotext.payload[0].url;
var accountSid = ConfigurationManager.AppSettings["TwilioAccountSid"];
var apiKey = ConfigurationManager.AppSettings["TwilioAuthToken"];
var reqCredentials = Convert.ToBase64String(ASCIIEncoding.ASCII.GetBytes(accountSid + ":" + apiKey));
var request = (HttpWebRequest)WebRequest.Create(payloadUrl);
request.Headers.Add("Authorization", "Basic " + reqCredentials);
var response = (HttpWebResponse)request.GetResponse();
Stream stream = response.GetResponseStream();
StreamReader streamreader = new StreamReader(stream);
var responseBody = streamreader.ReadToEnd();
var results = JsonConvert.DeserializeObject<WatsonResponse.RootObject>(responseBody).results[0].results;
var transcripts = string.Join("",
results.ConvertAll<string>(item => item.alternatives[0].transcript).ToArray()
);
return Content(transcripts);
}
}
Lastly, the CallbackController will kick, receive the recorded data transcription and return on the HTTP response.
This application gets the transcription for the recorded call from IBM Watson Speech to Text Add-on we configured in the Twilio console. You can find a detailed specification of the data structure for an individual add-on within that add-on's page in the catalog.
And that’s it! Try giving your application a call to see your add-on in action. If you've never received a Twilio phone call with Ruby before, check out our guide on that topic.
Where to next?
We focused on the IBM Watson Speech to Text Add-on in this guide, but you can just as easily use any of the add-ons available in the catalog in your Twilio Console.
Check out the full Add-Ons API Reference documentation if you want to learn more about how add-ons work, or even how to publish your own.
