By
When it comes down to it, audio quality is paramount for video experiences. In fact, 81% of virtual attendees regard flawless audio as a key requirement to a successful meeting (1). But in a world where virtual meetings are taken from anywhere, capturing clean audio is hard. After all, background noise happens!
To solve this problem, Twilio has partnered with Krisp Technologies Inc to launch AI-based Noise Cancellation! This feature is available as a plugin to all Twilio Video Group Rooms customers.
With Noise Cancellation, provide best-in-class audio experiences directly in your video application. Watch this short demo to experience Noise Cancellation in action!
Twilio Video Noise Cancellation Demo
Why Audio Quality Matters
Studies show that audio quality not only impacts a user’s video experience but also improves meeting effectiveness. Here are a few examples:
- Audio Quality Directly Impacts Credibility: Research suggests when our brains have a hard time understanding information, we are more inclined to dismiss the information.
- Background Noise Reduces the Ability to Concentrate: Background or low level noise often disrupts people’s ability to concentrate and can cause stress (ever heard of Zoom fatigue?).
- Audio Quality is more important than Video Quality: A 2022 research study showed that audio quality influences the perceived quality of the entire experience. Thus, bad audio lowers the perception of your overall experience, and this is the reason you do things like watch a grainy video with clear audio but stop watching an HD video with bad audio.
The bad news is that audio can happen from anyone, anywhere. It really only takes one person with bad background noise to ruin an entire call, and it’s for this reason we decided to build the Krisp audio plugin for Twilio Video.
Best-in-Class Noise Cancellation in Twilio Video
As the leader in noise suppression technology, Krisp.ai cleans over 75 billion minutes of noise per month. The Krisp.ai technology is built on a leading edge Deep Neural Network that can differentiate between background sounds and the human voice.
It’s with this technology that Krisp created the Krisp Audio Plugin for Twilio Video, a lightweight audio processor that can run inside your client application and create crystal clear audio. Using this plugin, any Twilio Video developer can eliminate background noise and power high quality audio experiences with Noise Cancellation!
Leveraging Twilio, Indeed has transformed the hiring process by building an all-in-one virtual interview platform. With over 200,000 virtual interviews every year, Indeed requires an AI-powered noise cancellation solution to increase accessibility and improve the virtual experience for both jobseekers and interviewers.
“Noise Cancellation is an essential feature for our Virtual Interview platform because it allows us to provide a more equitable interview experience for candidates and interviewers regardless of their environment or device.”
- Connie Cheng, Product Manager @ Indeed
Benefits of Noise Cancellation with Twilio Video:
- Industry Leading Audio Quality: In a SigmaConnectivity study comparing Zoom, Webex, Microsoft Teams, Google Meet, Krisp and many others, Krisp has the highest average G-MOS score on a standard evaluation dataset.
- Easy Implementation: The Krisp Plugin for Twilio makes it easy to add noise cancellation to a Twilio Video Group Room with just a few lines of code. The audio plugin is supported across multiple browsers, and the Twilio SDK will automatically check for browser support and gracefully return a regular audio track if the plugin is not supported. This allows you to have a simple implementation for the best possible audio experience across all browsers, all devices.
- Flexible Programming: The audio processor APIs in the SDK allow you to easily disable / enable the Krisp audio plugin during a call. This API gives you and your users full control over their sound.
“We are thrilled to partner with Twilio Inc, a global leader in the communications industry, and bring our Noise Cancellation technology to millions of users of the Twilio Video platform. Krisp is a natural technology fit within Twilio's solutions, complementing advanced voice and audio quality with market-leading Twilio Video for desktop and mobile devices. Customers will certainly notice the difference.”
- Robert Schoenfield, EVP at Krisp
Add Noise Cancellation to Twilio Video
Starting in JS SDK 2.24.0 (and coming soon to iOS and Android), there is a new audio processor API that works with published audio tracks. With this new API, the raw audio will route via the Krisp audio plugin for Twilio Video that powers AI-based Noise Cancellation.
The plugin is loaded alongside the Twilio SDK and runs as part of the audio pipeline between the microphone and audio encoder in a step called pre-processing. During this step, the AI based noise cancellation algorithm does its magic, removing unwanted sounds like barking dogs, construction noises, honking horns, and coffee shop chatter.
After the preprocessing step, the audio gets encoded and delivered to the end user. It’s very important to note that everything happens on your device, with almost no latency, and no media ever sent to a server.
You can add Noise Cancellation by following these 2 Steps:
Step 1: Install the Plugin
The plugin is available on npm as @twilio/krisp-audio-plugin. You can install it with npm like this:
npm install @twilio/krisp-audio-plugin
Once installed, you need to host the contents of ./node_modules/@twilio/krisp-audio-plugin/dist/ from your web server. We recommend that you add the plugin version number to the hosted path to ensure that the browser does not use a stale version when it's updated.
Step 2: Configure your Application
In your application, you will need to specify that you want to use krisp in noiseCancellationOptionswhen you create the local audio track. You also need to specify the path where your application server is hosting Krisp files from step 1 above.
The example below assumes that you have hosted the files at /noise_cancellation_plugin/1.0.0/dist on your web server.
const { connect, createLocalAudioTrack } = require('twilio-video');
// Create a LocalAudioTrack with Krisp noise cancellation enabled.
const localAudioTrack = await createLocalAudioTrack({
noiseCancellationOptions: {
sdkAssetsPath: 'path/to/hosted/krisp/audio/plugin/dist',
vendor: 'krisp'
}
});
if (!localAudioTrack.noiseCancellation) {
// If the Krisp audio plugin fails to load, then a warning message will be logged
// in the browser console, and the "noiseCancellation" property will be set to null.
// You can still use the LocalAudioTrack to join a Room. However, it will use the
// browser's noise suppression instead of the Krisp noise cancellation. Make sure
// the "sdkAssetsPath" provided in "noiseCancellationOptions" points to the correct
// hosted path of the plugin assets.
} else {
// Join a Room with the LocalAudioTrack.
const room = await connect('token', {
name: 'my-cool-room',
tracks: [localAudioTrack]
});
if (!localAudioTrack.noiseCancellation.isEnabled) {
// Krisp noise cancellation is permanently disabled in Peer-to-Peer and Go Rooms.
}
}
/**
* Enable/disable noise cancellation.
* @param {boolean} enable - whether noise cancellation should be enabled
*/
function setNoiseCancellation(enable) {
const { noiseCancellation } = localAudioTrack;
if (noiseCancellation) {
if (enable) {
// If enabled, then the LocalAudioTrack will use the Krisp noise
// cancellation instead of the browser's noise suppression.
noiseCancellation.enable();
} else {
// If disabled, then the LocalAudioTrack will use the browser's
// noise suppression instead of the Krisp noise cancellation.
noiseCancellation.disable();
}
}
}
Try it out Today
Noise Cancellation for Twilio Video was introduced in Twilio Video JavaScript SDK 2.24.0 and it will be coming soon to the Twilio Video iOS and Android SDKs. This feature is available to all Twilio Video users of Group Rooms. For additional information, see our developer documentation. We can’t wait to see what you build!
