Skip to contentSkip to navigationSkip to topbar
Rate this page:
On this page

Building a JS Video App: Recommendations and Best Practices


(warning)

Warning

This page is for reference only. We are no longer onboarding new customers to Programmable Video. Existing customers can continue to use the product until December 5, 2024(link takes you to an external page).
We recommend migrating your application to the API provided by our preferred video partner, Zoom. We've prepared this migration guide(link takes you to an external page) to assist you in minimizing any service disruption.


Overview

overview page anchor

This guide provides recommendations and best practices for building a Video Application using twilio-video.js.



This table shows the browsers and platforms that are supported by twilio-video.js. Please use the isSupported(link takes you to an external page) flag to find out if twilio-video.js supports the browser in which your application is running.


_10
const { isSupported } = require('twilio-video');
_10
if (isSupported) {
_10
// Set up your video app.
_10
} else {
_10
console.error('This browser is not supported by twilio-video.js.');
_10
}


Please take a look at this guide to choose the right ConnectOptions(link takes you to an external page) values for your use case.


Application Domain

application-domain page anchor

twilio-video.js relies on getUserMedia(link takes you to an external page) to acquire local media. In order for this API to be available, please ensure that your application is running either on localhost or an https domain.

The autoplay policy does not allow you to autoplay audio using unmuted <audio> or <video> elements unless the user has interacted with your application (clicking on a button, for example), especially if your application's media engagement score is not high enough. Please refer to "Working around the browsers' autoplay policy" in the JavaScript SDK's COMMON_ISSUES.md(link takes you to an external page) to work around different browsers' autoplay policies.

Acquiring Camera in Mobile Browsers

acquiring-camera-in-mobile-browsers page anchor

In mobile browsers, the camera can be reserved by only one LocalVideoTrack at any given time. If you attempt to create a second LocalVideoTrack, video frames will no longer be supplied to the first LocalVideoTrack. So, we recommend that:

If you want to display your camera preview, pre-acquire media using createLocalTracks(link takes you to an external page). You can then pass these LocalTracks to connect(link takes you to an external page).


_13
const { createLocalTracks, connect } = require('twilio-video');
_13
_13
const tracks = await createLocalTracks();
_13
_13
// Display camera preview.
_13
const localVideoTrack = tracks.find(track => track.kind === 'video');
_13
divContainer.appendChild(localVideoTrack.attach());
_13
_13
// Join the Room with the pre-acquired LocalTracks.
_13
const room = await connect('token', {
_13
name: 'my-cool-room',
_13
tracks
_13
});

If you want to switch between the front and back facing cameras, starting from SDK version 2.7.0(link takes you to an external page), you can restart(link takes you to an external page) the existing LocalVideoTrack.


_17
const { createLocalTracks, connect } = require('twilio-video');
_17
_17
const tracks = await createLocalTracks({
_17
audio: true,
_17
video: { facingMode: 'user' }
_17
});
_17
_17
// Join the Room with the pre-acquired LocalTracks.
_17
const room = await connect('token', {
_17
name: 'my-cool-room',
_17
tracks
_17
});
_17
_17
const cameraTrack = tracks.find(track => track.kind === 'video');
_17
_17
// Switch to the back facing camera.
_17
cameraTrack.restart({ facingMode: 'environment' });

In SDK versions 2.6.0 and below, you can stop and unpublish the existing LocalVideoTrack, use createLocalVideoTrack(link takes you to an external page) to create a new LocalVideoTrack and publish it to the Room.


_21
const { createLocalTracks, createLocalVideoTrack, connect } = require('twilio-video');
_21
_21
const tracks = await createLocalTracks({
_21
audio: true,
_21
video: { facingMode: 'user' }
_21
});
_21
_21
// Join the Room with the pre-acquired LocalTracks.
_21
const room = await connect('token', {
_21
name: 'my-cool-room',
_21
tracks
_21
});
_21
_21
// Capture the back facing camera.
_21
const backFacingTrack = await createLocalVideoTrack({ facingMode: 'environment' });
_21
_21
// Switch to the back facing camera.
_21
const frontFacingTrack = tracks.find(track => track.kind === 'video');
_21
frontFacingTrack.stop();
_21
room.localParticipant.unpublishTrack(frontFacingTrack);
_21
room.localParticipant.publishTrack(backFacingTrack);

Testing the Microphone and Camera

testing-the-microphone-and-camera page anchor

In mobile browsers, getUserMedia(link takes you to an external page) is successful even when your microphone and/or camera are reserved by another tab or application. This can result in mobile Participants not being seen and/or heard by others in a Room. In order to work around this, we recommend that your application prompt users to test their microphone and camera before joining a Room. You can use createLocalAudioTrack(link takes you to an external page) to acquire the microphone, and use the Web Audio API(link takes you to an external page) to calculate its level. If the level is 0 even when the user is talking, then most likely the microphone is reserved by either another tab or application. You can then recommend that the user close all the other applications and reload your application, or worst case, restart the browser.

testmic.js


_10
const { createLocalAudioTrack } = require('twilio-video');
_10
const pollAudioLevel = require('./pollaudiolevel');
_10
_10
const audioTrack = await createLocalAudioTrack();
_10
_10
// Display the audio level.
_10
pollAudioLevel(audioTrack, level => {
_10
/* Update audio level indicator. */
_10
});

pollaudiolevel.js


_55
const AudioContext = window.AudioContext || window.webkitAudioContext;
_55
const audioContext = AudioContext ? new AudioContext() : null;
_55
_55
function rootMeanSquare(samples) {
_55
const sumSq = samples.reduce((sumSq, sample) => sumSq + sample * sample, 0);
_55
return Math.sqrt(sumSq / samples.length);
_55
}
_55
_55
async function pollAudioLevel(track, onLevelChanged) {
_55
if (!audioContext) {
_55
return;
_55
}
_55
_55
// Due to browsers' autoplay policy, the AudioContext is only active after
_55
// the user has interacted with your app, after which the Promise returned
_55
// here is resolved.
_55
await audioContext.resume();
_55
_55
// Create an analyser to access the raw audio samples from the microphone.
_55
const analyser = audioContext.createAnalyser();
_55
analyser.fftSize = 1024;
_55
analyser.smoothingTimeConstant = 0.5;
_55
_55
// Connect the LocalAudioTrack's media source to the analyser.
_55
const stream = new MediaStream([track.mediaStreamTrack]);
_55
const source = audioContext.createMediaStreamSource(stream);
_55
source.connect(analyser);
_55
_55
const samples = new Uint8Array(analyser.frequencyBinCount);
_55
let level = null;
_55
_55
// Periodically calculate the audio level from the captured samples,
_55
// and if changed, call the callback with the new audio level.
_55
requestAnimationFrame(function checkLevel() {
_55
analyser.getByteFrequencyData(samples);
_55
const rms = rootMeanSquare(samples);
_55
const log2Rms = rms && Math.log2(rms);
_55
_55
// Audio level ranges from 0 (silence) to 10 (loudest).
_55
const newLevel = Math.ceil(10 * log2Rms / 8);
_55
if (level !== newLevel) {
_55
level = newLevel;
_55
onLevelChanged(level);
_55
}
_55
_55
// Continue calculating the level only if the audio track is live.
_55
if (track.mediaStreamTrack.readyState === 'live') {
_55
requestAnimationFrame(checkLevel);
_55
} else {
_55
requestAnimationFrame(() => onLevelChanged(0));
_55
}
_55
});
_55
}
_55
_55
module.exports = pollAudioLevel;

You can use createLocalVideoTrack(link takes you to an external page) to acquire the camera, and attach its corresponding <video> element to the DOM. If there are no video frames, then most likely the camera is reserved by either another tab or application. Your can then recommend that the user close all the other applications and reload your application, or worst case, restart the browser.

testcamera.js


_10
const { createLocalVideoTrack } = require('twilio-video');
_10
_10
const videoTrack = await createLocalVideoTrack();
_10
_10
// Display the video preview.
_10
const divContainer = document.getElementById('local-video');
_10
const videoElement = videoTrack.attach();
_10
divContainer.appendChild(videoElement);

NOTE: In iOS Safari, because of this WebKit bug(link takes you to an external page), calling getUserMedia(link takes you to an external page) again will mute previously acquired LocalTracks. So, please make sure that the LocalTracks that you pass in ConnectOptions are neither muted nor stopped.


Application Backgrounding in Mobile Browsers

application-backgrounding-in-mobile-browsers page anchor

When an application that is running on a mobile browser is backgrounded, it will not have access to the video feed from the camera until it is foregrounded. So, we recommend that you stop and unpublish the camera's LocalVideoTrack, and publish a new LocalVideoTrack once your application is foregrounded. On the remote side, you can listen to the unsubscribed(link takes you to an external page) and subscribed(link takes you to an external page) events on the corresponding RemoteVideoTrackPublication in order to notify the user accordingly. You can use the Page Visibility API(link takes you to an external page) to detect backgrounding and foregrounding.

mobileuser.js


_24
const { connect, createLocalTracks, createLocalVideoTrack } = require('twilio-video');
_24
_24
const tracks = await createLocalTracks();
_24
_24
let videoTrack = tracks.find(track => track.kind === 'video');
_24
_24
const room = await connect('token1', {
_24
name: 'my-cool-room',
_24
tracks
_24
});
_24
_24
if (/* isMobile */) {
_24
document.addEventListener('visibilitychange', async () => {
_24
if (document.visibilityState === 'hidden') {
_24
// The app has been backgrounded. So, stop and unpublish your LocalVideoTrack.
_24
videoTrack.stop();
_24
room.localParticipant.unpublishTrack(videoTrack);
_24
} else {
_24
// The app has been foregrounded, So, create and publish a new LocalVideoTrack.
_24
videoTrack = await createLocalVideoTrack();
_24
await room.localParticipant.publishTrack(videoTrack);
_24
}
_24
});
_24
}

remoteuser.js


_35
const { connect } = require('twilio-video');
_35
_35
function setupRemoteVideoNotifications(publication) {
_35
if (publication.isSubscribed) {
_35
// Indicate to the user that the mobile user has added video.
_35
}
_35
_35
publication.on('subscribed', track => {
_35
// Indicate to the user that the mobile user has added video.
_35
});
_35
_35
publication.on('unsubscribed', track => {
_35
// Indicate to the user that the mobile user has removed video.
_35
});
_35
}
_35
_35
function setupRemoteVideoNotificationsForParticipant(participant) {
_35
// Set up remote video notifications for the VideoTracks that are
_35
// already published.
_35
participant.videoTracks.forEach(setupRemoteVideoNotifications);
_35
_35
// Set up remote video notifications for the VideoTracks that will be
_35
// published later.
_35
participant.on('trackPublished', setupRemoteVideoNotifications);
_35
}
_35
_35
const room = await connect('token2', { name: 'my-cool-room' });
_35
_35
// Set up remote video notifications for the VideoTracks of RemoteParticipants
_35
// already in the Room.
_35
room.participants.forEach(setupRemoteVideoNotificationsForParticipant);
_35
_35
// Set up remote video notifications for the VideoTracks of RemoteParticipants
_35
// that will join the Room later.
_35
room.on('participantConnected', setupRemoteVideoNotificationsForParticipant);


When the user closes the tab/browser or navigates to another web page, we recommend that you disconnect from the Room so that other Participants are immediately notified.


_16
const { createLocalTracks, connect } = require('twilio-video');
_16
_16
const tracks = await createLocalTracks();
_16
_16
const room = await connect('token', {
_16
name: 'my-cool-room',
_16
tracks
_16
});
_16
_16
// Listen to the "beforeunload" event on window to leave the Room
_16
// when the tab/browser is being closed.
_16
window.addEventListener('beforeunload', () => room.disconnect());
_16
_16
// iOS Safari does not emit the "beforeunload" event on window.
_16
// Use "pagehide" instead.
_16
window.addEventListener('pagehide', () => room.disconnect());


This section lists some of the important errors raised by twilio-video.js and provides recommendations on how best to handle them.

These errors are raised when twilio-video.js fails to acquire the user's local media (camera and/or microphone). Your app can catch these errors as shown below:


_31
const { connect, createLocalAudioTrack, createLocalTracks, createLocalVideoTrack } = require('twilio-video');
_31
_31
function handleMediaError(error) {
_31
console.error('Failed to acquire media:', error.name, error.message);
_31
}
_31
_31
// Handle media error raised by createLocalAudioTrack.
_31
createLocalAudioTrack().catch(handleMediaError);
_31
_31
// Handle media error raised by createLocalVideoTrack.
_31
createLocalVideoTrack().catch(handleMediaError);
_31
_31
// Handle media error raised by createLocalTracks.
_31
createLocalTracks().catch(handleMediaError);
_31
_31
const mediaErrors = [
_31
'NotAllowedError',
_31
'NotFoundError',
_31
'NotReadableError',
_31
'OverconstrainedError',
_31
'TypeError'
_31
];
_31
_31
// Since connect() will acquire media for the application if tracks are not provided in ConnectOptions,
_31
// it can raise media errors.
_31
connect(token, { name: 'my-cool-room' }).catch(error => {
_31
if (mediaErrors.includes(error.name)) {
_31
// Handle media error here.
_31
handleMediaError(error);
_31
}
_31
});

The following table describes the possible media errors and proposes ways for the application to handle them:

NameMessageCauseSolution
NotFoundError1. Permission denied by system 2. The object cannot be found here 3. Requested device not found1. User has disabled the input device for the browser in the system settings 2. User's machine does not have any such input device connected to it1. User should enable the input device for the browser in the system settings 2. User should have at lease one input device connected
NotAllowedError1. Permission denied 2. Permission dismissed 3. The request is not allowed by the user agent or the platform in the current context 4. The request is not allowed by the user agent or the platform in the current context, possibly because the user denied permission1. User has denied permission for your app to access the input device, either by clicking the "deny" button on the permission dialog, or by going to the browser settings 2. User has denied permission for your app by dismissing the permission dialog1. User should allow your app to access the input device in the browser settings and then reload 2. User should reload your app and grant permission to access the input device
TypeError1. Cannot read property 'getUserMedia' of undefined 2. navigator.mediaDevices is undefinedYour app is being served from a non-localhost non-secure contextYour app should be served from a secure context (localhost or https)
NotReadableError1. Failed starting capture of a audio track 2. Failed starting capture of a video track 3. Could not start audio source 4. Could not start video source 5. The I/O read operation failedThe browser could not start media capture with the input device even after the user gave permission, probably because another app or tab has reserved the input deviceUser should close all other apps and tabs that have reserved the input device and reload your app, or worst case, restart the browser
OverconstrainedErrorN/AThe input device could not satisfy the requested media constraintsIf this exception was raised due to your app requesting a specific device ID, then most likely the input device is no longer connected to the machine, so your app should request the default input device

NOTE: Each error can log a different message depending on the browser and OS. This table lists all possible messages associated with each error.

These errors are raised by twilio-video.js when it fails to join a Room. Your app can catch these errors as shown below:


_10
const { connect } = require('twilio-video');
_10
_10
connect(token, { name: 'my-cool-room' }).catch(error => {
_10
if ('code' in error) {
_10
// Handle connection error here.
_10
console.error('Failed to join Room:', error.code, error.message);
_10
}
_10
});

The following table describes the most common connection errors and proposes ways for the application to handle them:

ErrorCodeCauseSolution
SignalingConnectionError53000The client could not establish a connection to Twilio's signaling serverUser should make sure to have a stable internet connection
SignalingServerBusyError53006Twilio's signaling server is too busy to accept new clientsUser should try joining the Room again after some time
RoomMaxParticipantsExceededError53105The Room cannot allow in any more Participants to joinYour app should notify the user that the Room is full
RoomNotFoundError53106The client attempted to connect to a Room that does not existIf ad-hoc Room creation is disabled, then your app should make sure that the Room is created using the REST API before clients attempt to join
MediaConnectionError53405The client failed to establish a media connection with the Room1. User should make sure to have a stable internet connection 2. If the user is behind a firewall, then it should allow media traffic to and from Twilio to go through

These errors are raised by twilio-video.js when it is inadvertently disconnected from the Room. Your app can catch these errors as shown below:


_10
const { connect } = require('twilio-video');
_10
_10
connect(token, { name: 'my-cool-room' }).then(room => {
_10
room.once('disconnected', (room, error) => {
_10
if (error) {
_10
console.log('You were disconnected from the Room:', error.code, error.message);
_10
}
_10
});
_10
});

The following table describes the most common disconnection errors and proposes ways for the application to handle them:

ErrorCodeCauseSolution
SignalingConnectionDisconnectedError53001The client failed to reconnect to Twilio's signaling server after a network disruption or handoffUser should make sure to have a stable internet connection
SignalingConnectionTimeoutError53002The liveliness checks for the connection to Twilio's signaling server failed, or the current session expiredUser should rejoin the Room
ParticipantDuplicateIdentityError53205Another client joined the Room with the same identityYour app should make sure each client creates an AccessToken with a unique identity string
MediaConnectionError53405The client failed to re-establish its media connection with the Room after a network disruption or handoff1. User should make sure to have a stable internet connection 2. If the user is behind a firewall, then it should allow media traffic to and from Twilio to go through

This section lists some of the important warnings raised by twilio-video.js and provides recommendations on how best to handle them.

(information)

Info

The JavaScript SDK raises Media Warnings whenever the Twilio media server is not able to detect media from a published audio or video track. You can enable Media Warnings starting from version 2.22.0 of the Twilio Video JavaScript SDK(link takes you to an external page).

Enable Media Warnings

enable-media-warnings page anchor

You can enable Media Warnings with the notifyWarnings option in the SDK's ConnectOptions object(link takes you to an external page) when connecting to a Twilio Room:


_10
// Enable Media Warnings
_10
const room = await connect('token', {
_10
notifyWarnings: [ 'recording-media-lost' ]
_10
// Other connect options
_10
});

notifyWarnings takes an array of warnings to listen for. By default, this array is empty and no warning events will be raised.

Possible values to provide in the notifyWarnings array are:

  • recording-media-lost - Raised when the media server has not detected any media on the published track that is being recorded in the past 30 seconds. This usually happens when there are network interruptions or when the track has stopped.

Listen for Media Warning events

listen-for-media-warning-events page anchor

The SDK raises Media Warning events when it detects the conditions specified in the notifyWarnings options above. You can implement callbacks on these events to act on them when they happen, or to alert the user of an issue.

The warningsCleared event is raised when conditions have returned to normal.


_19
// Catch Media Warnings
_19
Array.from(room.localParticipant.tracks.values()).forEach(publication => {
_19
publication.on('warning', name => {
_19
if (name === 'recording-media-lost') {
_19
console.log(`LocalTrack ${publication.track.name} is not recording media.`);
_19
_19
// Wait a reasonable amount of time to clear the warning.
_19
const timer = setTimeout(() => {
_19
// If the warning is not cleared, you can manually
_19
// reconnect to the room, or show a dialog to the user
_19
}, 5000);
_19
_19
publication.once('warningsCleared', () => {
_19
console.log(`LocalTrack ${publication.track.name} warnings have cleared!`);
_19
clearTimeout(timer);
_19
});
_19
}
_19
});
_19
});

Media Warning events
media-warning-events page anchor
  • LocalTrackPublication.on('warning', callback(name)) - Raised when the published Track encounters a warning.
  • LocalTrackPublication.on('warningsCleared', callback()) - Raised when the published Track cleared all warning.
  • LocalParticipant.on('trackWarning', callback(name, publication)) - Raised when one of the LocalParticipant's published tracks encounters a warning.
  • LocalParticipant.on('trackWarningsCleared', callback(publication)) - Raised when one of the LocalParticipant's published tracks cleared all warning.
  • Room.on('trackWarning', callback(name, publication, participant)) - Raised when one of the LocalParticipant's published tracks in the Room encounters a warning.
  • Room.on('trackWarningsCleared', callback(publication, participant)) - Raised when one of the LocalParticipant's published tracks in the Room clears all warnings.

Rate this page: