Video Media SDK for Node.js quickstart

To use the Video Media SDK, you need the following prerequisites:

• Create a Twilio account.

• Install Node.js version 24.0.0 or later on Linux x86-64, or on macOS x86-64 for local development. On an Apple Silicon Mac, you need an x64 build of Node.js. To learn more, see system requirements.

• Create an API key SID and secret.

  View how to create an API Key

• To run the virtual_camera.js example, install FFmpeg and make sure ffmpeg is on your PATH. On macOS, run brew install ffmpeg. On Debian or Ubuntu, run sudo apt install ffmpeg.

1. Install the SDK and the Twilio helper library:
   Copy code block

   1
   npm install twilio
   2
   npm install @twilio/video-node-sdk

   You use the twilio helper library to create Access Tokens.
2. In your app, add the import statement for the SDK:
   Copy code block

   const { connect, createLocalVideoTrack } = require('@twilio/video-node-sdk');

The connect() function takes a standard Twilio Video Access Token with a VideoGrant, the same token format the JavaScript SDK uses. Generate an Access Token on your server with the twilio helper library.

Create a local video track, and then pass it to connect(). The returned promise resolves after the Room connects. Because connect() returns a promise, call it from an async function. The code in the following sections runs inside that function.

Unlike the client-side SDKs, a local track lacks a camera. To supply raw I420 video frames, call the write() method on the track. Each call takes the frame dimensions and the y, u, and v planes. Each plane object contains its data as a Buffer, its stride, and its own width and height.

1
videoTrack.write({
2
  format: 'I420',
3
  width: 1280,
4
  height: 720,
5
  y: { data: yPlane, stride: 1280, width: 1280, height: 720 },
6
  u: { data: uPlane, stride: 640, width: 640, height: 360 },
7
  v: { data: vPlane, stride: 640, width: 640, height: 360 },
8
});

To learn about I420 video planes and strides and PCM audio, see Work with media frames.

Remote media arrives as raw decoded frames. Listen for trackSubscribed, then read frames from each video or audio track with the frames() async iterator.

1
async function trackSubscribed(track) {
2
  if (track.kind !== 'video') return;
3
  try {
4
    // Frames that arrive while you process one are queued, and the SDK drops
5
    // them if the queue fills. The loop ends by itself when the track is
6
    // unsubscribed or the Room disconnects.
7
    for await (const frame of track.frames()) {
8
      console.log(`${frame.width}x${frame.height} @ ${frame.timestamp}us`);
9
      frame.close?.();
10
    }
11
  } catch (err) {
12
    // Nothing awaits this function, so an error that escapes here becomes an
13
    // unhandled rejection and stops the process.
14
    console.error('Frame loop failed:', err);
15
  }
16
}
17

18
function participantConnected(participant) {
19
  participant.on('trackSubscribed', trackSubscribed);
20

21
  participant.tracks.forEach(publication => {
22
    if (publication.isSubscribed) {
23
      trackSubscribed(publication.track);
24
    }
25
  });
26
}
27

28
// participantConnected doesn't fire for Participants already in the Room, and a
29
// track can finish subscribing before the trackSubscribed listener is attached.
30
// Call participantConnected for each Participant in room.participants, and check
31
// isSubscribed on each publication.
32
room.participants.forEach(participantConnected);
33
room.on('participantConnected', participantConnected);
34

35
room.on('disconnected', () => {
36
  room.dispose();
37
});

When you finish with a Room, call room.dispose(). Until you do, the Node.js process doesn't exit. Call it with the disconnected event, as in the preceding example.

The SDK repository includes runnable examples. The virtual_camera.js example decodes an MP4 with ffmpeg and sends its frames into a Room.

1
git clone https://github.com/twilio/twilio-video-node.git
2
cd twilio-video-node
3
npm install --prefix examples
4
cp .env.example .env
5
# Edit .env and set TWILIO_ACCOUNT_SID, TWILIO_API_KEY, and TWILIO_API_SECRET.
6
node examples/virtual_camera.js my-room

To review every example, see the examples directory.

• Work with media frames: Learn the I420 video and PCM audio frame formats in depth.
• Differences from the JavaScript SDK: Map what you know from the browser SDK to the server.
• Best practices: Pace frames, manage resources, and troubleshoot common issues.