---
"@context": https://schema.org
"@type": TechArticle
"@id": https://www.twilio.com/docs/video/node-working-with-media-frames#article
headline: Work with media frames in Node.js
description: Learn the I420 video and PCM audio frame formats the Video Media SDK for Node.js uses, and how to send and receive raw media frames in a Video Room.
url: https://www.twilio.com/docs/video/node-working-with-media-frames
inLanguage: en
dateModified: 2026-09-15T15:39:36.000Z
author:
  "@type": Organization
  name: Twilio Developer Education Team
publisher:
  "@type": Organization
  name: Twilio
---

# Work with media frames in Node.js

The Video Media SDK for Node.js works with media one *frame* at a time. A frame represents one unit of decoded media. You can send either one frame of video as one picture in [I420][] format or one frame of audio frame as a [sample of sound][pcm].

Using the SDK, write code that sends frames into a Room with the `write()` method and receives frames with an `onFrame()` callback.

## Video frames

Video frames use the [I420][] format. Each frame stores video data in three *planes* or memory blocks of luminance (`Y`) set per pixel and chrominance (`U`, `V`) set per four-pixel block. Each plane serves as a `Buffer`. Each also has a stride: the number of bytes per row. The stride spans the plane width at minimum. When rows get padded for alignment, the stride can exceed the plane width.

### Video frame data parameters

Each video frame consists of the following parameters:

| Parameter     | Type    | Necessity | Accepted values                                                                 |
| ------------- | ------- | --------- | ------------------------------------------------------------------------------- |
| `y`           | yPlane  | Required  | [Luminance][luma] frame data buffer                                             |
| `u`           | uPlane  | Required  | Blue-difference [chrominance][] (`Cb`) frame data buffer                        |
| `v`           | vPlane  | Required  | Red-difference chrominance (`Cr`) frame data buffer                             |
| `yStride`     | integer | Required  | Length of the luminance stride expressed in number of pixels.                   |
| `uStride`     | integer | Required  | Length of the blue-difference chrominance stride expressed in number of pixels. |
| `vStride`     | integer | Required  | Length of the red-difference chrominance stride expressed in number of pixels.  |
| `width`       | integer | Required  | Width of the frame expressed in number of pixels.                               |
| `height`      | integer | Required  | Height of the frame expressed in number of pixels.                              |
| `rotation`    | integer | Optional  | Degrees of rotation expressed as one of four values: `0`, `90`, `180`, `270`.   |
| `timestampNs` | bigint  | Required  | Timestamp expressed in number of nanoseconds.                                   |

### Video frame plane sizes

The luminance plane dimensions get set to the full size of the frame and the chrominance planes get set to half of the frame size.

| Plane | Logical size               | Buffer size              | Description                        |
| ----- | -------------------------- | ------------------------ | ---------------------------------- |
| `Y`   | `width` × `height`         | `yStride` × `height`     | Luminance                          |
| `U`   | `⌈width/2⌉` × `⌈height/2⌉` | `uStride` × `⌈height/2⌉` | Blue-difference chrominance (`Cb`) |
| `V`   | `⌈width/2⌉` × `⌈height/2⌉` | `vStride` × `⌈height/2⌉` | Red-difference chrominance (`Cr`)  |

Input and output use slightly different shapes:

* When you send a frame with the `write()` method, you pass a flat object with `y`, `u`, and `v` `Buffer` objects and their `yStride`, `uStride`, and `vStride` values.
* When you receive a frame, each plane arrives wrapped in an object with `data`, `stride`, `width`, and `height` fields.

## Audio frames

The input [audio frames][aframes] carry interleaved 48 kHz mono `S16LE` `PCM` samples in a single `Buffer`. You pass only the `pcm` buffer and the number of `frames`.

* `S`: Use *Signed* positive or negative integer values.
* `16`: Store *16 bits* (or two bytes) of data per audio sample.
* `LE`: Use the *little-endian* method to store data placing the least significant byte in the smallest memory address.
* `PCM`: Convert data using [*Pulse-code modulation*][pcm]: the raw, uncompressed audio wave data.

The output audio frames can vary. Each frame reports its own `sampleRate`, `channels`, and `frames`, along with the `pcm` buffer and a `bigint` `timestampNs`.

## Send frames into a Room

Until you connect to a Room, you can't access a track source. When the source isn't ready, the `write()` method returns `false` and throws an error on invalid input. After the `connect()` method returns a response, start your send loop.

### Send video frames

To publish video, create a local video track and call the `write()` method for each I420 frame.

```javascript title="Send one video frame"
const { createLocalVideoTrack } = require('@twilio/video-node-sdk');

const videoTrack = createLocalVideoTrack('virtual-camera');
// Pass videoTrack to connect() or publish it later.

videoTrack.write({
  y: yPlane,
  u: uPlane,
  v: vPlane,
  yStride: 1280,
  uStride: 640,
  vStride: 640,
  width: 1280,
  height: 720,
  timestampNs: process.hrtime.bigint(), // optional; defaults to now
});
```

A real app loops `write()` method calls at the source frame rate.

### Send audio frames

To publish audio, create an audio track and call the `write()` method for the 48 kHz mono PCM buffer of a certain number of frames:

```javascript title="Send a series of audio frames"
const { createLocalAudioTrack } = require('@twilio/video-node-sdk');

const audioTrack = createLocalAudioTrack('mic');

audioTrack.write({
  pcm: pcmBuffer, // interleaved int16 samples
  frames: 480, // samples in this buffer
});
```

## Receive frames from a Room

Register a callback with `onFrame()` on a subscribed remote track.

* With video, each plane arrives as an object with a `data` buffer and a `stride`.
* With audio, each frame arrives as an object with four properties.

```javascript title="Receive video or audio frames"
function trackSubscribed(track) {
  if (track.kind === 'video') {
    track.onFrame(frame => {
      const { width, height } = frame;
      const yData = frame.y.data;
      const yStride = frame.y.stride;
      // Process the frame, then return.
    });
  }

  if (track.kind === 'audio') {
    track.onFrame(frame => {
      // frame.pcm, frame.sampleRate, frame.channels, frame.frames
    });
  }
}
```

To stop receiving, call `removeFrameCallback()` on the track.

The following example comes from the [`video_mirror.js`][] example in the [SDK repository][]. This code receives remote video and sends it straight back into the Room, mapping each received plane onto a `write()` input.

```javascript title="Remote video sent to a room"
track.onFrame(frame => {
  videoTrack.write({
    y: frame.y.data,
    u: frame.u.data,
    v: frame.v.data,
    width: frame.width,
    height: frame.height,
    yStride: frame.y.stride,
    uStride: frame.u.stride,
    vStride: frame.v.stride,
    timestampNs: frame.timestampNs,
    rotation: frame.rotation,
  });
});
```

## Frame timing and pacing

Send frames at their real frame rate and keep timestamps moving forward.

Audio has extreme time-sensitivity. If you write frames on a plain `setInterval`, playback might drift and click. Pace the playback using a drift-compensated writer that drains a buffer queue at exactly 48 kHz.

To review a working example, see [`audio_push.js`][] and its [`helpers/paced-audio-writer.js`][] in the [SDK repository][].

## Next steps

* [Best practices][]: Pace frames, manage memory, and troubleshoot common issues.
* [API Reference][]: Browse the full frame and track APIs.

[`video_mirror.js`]: https://github.com/twilio/twilio-video-node/blob/main/examples/video_mirror.js

[`helpers/paced-audio-writer.js`]: https://github.com/twilio/twilio-video-node/blob/main/examples/helpers/paced-audio-writer.js

[`audio_push.js`]: https://github.com/twilio/twilio-video-node/blob/main/examples/audio_push.js

[aframes]: /docs/video/node-working-with-media-frames#audio-frames

[API Reference]: https://twilio.github.io/twilio-video-node/docs/latest

[Best practices]: /docs/video/node-best-practices

[chrominance]: https://en.wikipedia.org/wiki/Chrominance

[i420]: /docs/glossary/i420

[le]: https://en.wikipedia.org/wiki/Endianness

[luma]: https://en.wikipedia.org/wiki/Luma_\(video\)

[pcm]: https://en.wikipedia.org/wiki/Pulse-code_modulation

[SDK repository]: https://github.com/twilio/twilio-video-node/tree/main/examples

[tw-support]: https://help.twilio.com

[uid]: /docs/video/tutorials/user-identity-access-tokens

[vframes]: /docs/video/node-working-with-media-frames#video-frames

[Video Insights]: /docs/video/troubleshooting/insights

[VideoGrant]: /docs/video/tutorials/user-identity-access-tokens#generate-helper-lib

[Y'UV]: https://en.wikipedia.org/wiki/Y%E2%80%B2UV