Node.js Media SDK best practices

Push frames at their real frame rate and keep timestamps moving forward.

Check the return values of LocalVideoTrack.write() and LocalAudioTrack.write(). A return value of false always means the SDK doesn't send the frame:

• Video: The SDK rejects the frame, most often because you write it before connect() resolves. Video has no send queue, so don't retry the same frame. Send the next frame instead.
• Audio: The frame doesn't fit in the bounded publish queue, which holds about 500 ms of audio by default. The SDK doesn't queue any part of that write.

If the input is invalid, write() throws a TypeError or RangeError exception instead of returning false.

Audio has extreme time-sensitivity. If you write frames on a plain setInterval, playback might drift and click. Pace the playback using a drift-compensated writer that drains a buffer queue at exactly 48 kHz.

To review a working example, see audio_push.js and its helpers/paced-audio-writer.js in the SDK repository.

When you receive frames, the frames() queues are bounded and drop frames when your code falls behind. By default, video keeps only the newest frame, and audio buffers up to 10 frames. If your processing is slow, expect to skip video frames. To change this behavior, pass the mode, maxQueue, and drop options to frames(). To count drops, listen for the frameDropped event on the track.

A busy Room produces many large video frames. A 720p video track at 30 frames per second delivers about 41 MB of frame data each second.

• Each received frame is your own copy, so you can hold it across an await or pass it to a worker thread. Holding many frames keeps that memory in use.
• When you finish with a frame, call its close() method to release the plane buffers without waiting for garbage collection. After you call close(), reading the frame's plane data throws an error.
• The write() method copies your buffers before it returns, so you can reuse the buffers you send immediately.

Write checks for events about the connection state.

• To track connection state, listen for the reconnecting and reconnected events.
• To release its native resources when the Room emits disconnected, call room.dispose().

To stop using a track while you stay in the Room, do either of the following:

• To stop receiving a remote track's frames, exit its frames() loop with a break statement.
• To stop publishing a local track, call the LocalTrackPublication.unpublish() method.

When you finish with a Room, follow these steps:

1. Stop any ongoing push loops.
2. Leave the Room with the room.disconnect() method. Any frames() loops end on their own.
3. Release the Room's native resources with the room.dispose() method. Until you do, the Node.js process doesn't exit.

While developing with setLogLevel(), set the native log level. This method accepts a level name from off through all.

• If the connection fails, connect() rejects with a TwilioError. After you connect, the disconnected event passes a TwilioError when the Room disconnects because of an error. Branch on its subclass, such as AccessTokenInvalidError, RoomNotFoundError, or SignalingConnectionError.
• The ErrorCode enum lists the Twilio Video error codes.