By
Today, we’re proud to announce a host of new Programmable Video features that will help you reach more users with an even better live video experience.
- Support for Safari 11 on iOS and macOS: Reach nearly any web user on any platform with live video within the web browser. Support for Safari on macOS was added in version 1.2.1 of twilio-video.js, and support for Safari on iOS was added in 1.3.0.
- Updated SDKs and Group Rooms features to enable support for hardware-accelerated H.264 video codec: Support for H.264 makes it possible to connect native mobile apps with Safari 11, and taking advantage of hardware acceleration can improve mobile device battery life in your video apps.
- APIs for Codec Preferences and Bandwidth Constraints: Have more control over the quality of service features in your application.
Read on for more information.
Using Programmable Video in Safari 11
Apple’s latest release of Safari 11 added support for WebRTC—a major milestone for the web, with all major browsers now supporting the standard. Safari has an estimated 15% share of the worldwide browser market. It is the only solution for browsing the web on iOS. We’re thrilled that all major browsers now support the standard, and your video apps can now reach nearly every web user on the planet.
Programmable Video uses WebRTC to enable real-time audio and video in your web applications. To build a video app for Safari 11, you will need a development device that’s running the latest release. Safari 11 should be generally available next week with the release of iOS 11 and on macOS Sierra via Software Update, but you can grab the Technology Preview release on Apple’s Developer site—it has a few tools that will come in handy during development, such as the ability to capture media on insecure sites.
After you’ve installed Safari 11, grab the latest release of twilio-video.js via NPM:
npm install --save twilio-video
If you don’t use NPM, check out the twilio-video.js Github project for other ways to install the library.
If you’re adding Safari 11 support to an existing Programmable Video application that’s using an earlier version of twilio-video.js, the upgrade will be seamless. If this this is your first Programmable Video app, try starting with our JavaScript quickstart.
There are some other nuances to developing a video app in Safari, so make sure you read our guide to Programmable Video in Safari 11 as you plan your implementation–but the most important items are outlined below.
Safari 11 only supports H.264 video codec
Safari 11 only supports the H.264 video codec. This means that you will need to either use Peer-to-Peer Rooms, or configure your Group Rooms to use the H.264 codec to connect from Safari 11 (see below for more information on this).
If your app uses Peer-to-Peer Rooms, you shouldn’t have to do anything: Chrome and Firefox both support H.264, and will automatically send and receive H.264 video tracks to any Safari 11 users who join the Room. Our native SDKs at 2.0.0-preview1 and greater will automatically use H.264 when required.
Connecting to Group Rooms from Safari 11
Today, we’re also releasing support for H.264 in Group Rooms and Recording. To create a Room which forces all participants to use H.264, use the new VideoCodecs property when creating the Room using the REST API:
curl -XPOST 'https://video.twilio.com/v1/Rooms' \
-u 'SKXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX:your_api_key_secret' \
-d 'UniqueName=MyH264Room' \
-d 'Type=group' \
-d 'VideoCodecs=H264'
H.264 Support in Mobile SDKs
We’re also happy to announce that we’ve added support for H.264 codec in our Android and iOS SDKs, starting at version 2.0.0-preview1. This change lets you to take advantage of hardware acceleration on all iOS devices and on many Android devices.
Because H.264 is required by Safari 11, you’ll need to update to 2.0.0-preview1 or higher if you ship a native mobile version of your app and you want it to be able to talk to Safari 11.
On iOS, get 2.0.0-preview1 by updating your podfile and running pod install, or downloading it here:
source 'https://github.com/CocoaPods/Specs'
platform :ios, '8.1'
target 'TARGET_NAME' do
pod 'TwilioVideo', '2.0.0-preview1'
End
On Android, get 2.0.0-preview1 by updating you build.gradle file or downloading it from jCenter:
allprojects {
repositories {
jcenter()
}
}
// The Video library resides on jcenter
compile 'com.twilio:video-android:2.0.0-preview1'
H.264 support in both Group Rooms and in our mobile SDKs is a developer preview feature. You can find an updated list of known issues in our changelogs for each SDK (iOS, Android), so you might find some occasional bugs in interoperability–please report them if you do!
Client-side Codec Preferences selection
While we’re on the topic of codecs, we’ve also added APIs that give you client-side control over codec preferences.
Our new Codec Preferences API allows you to set preferred codecs when publishing Tracks. When you connect to a Room, you can specify an optional preferredAudioCodecs array and an optional preferredVideoCodecs array.
These are codec “preferences” because they will only be applied if your device and the type of Room you are connected to support them. If a preference cannot be satisfied, we will fallback to the next-best codec.
For example, to tell our JavaScript SDK to connect with a preferred video codec of H.264:
const room = await connect(token, {
preferredVideoCodecs: ['H264']
});
You can also specify more than one preferred codec. For example, to connect with a preferred audio codec of iSAC, falling back to Opus if iSAC is unavailable:
const room = await connect(token, {
preferredAudioCodecs: ['isac', 'opus']
});
Send-side Bandwidth Constraints
Our SDKs, and WebRTC more generally, try to provide the highest-possible audio and video quality on all network conditions. By default, WebRTC-based applications will consume up to 40 kilobits per second for audio, and up to 2 megabits per second for video.
If available bandwidth changes during a live call, bandwidth adaptation and congestion control mechanisms will kick in—decreasing the quality of the video when bandwidth is low, and increasing video quality when more bandwidth is available.
This is great for services that always want to maximize the video resolution and frame rate. However, some developers want to more tightly control bandwidth usage, either to minimize data use for their end users, or to reduce variability in the video stream due to bandwidth adaptation.
With our new bandwidth constraints API, you can now specify an optional maxAudioBitrate and an optional maxVideoBitrate in bits per second (bps) when you connect to a Room. This sets a fixed ceiling on the bandwidth of the audio and video used when publishing tracks to the Room (these settings do not govern bandwidth consumed when subscribing to tracks). These values are set as hints for variable bitrate codecs, but will not take effect for fixed bitrate codecs.
For example, to connect with a maximum audio bitrate of 64 kilobits per second and a maximum video bitrate of 500 kilobits per second using our JavaScript SDK:
const room = await connect(token, {
maxAudioBitrate: 64000,
maxVideoBitrate: 500000
});
You can also update your maxAudioBitrate and maxVideoBitrate while participating in a Room. For example, to reset your maximum bitrates for audio and video, you could set each to null:
room.localParticipant.setParameters({
maxAudioBitrate: null,
maxVideoBitrate: null
});
For more information, read the API docs for Android, iOS, and JavaScript.