Additions and changes to the Twilio platform
Video Log Analyzer API is now generally available.
This API allows developers to simplify troubleshooting and help identify commonalities in reported issues by programmatically retrieving critical metadata about Participant and Room behavior. The information provided includes: why a Participant left a Room, how the Room was ended, browser/operating system details, Participant error codes, and more.
The new Recording Rules API is now Generally Available. Previously, the video recording system would record all Room participants or none based on your initial decision. As such, it did not allow any changes to the recording while the Room was in progress until it was concluded.
Using the new Recording Rules API, you can programmatically introduce a set of rules which will give you more granular control over what you choose to record in every possible situation. You will be able to start and stop the recording of the Room tracks at any moment in time and as many times as you want. You will also be able to record either video, audio or both tracks from any of the participants in the Room.
Please visit the Recording Rules API documentation page to learn more about it, including code samples for making it easier for you to use.
The following Programmable Video SDK versions will End of Life on September 8th, 2021 for all customers:
- Android - 4.x and below
- iOS - 2.x and below
The above versions of the Video SDKs utilize infrastructure that is no longer considered reliable for customers. Therefore, they are currently not supported, and as of 5:00pm PT on September 8th, 2021, they will cease to function for all customers.
We are consistently updating the Twilio Video SDKs in order to provide the best possible experience for you. Upgrading to the latest SDK version will not only ensure you benefit from various performance enhancements and bug fixes, but will also allow you to take advantage of new capabilities such as the Track Priority API, Network Bandwidth API, and much more.
If you are still using one of these SDK versions and wish to continue using Twilio Programmable Video, you must upgrade your SDK to a supported version. If you are using a mobile SDK, please ensure your apps have been released and your end-users have updated before the End of Life date.
When upgrading your SDK, please refer to the following migration guides:
Today we are releasing two major version increments of our mobile SDKs for Video, iOS 4.0 and Android 6.0. The Chromium WebRTC engine in these SDKs has been upgraded from m67 to m83. In addition to the security and performance improvements, this WebRTC engine supports Simulcast screensharing and private IP concealment via mDNS.
These iOS and Android SDKs have added support for remote participant reconnecting events. Opus DTX is now included and enabled by default. In addition, the Android capturing and rendering programming models have been updated to provide developers with more flexibility and control. The iOS SDK is now built with Xcode 12.0 and delivered as .xcframework, and the application screen sharing capability has been simplified.
With the WebRTC upgrade developers get the latest enhancements to WebRTC and ensure compatibility with recent browser releases. Opus DTX significantly reduces the bandwidth consumed by silent participants. Remote reconnecting events provide developers a way to provide visual cues to users when automatic reconnections are happening.
Video applications typically have a small number of participants speaking and so the audio bandwidth consumed by silent participants is wasted. Although this feature will improve performance in all Video applications the impact will be most noticeable in larger multi-party Video Rooms.
This resource allows developers to simplify troubleshooting and help identify commonalities in reported issues by programmatically retrieving critical metadata about Participant and Room behavior. The information provided includes: why a Participant left a Room, how the Room was ended, browser/operating system details, Participant error codes, and more.
For more information, see the docs.
Twilio Video WebRTC Go is a new product offering that allows developers to build and operate 1:1 WebRTC based applications for free. This developer package provides the infrastructure components (signaling, TURN, logging, & diagnostics) required to run WebRTC applications in production. Participant minutes are free and the 25 GB/month of TURN server usage ensures enough relay support for roughly 100K minutes. To learn more, visit the documentation here.
In addition, a new one-on-one UI layout has been added to the Twilio Video quick deploy apps to support the new WebRTC Go Rooms and provide developers with a clean, modern visual design. Developers can now specify the desired Room type (group, peer-to-peer, or go) when building the quick deploy video app via the Twilio CLI RTC plugin. All the source code for the apps is available here:
- Web - https://github.com/twilio/twilio-video-app-react
- iOS - https://github.com/twilio/twilio-video-app-ios
- Android - https://github.com/twilio/twilio-video-app-android
For additional information about WebRTC Go and details on the new Video troubleshooting tools visit our blog post.
To help developers troubleshoot their applications at scale we are releasing three new Video troubleshooting tools. The first is a console tool called the Video Log Analyzer. It provides detailed information about Video Rooms and Participants. This can be used to quickly investigate Room and Participant issues that occurred in the past 2 days for WebRTC Go Rooms and 7 days for all other Rooms types.
The second is the RTC Diagnostics SDK 1.0.0-beta2, which provides test functions to allow developers troubleshoot camera, microphone, speaker, and network issues. This SDK is a general purpose WebRTC SDK and does not rely on Twilio infrastructure. Developers can incorporate it into their applications for pre-call testing or troubleshooting user issues. The source code and information on using the SDK is available here.
With this SDK release the local network path is no longer automatically included as one of the connection candidates. This modification ensures that users will not see a change in application behavior when upgrading to iOS 14.0 or iPadOS 14.0. Developers can still opt to include the local network as a connection candidate by setting a parameter. We recommend developers upgrade their Peer-to-Peer application to ensure correct behavior with iOS 14.0 and avoid any adverse changes to the experience.
We are introducing limits for the number of Rooms and Participants that an account can have in flight at the same time. These concurrency limits ensure a smooth consistent operation of the Twilio Video platform and prevent erroneous behavior on any one account from impacting the experience of other accounts.
The limits for all existing accounts have been set to at least double their peak production traffic, which ensures sufficient headroom for growth. These account limits can be increased on request. If a limit is reached an error code (53119, 53206) is returned and it is also displayed in the Console Debugger.
To learn more, visit the Programmable Video Limits page in our developer documentation.
The Network Bandwidth Profile API and associated Track Priority API provide developers with fine grained control over the allocation of bandwidth used by video tracks. Furthermore, the three profiles, “grid”, “collaboration”, and “presentation”, specify when tracks should be switched on or off to conserve bandwidth for the highest priority tracks. Using these controls developers can ensure their end users have the best possible experience.
To learn more visit the documentation on Using the Network Bandwidth Profile API and Using the Track Priority API. To see an example of how to use Network Bandwidth Profile API in a mobile application, visit the Video iOS Collaboration App or the Video Android Collaboration App.
These changes will reduce the connection failure rate and help developers to manage the end user experience if connection failures do occur. The SDK versions are as follows:
- Twilio Video iOS 3.3.0
- Twilio Video Android 5.7.0
Embedding VoIP calling functionality on Android requires adding support for Audio Device switching and routing. This functionality is complex as it requires development and testing to support various Android versions and the variety of input and output device types available.
The new open source Twilio AudioSwitch Library for Android removes this complexity and can be easily embedded in just a few lines of code to handle switching to and selecting the built-in, wired, or Bluetooth mics and speakers seamlessly.
This library is available in open source format and can be found here.
We are adding a new IP address block for the Media Server range in our Frankfurt, Germany (de1) region. This range will be used for the Programmable Video service and our Network Traversal Service. The new IP block is listed on the IP Addresses page in our Video documentation and on the Network Traversal Service Regions page.
If you have allowed the IP addresses of the de1 region in your Firewall, then your Firewall configuration will have to be updated to include the new IP block. This new block will go into rotation on April 16th, 2020.
The following additions to the Video Recording service are now Generally Available:
- Recording Composition API: provides developers flexibility to mix the media files of a Room recording into a high-quality, custom layout that matches their specific use case.
- Composition Hooks API: allows developers to streamline the recording workflow by automating the creation of Recording Compositions for every Room.
- Public Key Encryption for Recordings & Compositions: for enhanced security we are offering Enterprise customers the ability to encrypt recordings and compositions with a customer provided public encryption key
- External AWS S3 Storage: Enterprise customers can configure the Recording service to push Recordings or Compositions directly to their AWS S3 Bucket.
These Video recording features allow developers to create a recording workflow that matches their specific use case and security needs. For example, a customer in the Healthcare industry may decide to encrypt all recordings with their Public Key and have them automatically stored to their external storage, and a customer in the Education space may decide to compose the recording of a virtual classroom into a presentation layout for students.
Today we are excited to announce that we are open sourcing three video collaboration applications, one for iOS, one for Android, and a ReactJS one for the Web. These fully featured apps can be deployed to the Twilio cloud in minutes for your own personal, globally scalable, multiplatform video collaboration solution.
Access to the source code will empower developers to tailor the application to their use case and create a truly unique collaboration experience. The code also provides a canonical reference for developers building on the Twilio Programmable Video APIs.
For additional information on getting started see the blog post here and links to the source code repos are below.
Programmable Video can now support compliant healthcare uses cases that contain protected health information (PHI) for organizations that are subject to the Health Insurance Portability and Accountability Act (HIPAA). Twilio will sign Business Associate Addendums (BAA) with covered entities and business associates for HIPAA Eligible Products and Services.
Click here to learn more about how to build on Twilio for HIPAA compliance.
Track Priority and Network Bandwidth Profile API gives developers the ability to specify how bandwidth should be allocated between video tracks. Furthermore, the three profiles, (“grid”, “collaboration”, and “presentation”), specify when tracks should be switched off (or not) to conserve bandwidth for the highest priority tracks.
The Reconnection States and Events will automatically attempt to reconnect when a transient network error is encountered.
With Region Selection API, the SDK will automatically connect to the lowest latency data center. This API can also be configured to connect to a specific data center for cases where compliance might be required.
If migrating from a 1.x version, please refer to our migration guide.
To get started with Twilio Video JS, check our Getting Started Guide
The Video Composition Hooks REST API allows developers to automate the creation of Video Compositions. Using it, developers can create, fetch, edit and delete Composition Hooks. A Composition Hook is a template that defines how the Recordings of a Group Room should be mixed. When a Group Room is completed, all the enabled Composition Hooks in the Twilio Account will be executed against that room recordings and the corresponding compositions will be automatically created.
This release of Video Android SDK introduces the ability to select the region to use when joining a Room. The default setting uses Global Low Latency (GLL), a latency based routing algorithm that selects the region with the least latency. If you don’t want to use latency based selection, you can specify any of the supported regions to connect to via this new API.
Refer to this guide for more information about region selection.
For full changelog, visit the Video Android SDK release notes.
This feature is already available for iOS. For more information, visit the Video iOS SDK release notes.