Changelog: Twilio Video iOS 3.x
Warning
The Twilio Programmable Video SDKs use Semantic Versioning. Twilio supports version N-1 for 12 months after the first GA release of version N. We recommend you upgrade to the latest version as soon as possible to avoid any breaking changes. Version 4.x is the latest Video iOS version.
Support for 3.x will cease on December 4th, 2021. This branch will only receive fixes for critical issues until that date. Check this guide when planning your migration to 4.x.
Support for 2.x will cease on January 22nd, 2021. This branch will only receive fixes for critical issues until that date.
Support for iOS SDK version 1.x ended on April 26, 2019.
To make sure your app is ready for iOS 14 please visit this page.
The Twilio Programmable Video SDKs use Semantic Versioning.
Enhancements
isRecording
property inTVIRoom
now accurately reflects the current recording state of theTVIRoom
. In the previous versions of the SDK,isRecording
could return false positives. InTVIRoomDelegate
theroomDidStartRecording
androomDidStopRecording
callbacks will now be invoked when recording for at least a single track in theTVIRoom
has begun and ended respectively.
Known Issues
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
arm64 | 4.6 MB | 10.8 MB |
Bug Fixes
- Fixed a race condition that could cause a crash or unnecessary web socket signaling reconnection.
- Fixed a race condition that could cause a crash when closing the signaling web socket.
- Fixed a bug where private IP addresses could be sent to insights when
TVIConnectOptions.networkPrivacyPolicy
was set toTVILocalNetworkPrivacyPolicyBlockLocal
. - Two small memory leaks were fixed, one when iced response is received in icing state, and one for signaling data tracks. [ISDK-3046]
Known Issues
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
arm64 | 4.6 MB | 10.8 MB |
Bug Fixes
- Fixed an SCTP related WebRTC security vulnerability detailed here.
Known Issues
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
arm64 | 4.6 MB | 10.8 MB |
This release improves support for local network privacy on iOS 14.0 and iPadOS 14.0.
By default, the SDK no longer uses your local network on iOS 14 and above. Twilio Video balances privacy and performance, blocking communication routes that use your local network inappropriately in Peer-to-Peer Rooms while still ensuring a high rate of direct connections on iOS 14 devices.
If you use Peer-to-Peer Rooms then we recommend that you update your applications to this release in order to prepare for iOS 14. If you need features like multi-party video, network quality or recordings, then use Group Rooms. These Rooms use media servers provided by Twilio that are available on the public internet.
By default, Twilio Video does not access the local network on iOS 14 and above. If you want to allow Twilio Video to use the local network in a Peer-to-Peer Room then you can override the TVILocalNetworkPrivacyPolicy
.
1let connectOptions = ConnectOptions( token: accessToken, builder {2// A permissions request may be triggered on iOS 14.3builder.networkPrivacyPolicy = .allowAll4})
If you select TVILocalNetworkPolicyAllowAll
then you must add a privacy usage description with the NSLocalNetworkUsageDescription
key in your application's .plist file.
You can find more information about network privacy in our Getting Started Guide.
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
arm64 | 4.5 MB | 10.8 MB |
Bug Fixes
- Fixed a memory corruption crash that occurs when recovering from a media server failure in Group Rooms.
Known Issues
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
arm64 | 4.5 MB | 10.8 MB |
API Changes
TVIError
is now defined using theNS_ERROR_ENUM
macro which makes consumingTVIError
objects more idiomatic in Swift, such as using in a switch/case statement.
1func roomDidDisconnect(room: Room, error: Error?) {2if let error = error as? TwilioVideoSDK.Error {3switch error.code {4case .roomRoomCompletedError:5print ("The room completed gracefully")6case .participantDuplicateIdentityError:7print("Disconnected becasue another participant with the same identity joined.")8default:9print("Something else happened.")10}11}12}
Bug Fixes
- The camera capture background task logic has been refactored and iOS will no longer report a background task risk of termination warning. [ISDK-2878]
Known Issues
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
arm64 | 4.5 MB | 10.8 MB |
Enhancements
- Reduced connection times by acquiring IceServers during the initial handshake with Twilio's signaling server rather than sending a request to a different endpoint
- IceServers are now localized to your Participant's region rather than using Global Low Latency routing
- The SDK no longer depends on
ecs.us1.twilio.com:443
API Changes
- The following
TVIIceOptions
properties are deprecated and setting them has no effect:TVIIceOptions.abortOnIceServersTimeout
TVIIceOptions.iceServersTimeout
Bug Fixes
- Fixed a bug where IceServers might not be fetched on a dual-stack device where the IPv6 network interface is not reachable. [CSDK-3295]
Known Issues
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
arm64 | 4.5 MB | 10.8 MB |
Bug Fixes
- Fixed the issue where unpublishing and then republishing a
TVILocalVideoTrack
using VP8 simulcast does not complete. [CSDK-3433]
Known Issues
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
arm64 | 4.6 MB | 10.8 MB |
You can now configure how your available downlink bandwidth will be distributed among your subscribed RemoteVideoTracks by using a new optional ConnectOptions parameter bandwidthProfile. For more details and best practices, visit the Network Bandwidth Profile API guide. Here is a brief example:
1let videoOptions = VideoBandwidthProfileOptions { builder in2// Minimum subscribe priority of Dominant Speaker's RemoteVideoTracks3builder.dominantSpeakerPriority = .high45// Maximum bandwidth (Kbps) to be allocated to subscribed RemoteVideoTracks6builder.maxSubscriptionBitrate = 600078// Max number of visible RemoteVideoTracks. Other RemoteVideoTracks will be switched off9builder.maxTracks = 51011// Subscription mode: collaboration, grid, presentation12builder.mode = .presentation1314// Configure remote track's render dimensions per track priority15let renderDimensions = VideoRenderDimensions()1617// Desired render dimensions of RemoteVideoTracks with priority low.18renderDimensions.low = VideoDimensions(width: 352, height: 288)1920// Desired render dimensions of RemoteVideoTracks with priority standard.21renderDimensions.standard = VideoDimensions(width: 640, height: 480)2223// Desired render dimensions of RemoteVideoTracks with priority high.24renderDimensions.high = VideoDimensions(width: 1280, height: 720)2526builder.renderDimensions = renderDimensions2728// Track Switch Off mode: .detected, .predicted, .disabled29builder.trackSwitchOffMode = .predicted30}31let bandwidthProfileOptions = BandwidthProfileOptions(videoOptions: videoOptions)3233// Use bandwidth profile to build the connectOptions34let connectOptions = ConnectOptions(token: accessToken) { builder in35builder.bandwidthProfileOptions = bandwidthProfileOptions36}3738// Connect to a Room with connectOptions39room = TwilioVideoSDK.connect(options: connectOptions, delegate: self)
As part of the Bandwidth Profile API, you can now specify the mode to control remote video track switch off behavior. The new Track.SwitchOffMode
enum can be specified in the VideoBandwidthProfileOptions
class and can be set to one of the following:
Track.SwitchOffMode.predicted
- In this mode,RemoteVideoTrack
s are pro-actively switched off when network congestion is predicted by the bandwidth estimation mechanism. This mode is used by default if not specified.Track.SwitchOffMode.detected
- In this mode,RemoteVideoTrack
s are switched off only when network congestion is detected.Track.SwitchOffMode.disabled
- In this mode,RemoteVideoTrack
s will not be switched off. Instead tracks will be adjusted to lower quality.
Track switch off events are provided via two additional callbacks to RemoteParticipantDelegate
.
1extension ViewController : RemoteParticipantDelegate {2func remoteParticipantSwitchedOnVideoTrack(participant: RemoteParticipant,3track: RemoteVideoTrack) {}45func remoteParticipantSwitchedOffVideoTrack(participant: RemoteParticipant,6track: RemoteVideoTrack) {}7}
While publishing a local track, you can now optionally specify it's publish priority.
1/*2* Publish a local track with specified priorities.3*4* In addition, local track publications now contain a priority property which reflect the priority of5* the published track.6*/7let localVideoTrackPublicationOptions = LocalTrackPublicationOptions(priority: .high)8localParticipant.publishVideoTrack(localVideoTrack, publicationOptions: localVideoTrackPublicationOptions);910let localAudioPublicationOptions = LocalTrackPublicationOptions(priority: .high)11localParticipant.publishAudioTrack(localAudioTrack, publicationOptions: localAudioPublicationOptions);1213let localDataPublicationOptions = LocalTrackPublicationOptions(priority: .low)14localParticipant.publishDataTrack(localDataTrack, publicationOptions: localDataPublicationOptions);1516/*17* The publish priority is also represented in remote audio, video, and data track publications.18*/19let publishPriority = remoteVideoTrackPublication.publishPriority
Also, the publisher's TrackPriority
for the corresponding local Track can be updated after a
track has been published.
localVideoTrackPublication.priority = .high
When a TVITrackPriority
is updated after the track has been published, remote participants will be alerted to the change via the [TVIRemoteParticipantDelegate remoteParticipant\:didChangePublishPriority:forAudioTrack]
, [TVIRemoteParticipantDelegate remoteParticipant\:didChangePublishPriority:forVideoTrack]
and [TVIRemoteParticipantDelegate remoteParticipant\:didChangePublishPriority:forDataTrack]
delegate methods.
1extension ViewController : RemoteParticipantDelegate {2func remoteParticipantDidChangeAudioTrackPublishPriority(participant: RemoteParticipant,3priority: Track.Priority,4publication: RemoteAudioTrackPublication) { }56func remoteParticipantDidChangeVideoTrackPublishPriority(participant: RemoteParticipant,7priority: Track.Priority,8publication: RemoteVideoTrackPublication) { }910func remoteParticipantDidChangeDataTrackPublishPriority(participant: RemoteParticipant,11priority: Track.Priority,12publication: RemoteDataTrackPublication) { }13}
Additionally, you can set a subscriber side priority for a TVIRemoteVideoTrack
.
1/*2* Set a remote video track priority to HIGH. Note that this feature is only available3* for RemoteVideoTrack.4*/5remoteVideoTrack.priority = .high67/*8* Get the current remote video track priority.9*/10let remoteVideoTrackPriority = remoteVideoTrack.priority
This signals to the media server the relative importance of the track with respect to other tracks that may be shared to the Room
. The media server takes this into account while allocating a subscribing RemoteParticipant
's bandwidth to the corresponding remote track. If you do not specify a priority, the RemoteVideoTrack.priority
property will return nil
, and the media server will use the RemoteVideoTrackPublication.publishPriority
assigned by the publisher instead.
New APIs
- Added
TVITrackPriority
for track priorities of Room. - Added a new readonly property
publishPriority
of typeTVITrackPriority
inTVIRemoteAudioTrackPublication
,TVIRemoteVideoTrackPublication
, andTVIRemoteDataTrackPublication
. These readonly properties fetch the priorities at which the remote Audio, Video, and Data Tracks were published. - Added a new class
TVILocalTrackPublicationOptions
to specify track publication options when publishing tracks viaTVILocalParticipant
. - Added additional publish track methods that allow developers to specify
TVILocalTrackPublicationOptions
-[TVILocalParticipant publishAudioTrack:publicationOptions:]
[TVILocalParticipant publishVideoTrack:publicationOptions:]
[TVILocalParticipant publishDataTrack:publicationOptions:]
- Added a new property
priority
of typeTVITrackPriority
inTVILocalAudioTrackPublication
,TVILocalVideoTrackPublication
andTVILocalDataTrackPublication
classes. - Added the
[TVIRemoteParticipantDelegate remoteParticipant\:didChangePublishPriority:forAudioTrack]
,[TVIRemoteParticipantDelegate remoteParticipant\:didChangePublishPriority:forVideoTrack]
and[TVIRemoteParticipantDelegate remoteParticipant\:didChangePublishPriority:forDataTrack]
delegate methods.
Bug Fixes
- Fixed a crash when the SDK was processing a local SDP that had incorrect number of SSRCs when applying VP8 simulcast.
Known Issues
- Unpublishing and then republishing a
TVILocalVideoTrack
using VP8 simulcast does not complete. As a workaround, disable the Track instead. [CSDK-3433] - Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
arm64 | 4.6 MB | 10.8 MB |
Enhancements
- Previously, twilio-video would fail connection attempts if the signaling server was busy with too many requests. Now, it will try again after a server specified backoff period either until it is successfully connected or the server asks it to stop trying. In this case, the
-[TVIRoomDelegate room\:didFailToConnectWithError:]
method is called withTVIErrorSignalingServerBusyError
.
1func roomDidFailToConnect(room: Room, error: Error) {2let theError = error as NSError3if theError.domain == TwilioVideoSDK.ErrorDomain,4theError.code == TwilioVideoSDK.Error.signalingServerBusyError {5print("Twilio's signaling server cannot accept connection requests at this time.")6}7}
API Changes
- Added error code definitions:
TVIErrorSignalingServerBusyError
TVIErrorRoomAccountLimitExceededError
TVIErrorParticipantAccountLimitExceededError
TVIErrorTrackDataTrackMessageTooLargeError
TVIErrorTrackDataTrackSendBufferFullError
TVIErrorMediaDtlsTransportFailedError
TVIErrorMediaIceRestartNotAllowedError
Known Issues
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
arm64 | 4.5 MB | 10.7 MB |
Enhancement
TVICameraSource
now stops the underlying AVCaptureSession during deallocation if the developer had not stopped it using one of thestopCapture
methods. It is still suggested that the developer calls one of thestopCapture
methods at a time that is appropriate for their application and not relying on the SDK to do it. #445- Publish stats reports every 10 seconds and active ICE candidate pair messages every 20 seconds. Previously both were reported every 4 seconds.
Known Issues
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
arm64 | 4.5 MB | 10.7 MB |
Bug Fixes
- Fixed a bug when establishing a signaling WebSocket that might cause only one address family (IPv4 or IPv6) to be tried.
- Fixed a network handoff bug where the media was not switching over to low cost network.
Known Issues
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
arm64 | 4.5 MB | 10.7 MB |
Bug Fixes
- Fixed a potential crash when destroying a
TVILocalVideoTrack
while the Track'sTVICameraSource
(or customTVIVideoSource
) is still delivering frames. [ISDK-2792]
Known Issues
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
arm64 | 4.5 MB | 10.6 MB |
Bug Fixes
- Fixed the issue where the
NS_SWIFT_NAME
mapping forCameraSourceOptionsBuilder.Block
would fail. #72
Known Issues
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
arm64 | 4.5 MB | 10.6 MB |
Bug Fixes
- Fixed parsing issue when a Remote Participant's Network Quality Level is zero. [CSDK-3325]
Known Issues
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
arm64 | 4.5 MB | 10.6 MB |
API Changes
- Implemented Network Quality for Remote Participants in Group Rooms:
- To enable the Network Quality functionality, set the
TVIConnectOptions.networkQualityEnabled
property toYES
when connecting to a Group Room. By default this enables network quality level changes to be reported for the Local Participant. To also receive network quality level changes for the Remote Participants, a configuredTVINetworkQualityConfiguration
object needs to be supplied to theTVIConnectOptions.networkQualityConfiguration
property. - The
networkQualityLevel
property has been promoted from theTVILocalParticipant
class to theTVIParticipant
base class. - To determine the current network quality level for both Local and Remote Participants, query the
networkQualityLevel
property. Note, this will returnTVINetworkQualityLevelUnknown
if:- The
TVIConnectOptions.networkQualityEnabled
property was set toNO
OR - Using a Peer-to-Peer room OR
- The network quality level has not yet been computed
- The
- Implementing the
[TVILocalParticipantDelegate localParticipant:networkQualityLevelDidChange:]
method on yourTVILocalParticipantDelegate
will allow you to receive callbacks when the network quality level changes for the Local Participant. - Implementing the
[TVIRemoteParticipantDelegate remoteParticipant:networkQualityLevelDidChange:]
method on yourTVIRemoteParticipantDelegate
will allow you to receive callbacks when the network quality level changes for the Remote Participant.
- To enable the Network Quality functionality, set the
1@IBAction func connectToRoom(sender: AnyObject) {2let connectOptions = ConnectOptions(token: accessToken) { (builder) in3builder.isNetworkQualityEnabled = true4builder.networkQualityConfiguration = NetworkQualityConfiguration(localVerbosity: .minimal,5remoteVerbosity: .minimal)6builder.roomName = "my-room"7}89room = TwilioVideoSDK.connect(options: connectOptions, delegate: self)10}1112...1314// MARK: LocalParticipantDelegate15func localParticipantNetworkQualityLevelDidChange(participant: LocalParticipant, networkQualityLevel: NetworkQualityLevel) {16print("Local Participant Network Quality Level Changed: \(networkQualityLevel)")17}1819// MARK: RemoteParticipantDelegate20func remoteParticipantNetworkQualityLevelDidChange(participant: RemoteParticipant, networkQualityLevel: NetworkQualityLevel) {21print("Remote Participant (\(participant.identity)) Network Quality Level Changed: \(networkQualityLevel)")22}
Enhancements
- Reduced bandwidth usage for Insights reporting by up to 75%.
- Data track per-message size limit has been increased to 64 KB. [CSDK-3197]
Bug Fixes
- Fixed intermittent crash when sending data on a data track while in the process of closing the track due to remote participant disconnect. [CSDK-3245]
TVIIceCandidatePairStats.availableOutgoingBitrate
is populated with the available send bandwidth when publishing Tracks.TVIIceCandidatePairStats.availableIncomingBitrate
is populated with the available receive bandwidth when subscribing to Tracks in a Group Room.- Fixed a crash which could occur when destroying a
TVIRoom
while the TCP handshake for the signaling connection is in progress.
Known Issues
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
arm64 | 4.5 MB | 10.6 MB |
Enhancements
- Reduced memory allocations when cropping is applied with
-[TVIVideoSource requestOutputFormat:]
by caching the cropping attributes. - Added
TVICameraSourceOptions.rotationTags
, which defaults toTVICameraSourceOptionsRotationTagsKeep
. You might want to set this option toTVICameraSourceOptionsRotationTagsRemove
in order to:- Save power and battery life when you know that the video will be published in a Group Room
- Receive images that are properly oriented for Computer Vision analysis
- Use
AVAssetWriter
for recordings without worrying about transformation matrices
- Performance Improvements:
- With
TVICameraSourceOptionsRotationTagsRemove
, we see significant improvement with the H.264 video codec on modern iPhones. - With
TVICameraSourceOptionsRotationTagsRemove
, we see small improvement with the VP8 video codec (non-simulcast) on modern iPhones.
- With
Codec | TVICameraSourceOptionsRotationTagsKeep (Average CPU) (*) | TVICameraSourceOptionsRotationTagsRemove (Average CPU) (*) |
---|---|---|
H.264 | 7.3% | 6.75% (**) |
VP8 | 12.8% | 12.7% (**) |
1* Tested on iPhone 112* Group Room3* Capture resolution, fps, encoding params: 640x480x30 @ 1200 kbps4* Video call duration: 5 mins / run5* Measured using Instruments6** Load scheduled mostly on high efficiency cores, with performance cores disabled.
Known Issues
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
arm64 | 4.5 MB | 10.7 MB |
Bug Fixes
- Fixed a possible deadlock that could occur when destroying a
TVIRoom
without first calling-[TVIRoom disconnect]
. - Fixed a crash which could occur when destroying a
TVIRoom
with dominant speaker enabled.
Known Issues
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
arm64 | 4.5 MB | 10.7 MB |
3.0 is now generally available. Thank you for all the feedback from our beta adopters.
This release includes:
- Global low latency (GLL) signaling with IPv4 and IPv6 support.
- Improved Swift APIs for a more idiomatic Swift development experience.
- Support for UIScene and multi-window apps on iOS 13 and iPadOS 13.
Bug Fixes
- Fixed a regression in 3.0.0-beta5 and 3.0.0-beta6 where the Participant's TLS handshake might be delayed by up to 250 milliseconds. [CSDK-3117]
Known Issues
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
arm64 | 4.5 MB | 10.7 MB |
This release completes the API changes needed for Twilio Video to be fully compatible with the iOS 13.0 and iPadOS 13.0 SDKs. This release also removes support for iOS 10. Twilio Video 2.x will continue to support iOS 9.0 and later.
API Changes
- The minimum supported iOS version has been raised to 11.0.
TVICameraSource
can monitorUIWindowScene
for orientation and state changes in iOS 13.0+.- Removed manual orientation control APIs from
TVICameraSource
and introduced theTVICameraSourceOrientationTracker
protocol along with theTVIUserInterfaceTracker
class. - The following example demonstrates how to track a
UIWindowScene
:
1// Track the orientation of the key window's scene.2let options = CameraSourceOptions { (builder) in3if let keyScene = UIApplication.shared.keyWindow?.windowScene {4builder.orientationTracker = UserInterfaceTracker(scene: keyScene)5}6}7let camera = CameraSource(options: options, delegate: self)
1// Forward UIWindowScene events2func windowScene(_ windowScene: UIWindowScene,3didUpdate previousCoordinateSpace: UICoordinateSpace,4interfaceOrientation previousInterfaceOrientation: UIInterfaceOrientation,5traitCollection previousTraitCollection: UITraitCollection) {6UserInterfaceTracker.sceneInterfaceOrientationDidChange(windowScene)7}
Known Issues
- The Participant's TLS handshake might be delayed by up to 250 milliseconds. [CSDK-3117]
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
arm64 | 4.5 MB | 10.7 MB |
API Changes
- Added
TVILocalParticipant.signalingRegion
. You can use this property to determine where your Participant connected to a Room, especially when using the default value of "gll" forTVIConnectOptions.region
. - Added
TVIRoom.mediaRegion
. You can use this property to determine where media is being processed in a Group Room. - Updated the
TVIConnectOptionsBuilder.region
property to no longer throw an exception when an empty string is provided and instead default togll
. - When a Participant connects to a Room, the WebSocket handshake is required to complete in 15 seconds or less, otherwise
TVIErrorSignalingConnectionError
is raised. TVIEncodingParameters
now expresses maximum bitrates in Kilobits per second (Kbps) instead of bits per second (bps).
1// Before2let params = EncodingParameters(audioBitrate: UInt(64000),3videoBitrate: UInt(800000))
1// After2let params = EncodingParameters(audioBitrate: UInt(64),3videoBitrate: UInt(800))
Enhancements
- The iOS SDK is built with Xcode 11.0
TVIVideoView
registers forUIScene
lifecycle notifications when it is contained within aUIWindowScene
on iOS 13.0+. Rendering is permitted in the following states:UISceneActivationStateForegroundActive
at 60 HzUISceneActivationStateForegroundInactive
at 30 Hz
TVIVideoView
suspends its internalCADisplayLink
timer when it is removed from aUIWindow
.- Increased the reliability of
TVILocalDataTrack
by monitoring its send buffer. Sending too many messages, or sending single messages larger than 16 KB no longer causes the underlying channel(s) to close. These requests are ignored instead. [CSDK-3037]
Bug Fixes
- Fixed a bug where
TVICameraSource
might produce distorted frames after being interrupted withAVCaptureSessionInterruptionReasonVideoDeviceInUseByAnotherClient
on iOS 13.0. #53 - Making the SDK more aggressive at establishing a connection by trying to connect multiple sockets to different endpoints in parallel. [CSDK-2942]
TVILocalDataTrack
should no longer end up in an inconsistent state after attempting to send a message larger than 16 KB. [CSDK-3081]
Known Issues
- The Participant's TLS handshake might be delayed by up to 250 milliseconds. [CSDK-3117]
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
Universal | 9.5 MB | 20.5 MB |
arm64 | 4.6 MB | 10.8 MB |
armv7 | 4.9 MB | 9.7 MB |
API Changes
- Introduced
TVIErrorSignalingDnsResolutionError
, which is now raised instead ofTVIErrorSignalingConnectionError
in the following scenarios:- The device has misconfigured DNS Server(s) on its active network interface.
- The region provided in
TVIConnectOptions
was invalid. - The device lost its Internet connection before the query could complete.
Enhancements
- Reduced connection times by removing a round trip when:
- Reconnecting after a signaling connection failure
- Connecting with
TVIConnectOptions.iceOptions
, and overridden Servers - Connecting with default ICE servers (us1 only)
- [CSDK-2902]
Bug Fixes
- Fixed a bug where Participants remain connected to a Room even after terminal media failures. [CSDK-3023]
- Fixed a crash when Network Quality or Dominant Speaker is enabled, connectivity to the Room is lost, and the Room is deleted from within
TVIRoomDelegate.roomDidDisconnect()
. [CSDK-2950] - Fixed a bug where resources might be leaked when destroying a Room. [CSDK-2822]
- Fixed a bug where network monitoring would continue on closed connections in a Peer-to-Peer Room. [CSDK-2947]
- Fixed a crash on the iOS 13 simulator when rendering received H.264 video, or any other video in the
TVIPixelFormatYUV420BiPlanarFullRange
orTVIPixelFormatYUV420BiPlanarVideoRange
formats. [ISDK-2631]
Known Issues
- In rare cases, the SDK might timeout during a TCP handshake and should be more aggressive at establishing a connection. [CSDK-2942]
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
Universal | 9.4 MB | 21.0 MB |
arm64 | 4.4 MB | 11.2 MB |
armv7 | 4.8 MB | 9.5 MB |
Enhancements
- The Participant's signaling connection now conforms to Twilio's TLS & Cipher Suite Security Policy. Support for TLS versions older than 1.2 has been removed. [CSDK-3020]
- Adjusted the buffer sizes used for signaling messages to reduce network fragmentation. [CSDK-2920]
- Setting
TVILogModuleSignaling
enables logging of low level connection events atTVILogLevelTrace
. [ISDK-2446]
Bug Fixes
- WebSocket errors are handled immediately, rather than waiting for a timeout to occur. [CSDK-3025]
- Handle rare exceptions when constructing a WebSocket. [CSDK-2755]
Known Issues
- Future 3.0.0-beta releases will reduce the number of round-trips required to connect to a Room. [CSDK-2902]
- In rare cases, the SDK might timeout during a TCP handshake and should be more aggressive at establishing a connection. [CSDK-2942]
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
Universal | 9.4 MB | 21.0 MB |
arm64 | 4.4 MB | 11.2 MB |
armv7 | 4.8 MB | 9.5 MB |
API Changes
- Renamed
+ [TwilioVideoSDK version]
to+ [TwilioVideoSDK sdkVersion]
to alleviate the clash withNSObject
'sversion
method. - Removed the
type
argument label for the Swift definition of[TVICameraSource captureDeviceForPosition:type]
. In Swift, the function is now declared asclass func captureDevice(position: AVCaptureDevice.Position, deviceType: AVCaptureDevice.DeviceType) -> AVCaptureDevice?
. - Updated the Swift audio sample rate constant names to remove the redundant
Audio
. For instanceAudioFormat.AudioSampleRate8000
is nowAudioFormat.SampleRate8000
. - Updated the Swift audio format channels constant names to remove the redundant
Audio
. For instanceAudioFormat.AudioChannelsMono
is nowAudioFormat.ChannelsMono
. - Refactored the
TVILocalParticipantDelegate
andTVIRemoteParticipantDelegate
methods to follow thedid
naming convention for consistency with the other delegate methods. See the Migrating from 2.x to 3.x guide for more information.
Bug Fixes
- The SDK will consistently recover the Participant's connection after multiple failed network handovers, or multiple failures on a single network. [CSDK-2957, CSDK-3007]
- Failures in the Participant's connection are always detected within 15 seconds. [CSDK-2927]
- Improved DNS retry logic and error handling. [CSDK-2801]
- Disabled server side deflate compression due to occasional errors when reading messages. [CSDK-3004]
- Security improvements. [CSDK-2986, CSDK-3005]
Known Issues
- Future 3.0.0-beta releases will reduce the number of round-trips required to connect to a Room. [CSDK-2902]
- Setting
TVILogModuleSignaling
does not produce any logging. [ISDK-2446] - In rare cases, the SDK might timeout during a TCP handshake and should be more aggressive at establishing a connection. [CSDK-2942]
- If a WebSocket error occurs, it might not be handled for up to 15 seconds. [CSDK-3025]
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
Universal | 9.4 MB | 20.9 MB |
arm64 | 4.4 MB | 11.1 MB |
armv7 | 4.7 MB | 9.5 MB |
The 3.0 release removes support for iOS 9.0, and raises the minimum supported iOS version to 10.0. Twilio Video 2.x releases will continue to support iOS 9.0 and later.
Enhancements
- The SDK uses a new WebSocket based signaling transport, and communicates with globally available signaling Servers over IPv4 and IPv6 networks.
- Added the
TVIConnectOptions.region
property. By default, the Client will connect to the nearest signaling Server determined by latency based routing. Setting a value other than "gll" bypasses routing and guarantees that signaling traffic will be terminated in the region that you prefer.
1let connectOptions = ConnectOptions(token: accessToken) { (builder) in2builder.region = "us1"3builder.roomName = "my-room"4}5self.room = TwilioVideoSDK.connect(with: connectOptions, delegate: self)
- Participants are considered to be reconnecting within 15 seconds, and are disconnected from a Room after 45 seconds of lost connectivity. #99
API Changes
- Twilio Video has been updated for better Swift interoperability. See the Migrating from 2.x to 3.x guide for more information.
- The
TwilioVideo
Obj-C class has been renamed toTwilioVideoSDK
to reduce possible namespace issues in Swift due to the Module containing a Type with the same name. - The
TVI
prefix has been removed from all Twilio Video types - Many of the delegate function declarations have been revamped for better clarity of their intent
- Some of the class method declarations have been revamped for better clarity of their intent
- The
- The
TVIAudioSessionActivated()
andTVIAudioSessionDeactivated()
methods have been removed as the Video SDK no longer needs to terminate the signaling connection when an app is backgrounded. - Removed the deprecated
TVIVideoCapturer
APIs:TVIVideoCapturer
TVICameraCapturer
TVIScreenCapturer
TVIVideoConstraints
TVILocalVideoTrack.capturer
TVILocalVideoTrack.constraints
TVILocalVideoTrack.trackWithCapturer:
TVILocalVideoTrack.trackWithCapturer:enabled:constraints:name:
- Removed
TVIAudioOptions
that are no longer supported by WebRTC:TVIAudioOptions.levelControl
TVIAudioOptions.levelControlInitialPeakLevelDBFS
TVIAudioOptionsBuilder.levelControl
TVIAudioOptionsBuilder.levelControlInitialPeakLevelDBFS
- Removed the
TVIVideoRenderer.optionalPixelFormats
property due to lack of use. ATVIVideoRenderer
is expected to support any validTVIPixelFormat
or drop incoming frames if it can't process them.
Bug Fixes
- Participants can send and receive messages that are larger than 16 KB.
- Renamed
kDefaultAVAudioSessionConfigurationBlock
tokTVIDefaultAVAudioSessionConfigurationBlock
so thatTwilioVoice.framework
andTwilioVideo.framework
may be used in the same application. - Fixed a crash that could occur when using a
TVIVideoSource
, and the source's buffer pool becomes empty.
Known Issues
- Future 3.0.0-beta releases will reduce the number of round-trips required to connect to a Room. [CSDK-2902]
- If heartbeats are not received, the signaling connection may take up to 45 seconds to be disconnected instead of 15 seconds. [CSDK-2927]
- The SDK may crash if Network Quality or Dominant Speaker is enabled, and connectivity to the Room is lost, and the Room is deleted from within
[TVORoomDelegate didDisconnectWithError:]
. [CSDK-2950] - Setting
TVILogModuleSignaling
does not produce any logging. [ISDK-2446] - In rare cases, the SDK might timeout during a TCP handshake and should be more aggressive at establishing a connection. [CSDK-2942]
- The WebSocket might close with a partial_deflate_block (code 5) error. [CSDK-2949]
- Unpublishing and republishing a
TVILocalAudioTrack
orTVILocalVideoTrack
might not be seen by Participants. #34 - Pre iPhone X iOS devices do not support more than three H.264 encoders. Refer to #17 for suggested work around.
- Publishing H.264 video at greater than 1280x720 @ 30fps is not supported. If a failure occurs then no error is raised to the developer. [ISDK-1590]
Size Impact
Architecture | Compressed Size | Uncompressed Size |
---|---|---|
Universal | 8.9 MB | 19.3 MB |
arm64 | 4.2 MB | 10.3 MB |
armv7 | 4.5 MB | 8.7 MB |