Skip to contentSkip to navigationSkip to topbar
Rate this page:
On this page

Screen Capture - iOS 3.x


(warning)

Warning

This page is for reference only. We are no longer onboarding new customers to Programmable Video. Existing customers can continue to use the product until December 5, 2024(link takes you to an external page).
We recommend migrating your application to the API provided by our preferred video partner, Zoom. We've prepared this migration guide(link takes you to an external page) to assist you in minimizing any service disruption.

In this guide we'll show you how to share your iOS screen with other Participants connected to a Room using ReplayKit.framework. By using ReplayKit.framework, you have the ability to consume CMSampleBuffer objects that are produced by ReplayKit and convert them into VideoFrame objects that can be shared via a LocalVideoTrack.

There are two main ways that this can be accomplished depending on what screen content you wish to share. If you only want to share your application's screens, you can use the In-App Capture method. All of the code required is contained inside your application. If you want to share screens outside of your application, such as the Home Screen or other applications, you can use the Broadcast Extension method. Both methodologies are implemented in our ReplayKit Example(link takes you to an external page) application and should be used for reference.


VideoSource

videosource page anchor

No matter which method you choose, you will need to implement an object that conforms to the VideoSource protocol. This object is then provided when you create a LocalVideoTrack object. Your VideoSource object is responsible for consuming the CMSampleBuffer objects provided by ReplayKit and providing VideoFrame objects to its VideoSink.

The ReplayKitVideoSource class in our example application(link takes you to an external page) shows an implementation of this. Each timeReplayKit.framework produces a CMSampleBuffer object, it gets passed in to the processFrame() method. The ReplayKitVideoSource object handles the work of converting the frame into a VideoFrame object and passes it off to the VideoSink.

For example, creating a LocalVideoTrack to transmit video frames produced by ReplayKit using ReplayKitVideoSouce would be as follows.


_12
_12
var screenTrack: LocalVideoTrack?
_12
var videoSource: ReplayKitVideoSource?
_12
_12
...
_12
_12
videoSource = ReplayKitVideoSource(isScreencast: true,
_12
telecineOptions: ReplayKitVideoSource.TelecineOptions.disabled)
_12
_12
screenTrack = LocalVideoTrack(source: videoSource!,
_12
enabled: true,
_12
name: "Screen")

Once configured, you can share screenTrack as you would any other LocalVideoTrack in your Room. You can provide it when you join the room via the videoTracks ConnectOption or adding it later via your LocalParticipant object.


When you only want to share the screens of your application, you can use the In-App Capturing method. All of the ReplayKit.framework code required resides in your application. To capture your application's screens, you use the RPScreenRecorder(link takes you to an external page) class. To begin recording, you call the startCapture() method. The startCapture method takes a block that handles the incoming sampleBuffer objects. In this block you can pass the ReplayKit produced video frame off to your VideoSource consumer.


_36
_36
// Start recording the screen.
_36
let recorder = RPScreenRecorder.shared()
_36
_36
// We are only using ReplayKit to capture the screen.
_36
_36
// Use a LocalAudioTrack to capture the microphone for sharing audio in the room.
_36
recorder.isMicrophoneEnabled = false
_36
// Use a LocalVideoTrack with a CameraSource to capture the camera for sharing camera video in the room.
_36
recorder.isCameraEnabled = false
_36
_36
recorder.startCapture(handler: { (sampleBuffer, type, error) in
_36
if error != nil {
_36
print("Capture error: ", error as Any)
_36
return
_36
}
_36
_36
switch type {
_36
case RPSampleBufferType.video:
_36
self.videoSource?.processFrame(sampleBuffer: sampleBuffer)
_36
break
_36
case RPSampleBufferType.audioApp:
_36
// This exmaple does not capture audio generated by the application.
_36
break
_36
case RPSampleBufferType.audioMic:
_36
// We use `TVIDefaultAudioDevice` to capture and playback audio for conferencing.
_36
break
_36
}
_36
_36
}) { (error) in
_36
if error != nil {
_36
print("Screen capture error: ", error as Any)
_36
} else {
_36
print("Screen capture started.")
_36
}
_36
}

When you are done sharing your screen, you can simply call the stopCapture() method of RPScreenRecorder and remove your LocalVideoTrack using the methods on your LocalParticipant object.

For further guidance on this approach, check out the ViewController(link takes you to an external page) class in the ReplayKitExample(link takes you to an external page) example application.


Broadcast Extension Method

broadcast-extension-method page anchor

With ReplayKit, it is possible to not only share your application's screens but also other applications on the device. This configuration requires that you create a ReplayKit broadcast extension that is included with your application. In your ReplayKit broadcast extension, you create a subclass of RPBroadcastSampleHandler(link takes you to an external page). This class will be responsible for handling the samples produced by ReplayKit. The SampleHandler(link takes you to an external page) class in the ReplayKitExample(link takes you to an external page) shows an example implementation.

To start the ReplayKit broadcast, your application must present the RPBroadcastActivityViewController(link takes you to an external page) which allows the user to select your broadcast extension as the broadcasting service. Once the user starts the broadcast, the broadcastStarted() method will be invoked. The broadcastStarted() method in the example SampleHandler class then creates a ReplayKitVideoSource and an associated LocalVideoTrack, and connects to a Room. Each time ReplayKit generates a video frame, the processSampleBuffer() method is invoked. As in the In-App Capture Method above, the sampleBuffer is provided to the ReplayKitVideoSource for handling.

It is important to note that a ReplayKit broadcast extension must operate with limited memory. It is highly recommended that you use Group Rooms and set the isAutomaticSubscriptionEnabled ConnectOptions property to false when connecting to a Room within in the extension.

For further guidance on this approach, check out the ReplayKitExample(link takes you to an external page) example application.


Rate this page: