Skip to contentSkip to navigationSkip to topbar
Rate this Page:

Storing into AWS S3


(warning)

Warning

This page is for reference only. We are no longer onboarding new customers to Programmable Video. Existing customers can continue to use the product until December 5, 2024(link takes you to an external page).
We recommend migrating your application to the API provided by our preferred video partner, Zoom. We've prepared this migration guide(link takes you to an external page) to assist you in minimizing any service disruption.


Overview

overview page anchor

You can write your Video Recordings and Compositions to your own AWS (Amazon Web Services) S3 bucket, rather than Twilio's cloud. This guide explains how you can set up your own Twilio account or project to use this capability.

Note: Once you activate external S3 storage, Twilio will stop storing Programmable Video audio/video recordings into the Twilio cloud. It will be your responsibility to manage the security and lifecycle of your recorded content.

Use this feature when you need to meet compliance requirements that excludes reliance on third-party storage.



Recordings and Compositions in AWS S3

recordings-vs-compositions page anchor

Video Recordings and Video Compositions have separated S3 storage settings. This means that S3 storage can be activated independently on them. However, composing Recordings requires access to their media, which is not available if they are in an external S3 bucket. As a result:

If you store your Recordings in S3, you will not be able to compose them.

If you need to compose your Recordings, you must store them in Twilio's cloud. However, those Recordings are only needed temporarily: as soon as the Composition is created, they can be deleted permanently and irrevocably using Twilio's Video Recordings API.


Preparing your AWS account

preparing-aws page anchor

To configure external S3 storage, you will need:

  • The AWS S3 Bucket URL: The URL for the AWS S3 bucket of your choice.
  • The AWS Credentials: AWS Credentials (i.e. an Access Key ID and a Secret Access Key) for an AWS Identity and Access Management (IAM) user with write access to the bucket.

The rest of this section explains how to gather the items above. If you already know how to obtain the AWS S3 Bucket URL and the AWS Credentials, you can skip step 1 and step 2 and jump to the next section.

Step 1: Create an AWS S3 bucket and obtain its URL

step-1 page anchor

Amazon Simple Storage Service (S3)(link takes you to an external page) is designed to let you store and retrieve data from anywhere on the web. In S3, objects are stored into "buckets". Those can be seen as virtual folders where objects can be written, read and deleted.

Create an AWS S3 bucket.

First, create an S3 bucket.(link takes you to an external page) Use whatever bucket configuration that makes sense for your application; Twilio does not have any special bucket requirements. Remember to make note of the following, which you will need later:

Get the AWS S3 Bucket URL.

Next, get the URL for your S3 bucket.(link takes you to an external page) We recommend that you use the virtual-host-style URL based on the scheme https://bucket-name.s3-aws-region-code.amazonaws.com. Note that bucket-name is the name of your bucket and that you must replace aws-region-code with the AWS region code corresponding to your bucket-region. Check the AWS documentation(link takes you to an external page) for getting your aws-region-code.

After completing this step, you should have an AWS S3 Bucket URL like:

  • https://my-new-bucket.s3-us-east-2.amazonaws.com

Step 2: Create an IAM user and get its credentials

step-2 page anchor

IAM(link takes you to an external page) is Amazon's product for controlling access to your AWS services. An IAM user is a person or service that can access your AWS resources.

Create an IAM user(link takes you to an external page) using the AWS Console. You should make note of the following:

Set Programmatic access as the "Access type" for your IAM user

Select Attach existing policies directly and press the Create policy button to configure the user permissions:

After that, select Create policy, pick the JSON editor and create a policy document with write permissions. You can use the following JSON snippet as a template for the policy document. Note:

  • Replace my_bucket_name at the bottom of the snippet with the actual bucket-name , as obtained in step 1 above.
  • Replace the string /folder/for/storage/ with the specific path where you want Twilio to store your recordings within your bucket (note that / is a valid path). Don't forget the * wildcard at the end.

_21
{
_21
"Version": "2012-10-17",
_21
"Statement": [
_21
{
_21
"Sid": "UploadUserDenyEverything",
_21
"Effect": "Deny",
_21
"NotAction": "*",
_21
"Resource": "*"
_21
},
_21
{
_21
"Sid": "UploadUserAllowPutObject",
_21
"Effect": "Allow",
_21
"Action": [
_21
"s3:PutObject"
_21
],
_21
"Resource": [
_21
"arn:aws:s3:::my_bucket_name/folder/for/storage/*"
_21
]
_21
}
_21
]
_21
}

Now, come back to the original browser tab and press the Refresh button to see the policy you created. You can select it and complete the IAM user creation.

Get the IAM Access Key ID, Secret Access Key, and Path

  • Once the IAM user is created, Amazon provides you its credentials. They include an Access Key ID and a Secret Access Key. Store them in a secure location for later use.
  • You must also note down the path (e.g. /folder/for/storage/ ) where you provide Twilio write permissions.

Configuring your Twilio Account

configuring-twilio page anchor

Step 3: Configure Twilio with the AWS Credentials you created

step-3 page anchor

Next, you need to add a new AWS Credential to your Twilio account. For this, go to the Twilio Console AWS Credentials page(link takes you to an external page) and press Create new AWS Credential.

On the popup that opens, specify the friendly name you wish. Then, provide the AWS Access Key ID and the AWS Access Secret Key that you obtained in [step 2](#step 2) above. Finally, press Create.

After that, a newly created Twilio AWS Credential is listed in the Credentials page. Write down the AWS Credential SID that has the form CRxx.

Step 4: Configure Twilio to store into the S3 bucket

step-4 page anchor

Remark that when you activate this feature in either Recordings or Compositions, a small .txt test file will appear into your bucket. Twilio uses that file for double-checking that the write permissions you provided are working. You can remove the file safely if you want.

Storing Recordings into the S3 bucket

step-4-recordings page anchor

You have two options to enable Recordings S3 storage:

Enabling S3 Recordings storage using the Twilio's console

Enabling S3 Recordings storage using the Recording Settings API

Check the Recording Settings API documentation for detailed information on how to enable programmatically external S3 storage for your recordings.

Storing Compositions into the S3 bucket

step-4-compositions page anchor

You have two options to enable Compositions S3 storage:

Enabling S3 Compositions storage using the Twilio's console

Enabling S3 Compositions storage using the Recording Settings API

Check the Composition Settings API documentation for detailed information on how to enable programmatically external S3 storage for your compositions.


Uploading to buckets with Server-Side Encryption (SSE)

sse page anchor

Amazon S3 buckets support SSE (Server-Side Encryption)(link takes you to an external page). When enabled, all data written to disk is encrypted at the object level.

If you want to use SSE buckets to store your Twilio Recordings and Compositions, you must be aware that the only option we support is
SSE-KMS (SSE with AWS KMS-Managed Keys)(link takes you to an external page).

In order to make SSE-KMS to work with Twilio, you must grant access to the KMS key in your policy document. The following template illustrates how to do it:


_35
{
_35
"Version": "2012-10-17",
_35
"Statement": [
_35
{
_35
"Sid": "UploadUserDenyEverything",
_35
"Effect": "Deny",
_35
"NotAction": "*",
_35
"Resource": "*"
_35
},
_35
{
_35
"Sid": "UploadUserAllowPutObject",
_35
"Effect": "Allow",
_35
"Action": [
_35
"s3:PutObject"
_35
],
_35
"Resource": [
_35
"arn:aws:s3:::my_bucket_name/folder/for/storage/*"
_35
]
_35
},
_35
{
_35
"Sid": "AccessToKmsForEncryption",
_35
"Effect": "Allow",
_35
"Action": [
_35
"kms:Encrypt",
_35
"kms:Decrypt",
_35
"kms:ReEncrypt*",
_35
"kms:GenerateDataKey*",
_35
"kms:DescribeKey"
_35
],
_35
"Resource": [
_35
"arn:aws:kms:region:account-id:key/key-id"
_35
]
_35
}
_35
]
_35
}

Remember that:


  • Twilio Programmable Video has a limitation with bucket names that contain non US characters (e.g. ü, ç, é, etc.). Please, make sure that the bucket names you use contain only 7-bit ASCII characters.

Rate this Page: