Media Support

Programmable Chat supports media messages, allowing your users to add photos, video or any other type of file to their chat. A media message appears on the channel similar to a text only message but with additional properties that let your client know the media's size as well as an optional content type and default filename. When creating your media message, you will pass media to the Programmable Chat SDK which will be read and saved to your Chat instance. Your application can later request a stream to retrieve the media associated with a message and display it to your users.

Please note Media messaging is currently a beta feature.

Using Media Messaging via the client SDKs (iOs, Android and Javascript)

A typical media message creation will include the following steps with the details depending on your client platform:

  • Create a new message, passing in a stream to the media and its mime content type. (The current maximum size of media accepted is 150mb.)
  • Optionally specify a default download filename to help your application display the media to other users.
  • Programmable Chat will provide you feedback of upload progress as well as an indication your media has been successfully saved. (iOS and Android only)
  • The message is created on the specified channel and the channel's members are notified.

On the receiving side of a media message, you will:

  • Receive a chat message that includes a media SID.
  • When you're ready to display the media to your user, you'll obtain the download URL (Javascript) or call the download method providing where you would like the media saved (Android and iOS.)
  • Programmable Chat will provide feedback as the download progresses and a completion call when the media is ready.
  • Your client will display or otherwise make available to the user the message's media content.

Media on messages within Programmable Chat are attachments which live separately from your chat message. They are associated to the owning message by a media SID. Media files cannot exist without their owning message, and deletion of a message will result in cleanup of it's associated media. Media files are immutable once created, you can modify other supported attributes of a message that has media content but the media itself is not changeable. Media messages do not contain a per-message text portion, instead being populated by the service with a placeholder message for legacy clients without media attachment support – this placeholder message is developer definable on the service instance.

Most developers will want to implement media caching to avoid unnecessary battery or network consumption should the application need to view the same media multiple times. An example of such a caching mechanism can be found in the helper classes of the Chat demo applications published to GitHub. It is also a good idea to prevent concurrent downloads of the same media in your clients, the helper library also handles queueing up requests for the same media until a single download completes before returning the requested media object.

Required Role Permission

For your users to create messages with media content, their Role must contain the sendMediaMessage and sendMessage permissions. On new Chat instances this is in the default channel roles, but must be added to existing instances if you wish to support it. For an example on how to apply this permission to your Channel User and Admin roles, see Update a Role or Roles and Permissions for more general information on Roles in Programmable Chat.

Platform Differences for Media

The media creation and download methods within the client SDK's take a stream or file as their parameter. How the media is expressed in the client SDK will depend on your platform:

Javascript

For Javascript, you can provide a FormData containing file information (including file name, content type, size and all FormData provided information), a String or a Node.js Buffer containing media byte stream to be used as the source for a new media message. On the receiver side media is exposed with help of temporary URL (the URL is invalidated after 300 seconds. You can request new temporary URL at any point of time).

iOS

Media files are uploaded by providing an NSInputStream compliant stream to the TCHMessageOptions for your new message. Likewise, any NSOutputStream can be provided as the destination for a media download. Often these streams will be backed by a file but they can just as easily live in memory, pass through to a network connection, or something else entirely.

Android

For Android, any java.io.InputStream compliant stream can be used as the source for a new media message. Likewise, any java.io.OutputSteram can be provided as the destination for a media download. Often these streams will be backed by a file but they can just as easily live in memory, pass through to a network connection, or something else entirely.

Creating a Media Message

Creating a media enriched message is very similar to creating a new text-only message - you start by creating a message as usual but instead of body for the content, you provide media for the message:

Loading Code Samples...
Language
// Messages messagesObject;
messagesObject.sendMessage(
    Message.options()
        .withMedia(new FileInputStream("/path/to/Somefile.txt"), "text/plain")
        .withMediaFileName("file.txt")
        .withMediaProgressListener(new ProgressListener() {
            @Override
            public void onStarted() {
                Timber.d("Upload started");
            }

            @Override
            public void onProgress(long bytes) {
                Timber.d("Uploaded " + bytes + " bytes");
            }

            @Override
            public void onCompleted(String mediaSid) {
                Timber.d("Upload completed");
            }
        }),
    new CallbackListener<Message>() {
        @Override
        public void onSuccess(Message msg) {
            Timber.d("Successfully sent MEDIA message");
        }

        @Override
        public void onError(ErrorInfo error) {
            Timber.e("Error sending MEDIA message");
        }
    });
// example for sending media message as FormData
// ---------------------------------------------
const formData = new FormData();
formData.append('file', $('#formInputFile')[0].files[0]);
// get desired channel (for example, with getChannelBySid promise)
chatClient.getChannelBySid(channelSid).then(function(channel) {
  // send media with all FormData parsed atrtibutes
  channel.sendMessage(formData);
});

// example for sending media message as String
// -------------------------------------------
// get desired channel (for example, with getChannelBySid promise)
chatClient.getChannelBySid(channelSid).then(function(channel) {
  // send SVG image as string with content type image/svg+xml; charset=utf-8
  channel.sendMessage({
    contentType: 'image/svg+xml; charset=utf-8',
    media:
      '<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 100 100">' +
      '<path d="M50,3l12,36h38l-30,22l11,36l-31-21l-31,21l11-36l-30-22h38z"' +
      ' fill="#FF0" stroke="#FC0" stroke-width="2"/></svg>',
  });
});

// example for sending media message as Buffer
// -------------------------------------------
// get desired channel (for example, with getChannelBySid promise)
chatClient.getChannelBySid(channelSid).then(function(channel) {
  // send PNG image as Buffer with content type image/png
  channel.sendMessage({
    contentType: 'image/png',
    media: fs.readFileSync(pngFile),
  });
});
// The data for the image you would like to send
NSData *data = nil;

// Prepare the upload stream and parameters
TCHMessageOptions *messageOptions = [[TCHMessageOptions alloc] init];
NSInputStream *inputStream = [NSInputStream inputStreamWithData:data];
[messageOptions withMediaStream:inputStream
                    contentType:@"image/jpeg"
                defaultFilename:@"image.jpg"
                      onStarted:^{
                          // Called when upload of media begins.
                          NSLog(@"Media upload started");
                      } onProgress:^(NSUInteger bytes) {
                          // Called as upload progresses, with the current byte count.
                          NSLog(@"Media upload progress: %ld", bytes);
                      } onCompleted:^(NSString * _Nonnull mediaSid) {
                          // Called when upload is completed, with the new mediaSid if successful.
                          // Full failure details will be provided through sendMessage's completion.
                          NSLog(@"Media upload completed");
                      }];

// Trigger the sending of the message.
[self.channel.messages sendMessageWithOptions:messageOptions
                                   completion:^(TCHResult *result, TCHMessage *message) {
                                       if (!result.isSuccessful) {
                                           NSLog(@"Creation failed: %@", result.error);
                                       } else {
                                           NSLog(@"Creation successful");
                                       }
                                   }]; 
// The data for the image you would like to send
let data = Data()

// Prepare the upload stream and parameters
let messageOptions = TCHMessageOptions()
let inputStream = InputStream(data: data)
messageOptions.withMediaStream(inputStream,
                               contentType: "image/jpeg",
                               defaultFilename: "image.jpg",
                               onStarted: {
                                // Called when upload of media begins.
                                print("Media upload started")
},
                               onProgress: { (bytes) in
                                // Called as upload progresses, with the current byte count.
                                print("Media upload progress: \(bytes)")
}) { (mediaSid) in
    // Called when upload is completed, with the new mediaSid if successful.
    // Full failure details will be provided through sendMessage's completion.
    print("Media upload completed")
}

// Trigger the sending of the message.
self.channel.messages?.sendMessage(with: messageOptions,
                                   completion: { (result, message) in
                                    if !result.isSuccessful() {
                                        print("Creation failed: \(String(describing: result.error))")
                                    } else {
                                        print("Creation successful")
                                    }
}) 
Creating a Media Message

Checking for Media Content

Only messages which contain media will have a media SID associated with them. The code sample provided here shows how to detect a media message on each platform:

Loading Code Samples...
Language
if (message.hasMedia()) {
    Message.Media media = message.getMedia();
}
// get desired channel (for example, with getChannelBySid promise)
chatClient.getChannelBySid(channelSid).then(function(channel) {
  // get channel's messages paginator
  channel.getMessages().then(function(messagesPaginator) {
    // check the first message type
    const message = messagesPaginator.items[0];
    if (message.type === 'media') {
      console.log('Message is media message');
      // log media properties
      console.log('Media properties', message.media);
    }
  });
});
if (message.hasMedia) {
    NSLog(@"mediaFilename: %@ (optional)", message.mediaFilename);
    NSLog(@"mediaSize: %ld", message.mediaSize);
} 
if (message.hasMedia) {
    print("mediaFilename: \(String(describing: message.mediaFilename)) (optional)")
    print("mediaSize: \(message.mediaSize)")
} 
Checking for Media Content

Retrieving Message Media Content

Samples on how to obtain a message's media content are provided here. It's important to note that at this time, Programmable Chat does not maintain an internal cache of media so you are encouraged to cache downloaded media in your application to avoid unnecessary downloads. It is also important for best performance to only allow a particular media file to be downloaded once at a time to avoid unnecessary battery and network consumption.

Loading Code Samples...
Language
if (message.hasMedia()) {
    Message.Media media = message.getMedia();

    String sid = media.getSid();
    String type = media.getType();
    String fn = media.getFileName();
    long size = media.getSize();

    Timber.d("This is a media message with SID "+sid+", type "+type+", name "+fn+", and size "+size);

    if (type.contentEquals("text/plain")) {
        final ByteArrayOutputStream out = new ByteArrayOutputStream();
        media.download(out, new StatusListener() {
            @Override
            public void onSuccess() {
                String content = out.toString();
                Timber.d("Downloaded media "+content);
            }

            @Override
            public void onError(ErrorInfo error) {
                Timber.e("Error downloading media");
            }
        }, new ProgressListener() {
            @Override
            public void onStarted() {
                Timber.d("Download started");
            }

            @Override
            public void onProgress(long bytes) {
                Timber.d("Downloaded "+bytes+" bytes");
            }

            @Override
            public void onCompleted(String mediaSid) {
                Timber.d("Download completed");
            }
        });
    }
}
// get desired channel (for example, with getChannelBySid promise)
chatClient.getChannelBySid(channelSid).then(function(channel) {
  // get channel's messages paginator
  channel.getMessages().then(function(messagesPaginator) {
    // check the first message type
    const message = messagesPaginator.items[0];
    if (message.type === 'media') {
      console.log('Message is media message');
      // log media properties
      console.log('Media attributes', message.media);
      // get media temporary URL for displaying/fetching
      message.media.getContentUrl().then(function(url) {
        // log media temporary URL
        console.log('Media temporary URL is ' + url);
      });
    }
  });
});
// Set up output stream for media content
NSString *tempFilename = [NSTemporaryDirectory() stringByAppendingPathComponent:message.mediaFilename ? @"file.dat" : message.mediaFilename];
NSOutputStream *outputStream = [NSOutputStream outputStreamToFileAtPath:tempFilename append:NO];

// Request the start of the download
[message getMediaWithOutputStream:outputStream
                        onStarted:^{
                            // Called when download of media begins.
                        } onProgress:^(NSUInteger bytes) {
                            // Called as download progresses, with the current byte count.
                        } onCompleted:^(NSString * _Nonnull mediaSid) {
                            // Called when download is completed, with the new mediaSid if successful.
                            // Full failure details will be provided through the completion block below.
                        } completion:^(TCHResult * _Nonnull result) {
                            if (!result.isSuccessful) {
                                NSLog(@"Download failed: %@", result.error);
                            } else {
                                NSLog(@"Download successful");
                            }
                        }]; 
// Set up output stream for media content
let tempFilename = (NSTemporaryDirectory() as NSString).appendingPathComponent(message.mediaFilename ?? "file.dat")
let outputStream = OutputStream(toFileAtPath: tempFilename, append: false)

// Request the start of the download
if let outputStream = outputStream {
    message.getMediaWith(outputStream,
                         onStarted: {
                            // Called when download of media begins.
    },
                         onProgress: { (bytes) in
                            // Called as download progresses, with the current byte count.
    },
                         onCompleted: { (mediaSid) in
                            // Called when download is completed, with the new mediaSid if successful.
                            // Full failure details will be provided through the completion block below.
    }) { (result) in
        if !result.isSuccessful() {
            print("Download failed: \(String(describing: result.error))")
        } else {
            print("Download successful")
        }
    }
}
Retrieving Message Media Content

Using Media Messaging via the Chat REST API

Your backend services can also use the Media feature off Chat - uploading new files for attachment with Messages, and of course sending Messages with Media attached.

Note: At present the underlying Media REST endpoint which is used to create (upload) the media (files) is a separate endpoint and not supported in the Twilio Helper Libraries.

Sending a Media Message

Sending a Media Message via REST is a 2 step process:

  1. Upload media to MCS
  2. Send Media Message to chat channel (linking/attaching the Media created in step 1 to the new Message)

Upload/Create the Media

A Curl example:
curl -u “<acount_sid>:<account_secret>” --data-binary “@<filename>” https://mcs.us1.twilio.com/v1/Services/<chat_service_sid>/Media

Sending the Message with the Media attached

Please see the REST Message endpoint documentation for information on sending Messages. To attach the Media to the new Message, you need to pass the Media SID for the Media created in step 1 when creating the Message - this attaches/links the Media to the Message to be sent.

A Curl example:
curl -u “<acount_sid>:<account_secret>” -X POST https://chat.twilio.com/v2/Services/<chat_service_sid>/Channels/<channel_sid>/Messages -d MediaSid=<media_sid>

Now that you're ready to enable your users to share the finest cute animal photos the internet has to offer, let's learn about Result Paging in Chat

Need some help?

We all do sometimes; code is hard. Get help now from our support team, or lean on the wisdom of the crowd browsing the Twilio tag on Stack Overflow.

Loading Code Samples...
// Messages messagesObject;
messagesObject.sendMessage(
    Message.options()
        .withMedia(new FileInputStream("/path/to/Somefile.txt"), "text/plain")
        .withMediaFileName("file.txt")
        .withMediaProgressListener(new ProgressListener() {
            @Override
            public void onStarted() {
                Timber.d("Upload started");
            }

            @Override
            public void onProgress(long bytes) {
                Timber.d("Uploaded " + bytes + " bytes");
            }

            @Override
            public void onCompleted(String mediaSid) {
                Timber.d("Upload completed");
            }
        }),
    new CallbackListener<Message>() {
        @Override
        public void onSuccess(Message msg) {
            Timber.d("Successfully sent MEDIA message");
        }

        @Override
        public void onError(ErrorInfo error) {
            Timber.e("Error sending MEDIA message");
        }
    });
// example for sending media message as FormData
// ---------------------------------------------
const formData = new FormData();
formData.append('file', $('#formInputFile')[0].files[0]);
// get desired channel (for example, with getChannelBySid promise)
chatClient.getChannelBySid(channelSid).then(function(channel) {
  // send media with all FormData parsed atrtibutes
  channel.sendMessage(formData);
});

// example for sending media message as String
// -------------------------------------------
// get desired channel (for example, with getChannelBySid promise)
chatClient.getChannelBySid(channelSid).then(function(channel) {
  // send SVG image as string with content type image/svg+xml; charset=utf-8
  channel.sendMessage({
    contentType: 'image/svg+xml; charset=utf-8',
    media:
      '<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 100 100">' +
      '<path d="M50,3l12,36h38l-30,22l11,36l-31-21l-31,21l11-36l-30-22h38z"' +
      ' fill="#FF0" stroke="#FC0" stroke-width="2"/></svg>',
  });
});

// example for sending media message as Buffer
// -------------------------------------------
// get desired channel (for example, with getChannelBySid promise)
chatClient.getChannelBySid(channelSid).then(function(channel) {
  // send PNG image as Buffer with content type image/png
  channel.sendMessage({
    contentType: 'image/png',
    media: fs.readFileSync(pngFile),
  });
});
// The data for the image you would like to send
NSData *data = nil;

// Prepare the upload stream and parameters
TCHMessageOptions *messageOptions = [[TCHMessageOptions alloc] init];
NSInputStream *inputStream = [NSInputStream inputStreamWithData:data];
[messageOptions withMediaStream:inputStream
                    contentType:@"image/jpeg"
                defaultFilename:@"image.jpg"
                      onStarted:^{
                          // Called when upload of media begins.
                          NSLog(@"Media upload started");
                      } onProgress:^(NSUInteger bytes) {
                          // Called as upload progresses, with the current byte count.
                          NSLog(@"Media upload progress: %ld", bytes);
                      } onCompleted:^(NSString * _Nonnull mediaSid) {
                          // Called when upload is completed, with the new mediaSid if successful.
                          // Full failure details will be provided through sendMessage's completion.
                          NSLog(@"Media upload completed");
                      }];

// Trigger the sending of the message.
[self.channel.messages sendMessageWithOptions:messageOptions
                                   completion:^(TCHResult *result, TCHMessage *message) {
                                       if (!result.isSuccessful) {
                                           NSLog(@"Creation failed: %@", result.error);
                                       } else {
                                           NSLog(@"Creation successful");
                                       }
                                   }]; 
// The data for the image you would like to send
let data = Data()

// Prepare the upload stream and parameters
let messageOptions = TCHMessageOptions()
let inputStream = InputStream(data: data)
messageOptions.withMediaStream(inputStream,
                               contentType: "image/jpeg",
                               defaultFilename: "image.jpg",
                               onStarted: {
                                // Called when upload of media begins.
                                print("Media upload started")
},
                               onProgress: { (bytes) in
                                // Called as upload progresses, with the current byte count.
                                print("Media upload progress: \(bytes)")
}) { (mediaSid) in
    // Called when upload is completed, with the new mediaSid if successful.
    // Full failure details will be provided through sendMessage's completion.
    print("Media upload completed")
}

// Trigger the sending of the message.
self.channel.messages?.sendMessage(with: messageOptions,
                                   completion: { (result, message) in
                                    if !result.isSuccessful() {
                                        print("Creation failed: \(String(describing: result.error))")
                                    } else {
                                        print("Creation successful")
                                    }
}) 
if (message.hasMedia()) {
    Message.Media media = message.getMedia();
}
// get desired channel (for example, with getChannelBySid promise)
chatClient.getChannelBySid(channelSid).then(function(channel) {
  // get channel's messages paginator
  channel.getMessages().then(function(messagesPaginator) {
    // check the first message type
    const message = messagesPaginator.items[0];
    if (message.type === 'media') {
      console.log('Message is media message');
      // log media properties
      console.log('Media properties', message.media);
    }
  });
});
if (message.hasMedia) {
    NSLog(@"mediaFilename: %@ (optional)", message.mediaFilename);
    NSLog(@"mediaSize: %ld", message.mediaSize);
} 
if (message.hasMedia) {
    print("mediaFilename: \(String(describing: message.mediaFilename)) (optional)")
    print("mediaSize: \(message.mediaSize)")
} 
if (message.hasMedia()) {
    Message.Media media = message.getMedia();

    String sid = media.getSid();
    String type = media.getType();
    String fn = media.getFileName();
    long size = media.getSize();

    Timber.d("This is a media message with SID "+sid+", type "+type+", name "+fn+", and size "+size);

    if (type.contentEquals("text/plain")) {
        final ByteArrayOutputStream out = new ByteArrayOutputStream();
        media.download(out, new StatusListener() {
            @Override
            public void onSuccess() {
                String content = out.toString();
                Timber.d("Downloaded media "+content);
            }

            @Override
            public void onError(ErrorInfo error) {
                Timber.e("Error downloading media");
            }
        }, new ProgressListener() {
            @Override
            public void onStarted() {
                Timber.d("Download started");
            }

            @Override
            public void onProgress(long bytes) {
                Timber.d("Downloaded "+bytes+" bytes");
            }

            @Override
            public void onCompleted(String mediaSid) {
                Timber.d("Download completed");
            }
        });
    }
}
// get desired channel (for example, with getChannelBySid promise)
chatClient.getChannelBySid(channelSid).then(function(channel) {
  // get channel's messages paginator
  channel.getMessages().then(function(messagesPaginator) {
    // check the first message type
    const message = messagesPaginator.items[0];
    if (message.type === 'media') {
      console.log('Message is media message');
      // log media properties
      console.log('Media attributes', message.media);
      // get media temporary URL for displaying/fetching
      message.media.getContentUrl().then(function(url) {
        // log media temporary URL
        console.log('Media temporary URL is ' + url);
      });
    }
  });
});
// Set up output stream for media content
NSString *tempFilename = [NSTemporaryDirectory() stringByAppendingPathComponent:message.mediaFilename ? @"file.dat" : message.mediaFilename];
NSOutputStream *outputStream = [NSOutputStream outputStreamToFileAtPath:tempFilename append:NO];

// Request the start of the download
[message getMediaWithOutputStream:outputStream
                        onStarted:^{
                            // Called when download of media begins.
                        } onProgress:^(NSUInteger bytes) {
                            // Called as download progresses, with the current byte count.
                        } onCompleted:^(NSString * _Nonnull mediaSid) {
                            // Called when download is completed, with the new mediaSid if successful.
                            // Full failure details will be provided through the completion block below.
                        } completion:^(TCHResult * _Nonnull result) {
                            if (!result.isSuccessful) {
                                NSLog(@"Download failed: %@", result.error);
                            } else {
                                NSLog(@"Download successful");
                            }
                        }]; 
// Set up output stream for media content
let tempFilename = (NSTemporaryDirectory() as NSString).appendingPathComponent(message.mediaFilename ?? "file.dat")
let outputStream = OutputStream(toFileAtPath: tempFilename, append: false)

// Request the start of the download
if let outputStream = outputStream {
    message.getMediaWith(outputStream,
                         onStarted: {
                            // Called when download of media begins.
    },
                         onProgress: { (bytes) in
                            // Called as download progresses, with the current byte count.
    },
                         onCompleted: { (mediaSid) in
                            // Called when download is completed, with the new mediaSid if successful.
                            // Full failure details will be provided through the completion block below.
    }) { (result) in
        if !result.isSuccessful() {
            print("Download failed: \(String(describing: result.error))")
        } else {
            print("Download successful")
        }
    }
}