Sounds and Audio in Flex

To receive ring notifications for incoming calls you can create a plugin to listen for the reservationCreated event, then playback audio.

For example, the following plugin code repeats an audio file (in this case, https://example.com/sounds/alert.mp3), until an agent accepts the task and answers the call:

1
let alertSound = new Audio("https://example.com/sounds/alert.mp3");
2
alertSound.loop = true;
3

4
const resStatus = ["accepted","canceled","rejected","rescinded","timeout"];
5

6
manager.workerClient.on("reservationCreated", function(reservation) {
7
  if (
8
     reservation.task.taskChannelUniqueName === "voice" &&
9
     reservation.task.attributes.direction === "inbound"
10
  ) {
11
    alertSound.play();
12
  }
13
  resStatus.forEach((e) => {
14
    reservation.on(e, () => {
15
      alertSound.pause()
16
    });
17
  });
18
});

Be sure to add an audio file URL. You can't use local files from your desktop as the audio file must be accessible by Twilio's proxy servers.

After you've added audio, add the code above to the init() function of your plugin. Visit the Flex UI Overview to learn more about building with plugins.

With the MediaDevices interface, you can request a list of available audio output devices with enumerateDevices():

navigator.mediaDevices.enumerateDevices();

Identify the desired device ID, then pass it off using the HTMLMediaElement.setSinkId() method.

The Flex AudioPlayerManager API supports playing, stopping, and muting sounds.

Sounds are divided into 2 categories:

• Repeatable media are played in a loop, like a phone ringing that goes on and on. The only way to stop repeatable media is to manually call the stop method.
• Non-repeatable media are played once, like a beep or bleep. Non-repeatable media automatically stop after being played once.

The media to be played is defined like so:

1
export interface MediaData {
2
    url: string;
3
    repeatable: boolean;
4
}

To play media, you must specify the URL where the media file is located, and declare whether or not the sound is repeatable:

1
const mediaId = Flex.AudioPlayerManager.play(
2
{
3
    url: "sound-url",
4
    repeatable: true
5
},
6
(error: Flex.AudioPlayerError) => {
7
    // handle error
8
}
9
);
10

If there is an error while playing the media, a notification will be shown with the error. Some possible errors include:

To stop media, use the media id returned from the play method (described above):

Flex.AudioPlayerManager.stop(mediaId);

To mute or unmute sounds, use the following method:

Flex.AudioPlayerManager.toggleSound(true/false);

Muting or unmuting doesn't stop current playing media. The media will keep playing, but with their volume at zero.

Repeatable and non-repeatable media can be played at the same time, but there can only be one repeatable and one non-repeatable media playing at a given time. Overall, this means that you can only have two media playing at once: one repeatable, and one non-repeatable media.

If the media is repeatable, calls to Flex.AudioPlayerManager.play will enqueue the media and play them one after another. You can stop a repeatable media that isn't playing (but is enqueued) to remove it from the queue. Stopping a repeatable media that is playing will stop the sound and play the next media in the queue (if the queue is not empty).

If the media is non-repeatable, it will be played only if no other non-repeatable media is already playing. Non-repeatable media are not enqueued. You can stop a non-repeatable media that is playing to stop the sound.

For example, to play a sound when a chat message is received:

1
Flex.Manager.getInstance().chatClient.on("messageAdded", () => {
2
    const mediaId = Flex.AudioPlayerManager.play({
3
        url: "sound-url",
4
        repeatable: true
5
    });
6
})

Browsers, especially Chrome, have strict policies that prevent sound playback or media autoplay for background tabs or tabs that didn't receive any user input (source: https://developers.google.com/web/updates/2017/09/autoplay-policy-changes). Some typical situations include:

• A tab that was refreshed and never received user input
• A tab with Flex in a separate window that was reopened automatically and never received user input
• A tab that was opened a while ago and hasn't received focus for a while
• A tab among other tabs that playback sounds

For media to be played, the agent must interact with the Flex UI first. Sounds that were triggered but weren't allowed to play, will start playing as soon as the agent interacts with the site.

• Repeatable media will play the first media that was triggered. Once the first one stops playing, the next in queue will be played.
• Non-repeatable media will play the first media that was triggered and hasn't been stopped.

Expand image

There are some workarounds:

• Allow the hostname: add the hostname at chrome://settings/content/sound to allow it to play sound. Note: this does not require enterprise policies.
• Change enterprise policies: add the hostname to the list of allowed hostnames. Note: this requires enterprise policies.

Allow hostname: add the hostname in the audio settings list about:preferences#privacy and allow it to always play sound.

Allow autoplay sound for hostname: in the Safari app, choose Safari > Settings for This Website, under Auto-Play select Allow All Auto-Play