Add Advanced Features to your CAF Receiver

This page contains code snippets and descriptions of the features available for customizing a CAF receiver app.

Creating a customized receiver app

The main structure of a customized CAF receiver app includes required elements (shown in bold) along with optional features to customize the app for your particular use case.

  1. A cast-media-player element that represents the built-in player UI provided with CAF.
  2. Custom CSS-like styling for the cast-media-player element to style various UI elements such as the background-image, splash-image, and font-family.
  3. A script element to load the Cast receiver framework.
  4. JavaScript code to customize receiver app by intercepting messages and handling events.
  5. Queue for autoplay.
  6. Options to configure playback.
  7. Options to set the receiver context.
  8. Options to set commands which are supported by the receiver app.
  9. A JavaScript call to start the receiver application.

Here is sample code for a CAF receiver application that illustrates this full structure.

Tip: Also see Loading media using contentId, contentUrl and entity.

<html>
<head>
</head>
<body>
  <cast-media-player id="player"></cast-media-player>
  <style>
    #player {
        --theme-hue: 210;
        --splash-image: url("my.png");
    }
  </style>
  <script type="text/javascript" src="//www.gstatic.com/cast/sdk/libs/caf_receiver/v3/cast_receiver_framework.js">
  </script>
  <script>
    const context = cast.framework.CastReceiverContext.getInstance();
    const playerManager = context.getPlayerManager();

    // intercept the LOAD request to be able to read in a contentId and get data
    playerManager.setMessageInterceptor(
        cast.framework.messages.MessageType.LOAD, loadRequestData => {
            if (loadRequestData.media && loadRequestData.media.contentId) {
                return thirdparty.getMediaById(loadRequestData.media.contentId)
                .then(media => {
                  if (media) {
                    loadRequestData.media.contentUrl = media.url;
                    loadRequestData.media.contentType = media.contentType;
                    loadRequestData.media.metadata = media.metadata;
                  }
                  return loadRequestData;
                });
            }
            return loadRequestData;
        });


    // listen to all Core Events
    playerManager.addEventListener(cast.framework.events.category.CORE,
        event => {
            console.log(event);
        });

    const MyCastQueue = class extends cast.framework.QueueBase {
        initialize(loadRequestData) {
            const media = loadRequestData.media;
            const items = [];
            items.push(myCreateItem(media)); // your custom function logic

            const queueData = new cast.framework.messages.QueueData();
            queueData.items = items;

            return queueData;
        }

        nextItems(itemId) {
           return [myCreateNextItem()]; // your custom function logic
        }
    };

    const playbackConfig = new cast.framework.PlaybackConfig();

    // Sets the player to start playback as soon as there are five seconds of
    // media contents buffered. Default is 10.
    playbackConfig.autoResumeDuration = 5;

    const myCastQueue = new MyCastQueue(); // create instance of queue Object

    context.start({
      queue: myCastQueue,
      playbackConfig: playbackConfig,
      supportedCommands: cast.framework.messages.Command.ALL_BASIC_MEDIA |
                         cast.framework.messages.Command.QUEUE_PREV |
                         cast.framework.messages.Command.QUEUE_NEXT
    });
  </script>
</body>
</html>

Application configuration and options

The CastReceiverContext is the outermost class exposed to the developer, and it manages loading of underlying libraries and handles the initialization of the receiver SDK.

If the CAF Receiver API detects that a sender is disconnected it will raise the SENDER_DISCONNECTEDevent. If the receiver has not been able to communicate with the sender for what we described as maxInactivity seconds, it will also raise the SENDER_DISCONNECTED event. During development it is a good idea to set maxInactivity to a high value so that the receiver app does not close when debugging the app with the Chrome Remote Debugger:

const context = cast.framework.CastReceiverContext.getInstance();
const options = new cast.framework.CastReceiverOptions();
options.maxInactivity = 3600; //Development only
context.start(options);

However, for a published receiver application it is better to not set maxInactivity and instead rely on the default value. Note that the Cast receiver options are set only once in the application.

The other configuration is the cast.framework.PlaybackConfig. This can be set as follows:

const playbackConfig = new cast.framework.PlaybackConfig();
playbackConfig.manifestRequestHandler = requestInfo => {
  requestInfo.withCredentials = true;
};
context.start({playbackConfig: playbackConfig});

This configuration affects each content playback and essentially provides override behavior. For a list of behaviors that developers can override, see the definition of cast.framework.PlaybackConfig. To change the configuration in between contents, one can use the PlayerManager to get its current playbackConfig, modify or add an override and reset the playbackConfig like this:

const playerManager =
    cast.framework.CastReceiverContext.getInstance().getPlayerManager();
const playbackConfig = (Object.assign(
            new cast.framework.PlaybackConfig(), playerManager.getPlaybackConfig()));
playbackConfig.autoResumeNumberOfSegments = 1;
playerManager.setPlaybackConfig(playbackConfig);

Note that if PlaybackConfig has never been overridden, the getPlaybackConfig() returns a null object. And any property on PlaybackConfig that has undefined as its value will default its behavior and implementations to CAF.

Event handling

The CAF Receiver SDK allows your receiver app to handle player events. The event handler takes a cast.framework.events.EventType parameter (or an array of these parameters) that specifies the event(s) that should trigger the listener. Preconfigured arrays of cast.framework.events.EventType that are useful for debugging can be found in cast.framework.events.category. The event parameter provides additional information about the event.

For example, if you want to know when a mediaStatus change is being broadcasted, you can use the following logic to handle the event:

const playerManager =
    cast.framework.CastReceiverContext.getInstance().getPlayerManager();
playerManager.addEventListener(
    cast.framework.events.EventType.MEDIA_STATUS, (event) => {
      // Write your own event handling code, for example
      // using the event.mediaStatus value
});

Note: The receiver framework automatically tracks when a sender connects or disconnects from it and doesn't require an explicit SENDER_DISCONNECTED event handler in your own receiver logic (as in Receiver v2).

Event errors

For detailed information on errors thrown by cast.framework.events.EventType.ERROR, see Media Player Library Error Codes.

Message interception

CAF Receiver SDK allows your receiver app to intercept messages and execute custom code on those messages. The message interceptor takes a cast.framework.messages.MessageType parameter that specifies what type of message should be intercepted.

Note: The interceptor should return the modified request or a Promise that resolves with the modified request value. Returning null will prevent calling the default message handler.

For example, if you want to change the load request data, you can use the following logic to intercept and modify it.

Tip: Also see Loading media using contentId, contentUrl and entity.

const context = cast.framework.CastReceiverContext.getInstance();
const player = context.getPlayerManager();

player.setMessageInterceptor(
    cast.framework.messages.MessageType.LOAD,
    request => {
      return new Promise((resolve, reject) => {
        // Write your own custom code if you would like to see the stream info
        const onSuccess = function(streamInfo) {
          const mediaUrl = streamInfo.url;
          const captionUrl = streamInfo.captionUrl;

          request.media.contentUrl = mediaUrl;
          request.media.contentType = 'application/dash+xml';
          request.media.tracks = [{
            trackId: 1,
            trackContentId: captionUrl,
            trackContentType: 'text/vtt',
            type: cast.framework.messages.TrackType.TEXT
          }];
          resolve(request);
        };
        const onFailure = function() {
          reject(new cast.framework.messages.ErrorData(
              cast.framework.messages.ErrorType.INVALID_REQUEST));
        };
      fetchData(request.media.contentId, onSuccess, onFailure)
    });

player.setMessageInterceptor(
    cast.framework.messages.MessageType.MEDIA_STATUS,
    status => {
      status.customData = {};
      return status;
    });

context.start();

Message interception vs event handling

Some key differences between message interception and event handling are as follows:

  • Event handling does not allow you to modify the request data.
  • Event handling is best used to trigger analytics or a custom function.

Note: Event handling can also be used to listen to an umbrella of events using the CORE, DEBUG, or FINE enum.

playerManager.addEventListener(cast.framework.events.category.CORE,
    event => {
        console.log(event);
    });
  • Message interception allows you to listen to a message, intercept it, and modify the request data itself.
  • Message interception is best used to handle custom logic with regards to request data.

Custom messages

Message exchange is the key interaction method for receiver applications.

A sender issues messages to a receiver using the sender APIs for the platform the sender is running (Android, iOS, Chrome). The event object (which is the manifestation of a message) that is passed to the event listeners has a data element (event.data) where the data takes on the properties of the specific event type.

A receiver application may choose to listen for messages on a specified namespace. By virtue of doing so, the receiver application is said to support that namespace protocol. It is then up to any connected senders wishing to communicate on that namespace to use the appropriate protocol.

All namespaces are defined by a string and must begin with "urn:x-cast:" followed by any string. For example, "urn:x-cast:com.example.cast.mynamespace".

Note: The media message namespace "urn:x-cast:com.google.cast.media" is reserved. Media messages sent using the Cast APIs on both the sender and receiver use the media namespace protocol by convention. See Media and players for more about media messages.

Here is a code snippet for the receiver to listen to custom messages from connected senders:

const context = cast.framework.CastReceiverContext.getInstance();

const CUSTOM_CHANNEL = 'urn:x-cast:com.example.cast.mynamespace';
context.addCustomMessageListener(CUSTOM_CHANNEL, function(customEvent) {
  // handle customEvent.
});

context.start();

Similarly, receiver applications can keep senders informed about the state of the receiver by sending messages to connected senders. A receiver application can send messages using sendCustomMessage(namespace, senderId, message) on CastReceiverContext. A receiver can send messages to an individual sender, either in response to a received message or due to an application state change. Beyond point-to-point messaging (with a limit of 64kb), a receiver may also broadcast messages to all connected senders.

Device capabilities

The getDeviceCapabilities method provides device information on the connected Cast device and the video or audio device attached to it. It can provide information about support for Google Assistant, Bluetooth, and the connected display and audio devices.

This method returns an object which you can query by passing in one of the specified enums to get the device capability for that enum. The enums are defined in cast.framework.system.DeviceCapabilities.

This example checks if the receiver device is capable of playing HDR and DolbyVision (DV) with the IS_HDR_SUPPORTED and IS_DV_SUPPORTED keys, respectively.

const context = cast.framework.CastReceiverContext.getInstance();
context.addEventListener(cast.framework.system.EventType.READY, () => {
  const deviceCapabilities = context.getDeviceCapabilities();
  if (deviceCapabilities &&
      deviceCapabilities[cast.framework.system.DeviceCapabilities.IS_HDR_SUPPORTED]) {
    // Write your own event handling code, for example
    // using the deviceCapabilities[cast.framework.system.DeviceCapabilities.IS_HDR_SUPPORTED] value
  }
  if (deviceCapabilities &&
      deviceCapabilities[cast.framework.system.DeviceCapabilities.IS_DV_SUPPORTED]) {
    // Write your own event handling code, for example
    // using the deviceCapabilities[cast.framework.system.DeviceCapabilities.IS_DV_SUPPORTED] value
  }
});
context.start();

Preserving Session state

The CAF Receiver SDK provides a default implementation for receiver apps to preserve session states by taking a snapshot of current media status, converting the status into a load request, and resuming the session with the load request.

The load request generated by the receiver can be overridden in the SESSION_STATE message interceptor if necessary. If you want to add custom data into the load request, we suggest putting them in loadRequestData.customData.

playerManager.setMessageInterceptor(
    cast.framework.messages.MessageType.SESSION_STATE,
    function (sessionState) {
        // Override sessionState.loadRequestData if needed.
        const newCredentials = updateCredentials_(sessionState.loadRequestData.credentials);
        sessionState.loadRequestData.credentials = newCredentials;

        // Add custom data if needed.
        sessionState.loadRequestData.customData = {
            'membership': 'PREMIUM'
        };

        return sessionState;
    });

The custom data can be retrieved from loadRequestData.customData in the RESUME_SESSION message interceptor.

let cred_ = null;
let membership_ = null;

playerManager.setMessageInterceptor(
    cast.framework.messages.MessageType.RESUME_SESSION,
    function (resumeSessionRequest) {
        let sessionState = resumeSessionRequest.sessionState;

        // Modify sessionState.loadRequestData if needed.
        cred_ = sessionState.loadRequestData.credentials;

        // Retrieve custom data.
        membership_ = sessionState.loadRequestData.customData.membership;

        return resumeSessionRequest;
    });

Check display type

The canDisplayType method checks for video and audio capabilities of the receiver device and display by validating the media parameters passed in, returning a boolean. All parameters but the first are optional — the more parameters you include, the more precise the check will be.

Its signature is canDisplayType(mimeType,codecs,width,height,framerate)

Examples:

Checks whether the receiver device and display support the video/mp4 mimetype with this particular codec, dimensions, and framerate:

canDisplayType("video/mp4", "avc1.42e015,mp4a.40.5", 1920, 1080, 30)

Checks whether the receiver device and display support 4K video format for this codec by specifying the width of 3840 and height of 2160:

canDisplayType("video/mp4", "hev1.1.2.L150", 3840, 2160)

Checks whether the receiver device and display support HDR10 for this codec, dimensions, and framerate:

canDisplayType("video/mp4", "hev1.2.6.L150", 3840, 2160, 30)

Checks whether the receiver device and display support Dolby Vision (DV) for this codec, dimensions, and framerate:

canDisplayType("video/mp4", "dvhe.04.06", 1920, 1080, 30)

Custom UI data binding

If you want to use your own custom UI element instead of cast-media-player, you can do that; instead of adding cast-media-player element to your HTML, instead use the PlayerDataBinder class to bind the UI to the player state. The binder also supports sending events for data changes, if the app does not support data binding.

const context = cast.framework.CastReceiverContext.getInstance();
const player = context.getPlayerManager();

const playerData = {};
const playerDataBinder = new cast.framework.ui.PlayerDataBinder(playerData);

// Update ui according to player state
playerDataBinder.addEventListener(
    cast.framework.ui.PlayerDataEventType.STATE_CHANGED,
    e => {
      switch (e.value) {
        case cast.framework.ui.State.LAUNCHING:
        case cast.framework.ui.State.IDLE:
          // Write your own event handling code
          break;
        case cast.framework.ui.State.LOADING:
          // Write your own event handling code
          break;
        case cast.framework.ui.State.BUFFERING:
          // Write your own event handling code
          break;
        case cast.framework.ui.State.PAUSED:
          // Write your own event handling code
          break;
        case cast.framework.ui.State.PLAYING:
          // Write your own event handling code
          break;
      }
    });
context.start();

You should add at least one MediaElement to the HTML so that the receiver can use it. If multiple MediaElement objects are available, you should tag the MediaElement that you want the receiver to use. You do this by adding castMediaElement in the video's class list, as shown below; otherwise, the receiver will choose the first MediaElement.

<video class="castMediaElement"></video>

Note, CAF Receiver currently discourages passing a custom MediaElement as you lose many benefits of using CAF Receiver SDK.

Content preload

Cast Application Framework supports preloading of media items after the current playback item in the queue. The preload operation pre-downloads several segments of the upcoming items. The specification is done on the preloadTime value in the QueueItem object (default to 20 seconds if not provided). The time is expressed in seconds, relative to the end of the currently playing item . Only positive values are valid. For example, if the value is 10 seconds, this item will be preloaded 10 seconds before the previous item has finished. If the time to preload is higher than the time left on the currentItem, the preload will just happen as soon as possible. So if a very large value of preload is specified on the queueItem, one could achieve the effect of whenever we are playing the current item we are already preloading the next item. However, we leave the setting and choice of this to developer as this value can affect bandwidth and streaming performance of the current playing item.

Note that preloading will work for HLS and Smooth streaming content by default. For DASH content, preloading works if useLegacyDashSupport is specified in CastReceiverOptions, since Media Player Library (MPL) supports preload while Shaka does not yet. For regular MP4 video and audio files such as MP3, those will not be preloaded, as Cast devices support one media element only and cannot be used to preload while an existing content item is still playing.

DRM

Note: One of the key benefits of using CAF Receiver SDK is that your app no longer needs to load MPL and handle media playback separately, as CAF Receiver SDK handles that for you.

Some media content requires Digital Rights Management (DRM). For media content that has its DRM license (and key URL) stored in their manifest (DASH or HLS), CAF handles this case for you. A subset of that content requires a licenseUrl which is needed to obtain the decryption key. In CAF, you can use PlaybackConfig to set the licenseUrl as needed.

The following code snippet shows how you can set request information for license requests such as withCredentials:

const context = cast.framework.CastReceiverContext.getInstance();
const playbackConfig = new cast.framework.PlaybackConfig();
// Customize the license url for playback
playbackConfig.licenseUrl = 'http://widevine/yourLicenseServer';
playbackConfig.licenseRequestHandler = requestInfo => {
  requestInfo.withCredentials = true;
};
context.start({playbackConfig: playbackConfig});

// Update playback config licenseUrl according to provided value in load request.
context.getPlayerManager().setMediaPlaybackInfoHandler((loadRequest, playbackConfig) => {
  if (loadRequest.media.customData && loadRequest.media.customData.licenseUrl) {
    playbackConfig.licenseUrl = loadRequest.media.customData.licenseUrl;
  }
  return playbackConfig;
});

If you have a Google Assistant integration, some of the DRM information such as the credentials necessary for the content might be linked directly to your Google account through mechanisms such as OAuth/SSO. In those cases, if the media content is loaded through voice or comes from the cloud, a setCredentials is invoked from the cloud to the Cast device providing that credentials. Applications writing a receiver app with CAF can then use the setCredentials information to operate DRM as necessary. Here is an example of using the credential to construct the media.

Tip: Also see Loading media using contentId, contentUrl and entity.

playerManager.setMessageInterceptor(
    cast.framework.messages.MessageType.LOAD, loadRequestData => {
      if (loadRequestData.media && loadRequestData.media.entity) {
        return thirdparty
            .getMediaById(
                loadRequestData.media.entity, loadRequestData.credentials)
            .then(media => {
              if (media) {
                loadRequestData.media.contentId = media.url;
                loadRequestData.media.contentType = media.contentType;
                loadRequestData.media.metadata = media.metadata;
              }
              return loadRequestData;
            });
      }
      return loadRequestData;
    });

Cast for audio devices

See Google Cast for audio devices guide for support on audio only playback.

Tip: Loading media using contentId, contentUrl and entity

We sugget to use entity in your implementation. The entity property is a deep link URL which can be used by Google Assistant. The entity value should be converted to contentUrl or contentId by the load interceptor as such:

const context = cast.framework.CastReceiverContext.getInstance();
const playerManager = context.getPlayerManager();

// Support load by entity by intercepting load request, and get media
// information by entity and credentials.
playerManager.setMessageInterceptor(
    cast.framework.messages.MessageType.LOAD, loadRequestData => {
      if (loadRequestData.media && loadRequestData.media.entity) {
        return thirdparty
            .getMediaById(
                loadRequestData.media.entity, loadRequestData.credentials)
            .then(media => {
              if (media) {
                loadRequestData.media.contentUrl = media.url;
                loadRequestData.media.contentType = media.contentType;
                loadRequestData.media.metadata = media.metadata;
              }
              return loadRequestData;
            });
      }
      return loadRequestData;
    });
context.start();

The contentId property is typically the URL of the media, and can be used as a real ID or key parameter for custom lookup. If contentId is used as a real ID (and is not the URL), the optional parameter contentUrl will be the URL of the media. In other words, contentId must either be used as a key/real ID with the optional contentUrl included OR contentId must be the URL itself.

The following is a snippet that shows how to fetch media by contentId (if it's not a URL) and use contentUrl for the media URL.

const context = cast.framework.CastReceiverContext.getInstance();
const playerManager = context.getPlayerManager();

// Support Load by contentId. It fetches media data by contentId and
// uses the contentUrl for the media URL.
playerManager.setMessageInterceptor(
    cast.framework.messages.MessageType.LOAD, loadRequestData => {
      if (loadRequestData.media && loadRequestData.media.contentId) {
        return thirdparty.getMediaById(loadRequestData.media.contentId)
            .then(media => {
              if (media) {
                loadRequestData.media.contentUrl = media.url;
                loadRequestData.media.contentId = media.id;
                loadRequestData.media.contentType = media.contentType;
                loadRequestData.media.metadata = media.metadata;
              }
              return loadRequestData;
            });
      }
      return loadRequestData;
    });
context.start();

Next Steps

This concludes the features that you can add to your CAF Receiver. You can now build a sender app on iOS, Android, or Chrome.