Add Core Features to Your CAF Receiver

This page contains code snippets and descriptions of the features available for customizing a CAF receiver app.

Creating a customized receiver app

The main structure of a customized CAF receiver app includes required elements (shown in bold) along with optional features to customize the app for your particular use case.

  1. A cast-media-player element that represents the built-in player UI provided with CAF.
  2. Custom CSS-like styling for the cast-media-player element to style various UI elements such as the background-image, splash-image, and font-family.
  3. A script element to load the Cast receiver framework.
  4. JavaScript code to customize receiver app by intercepting messages and handling events.
  5. Queue for autoplay.
  6. Options to configure playback.
  7. Options to set the receiver context.
  8. Options to set commands which are supported by the receiver app.
  9. A JavaScript call to start the receiver application.

See the CastReceiver GitHub sample for a CAF receiver application that illustrates this full structure.

Application configuration and options

The CastReceiverContext is the outermost class exposed to the developer, and it manages loading of underlying libraries and handles the initialization of the receiver SDK.

If the CAF Receiver API detects that a sender is disconnected it will raise the SENDER_DISCONNECTEDevent. If the receiver has not been able to communicate with the sender for what we described as maxInactivity seconds, it will also raise the SENDER_DISCONNECTED event. During development it is a good idea to set maxInactivity to a high value so that the receiver app does not close when debugging the app with the Chrome Remote Debugger:

const context = cast.framework.CastReceiverContext.getInstance();
const options = new cast.framework.CastReceiverOptions();
options.maxInactivity = 3600; //Development only
context.start(options);

However, for a published receiver application it is better to not set maxInactivity and instead rely on the default value. Note that the Cast receiver options are set only once in the application.

The other configuration is the cast.framework.PlaybackConfig. This can be set as follows:

const playbackConfig = new cast.framework.PlaybackConfig();
playbackConfig.manifestRequestHandler = requestInfo => {
  requestInfo.withCredentials = true;
};
context.start({playbackConfig: playbackConfig});

This configuration affects each content playback and essentially provides override behavior. For a list of behaviors that developers can override, see the definition of cast.framework.PlaybackConfig. To change the configuration in between contents, one can use the PlayerManager to get its current playbackConfig, modify or add an override and reset the playbackConfig like this:

const playerManager =
    cast.framework.CastReceiverContext.getInstance().getPlayerManager();
const playbackConfig = (Object.assign(
            new cast.framework.PlaybackConfig(), playerManager.getPlaybackConfig()));
playbackConfig.autoResumeNumberOfSegments = 1;
playerManager.setPlaybackConfig(playbackConfig);

Note that if PlaybackConfig has never been overridden, the getPlaybackConfig() returns a null object. And any property on PlaybackConfig that has undefined as its value will default its behavior and implementations to CAF.

Event listener

The CAF Receiver SDK allows your receiver app to handle player events. The event listener takes a cast.framework.events.EventType parameter (or an array of these parameters) that specifies the event(s) that should trigger the listener. Preconfigured arrays of cast.framework.events.EventType that are useful for debugging can be found in cast.framework.events.category. The event parameter provides additional information about the event.

For example, if you want to know when a mediaStatus change is being broadcasted, you can use the following logic to handle the event:

const playerManager =
    cast.framework.CastReceiverContext.getInstance().getPlayerManager();
playerManager.addEventListener(
    cast.framework.events.EventType.MEDIA_STATUS, (event) => {
      // Write your own event handling code, for example
      // using the event.mediaStatus value
});

Note: The receiver framework automatically tracks when a sender connects or disconnects from it and doesn't require an explicit SENDER_DISCONNECTED event listener in your own receiver logic (as in Receiver v2).

Message interception

CAF Receiver SDK allows your receiver app to intercept messages and execute custom code on those messages. The message interceptor takes a cast.framework.messages.MessageType parameter that specifies what type of message should be intercepted.

Note: The interceptor should return the modified request or a Promise that resolves with the modified request value. Returning null will prevent calling the default message handler.

For example, if you want to change the load request data, you can use the following logic to intercept and modify it.

Tip: Also see Loading media using contentId, contentUrl and entity.

const context = cast.framework.CastReceiverContext.getInstance();
const playerManager = context.getPlayerManager();

playerManager.setMessageInterceptor(
    cast.framework.messages.MessageType.LOAD, loadRequestData => {
      const error = new cast.framework.messages.ErrorData(
                      cast.framework.messages.ErrorType.LOAD_FAILED);
      if (!loadRequestData.media) {
        error.reason = cast.framework.messages.ErrorReason.INVALID_PARAM;
        return error;
      }

      if (!loadRequestData.media.entity) {
        return loadRequestData;
      }

      return thirdparty.fetchAssetAndAuth(loadRequestData.media.entity,
                                          loadRequestData.credentials)
        .then(asset => {
          if (!asset) {
            throw cast.framework.messages.ErrorReason.INVALID_REQUEST;
          }

          loadRequestData.media.contentUrl = asset.url;
          loadRequestData.media.metadata = asset.metadata;
          loadRequestData.media.tracks = asset.tracks;
          return loadRequestData;
        }).catch(reason => {
          error.reason = reason; // cast.framework.messages.ErrorReason
          return error;
        });
    });

context.start();

Error handling

When errors happen in message interceptor, your receiver app should return an appropriate cast.framework.messages.ErrorType and cast.framework.messages.ErrorReason.

playerManager.setMessageInterceptor(
    cast.framework.messages.MessageType.LOAD, loadRequestData => {
      const error = new cast.framework.messages.ErrorData(
                      cast.framework.messages.ErrorType.LOAD_CANCELLED);
      if (!loadRequestData.media) {
        error.reason = cast.framework.messages.ErrorReason.INVALID_PARAM;
        return error;
      }

      ...

      return fetchAssetAndAuth(loadRequestData.media.entity,
                               loadRequestData.credentials)
        .then(asset => {
          ...
          return loadRequestData;
        }).catch(reason => {
          error.reason = reason; // cast.framework.messages.ErrorReason
          return error;
        });
    });

Message interception vs event listener

Some key differences between message interception and event listener are as follows:

  • An event listener does not allow you to modify the request data.
  • An event listener is best used to trigger analytics or a custom function.

Note: An event listener can also be used to listen to an umbrella of events using the CORE, DEBUG, or FINE enum.

playerManager.addEventListener(cast.framework.events.category.CORE,
    event => {
        console.log(event);
    });
  • Message interception allows you to listen to a message, intercept it, and modify the request data itself.
  • Message interception is best used to handle custom logic with regards to request data.

Tip: Loading media using contentId, contentUrl and entity

We sugget you use entity in your implementation. both for your sender and receiver apps. The entity property is a deep link URL, which can be either a playlist or a specific media content.

The contentUrl is designed for a playable URL.

The contentId has been deprecated. It is typically the URL of the media, and can be used as a real ID or key parameter for custom lookup. We suggest you to use entity to store the real ID or key parameters, and use contentUrl for the URL of the media.

playerManager.setMessageInterceptor(
    cast.framework.messages.MessageType.LOAD, loadRequestData => {
      ...

      if (!loadRequestData.media.entity) {
        // Copy the value from contentId for legacy reasons if needed
        loadRequestData.media.entity = loadRequestData.media.contentId;
      }

      return thirdparty.fetchAssetAndAuth(loadRequestData.media.entity,
                                          loadRequestData.credentials)
        .then(asset => {
          loadRequestData.media.contentUrl = asset.url;
          ...
          return loadRequestData;
        });
    });

Device capabilities

The getDeviceCapabilities method provides device information on the connected Cast device and the video or audio device attached to it. It can provide information about support for Google Assistant, Bluetooth, and the connected display and audio devices.

This method returns an object which you can query by passing in one of the specified enums to get the device capability for that enum. The enums are defined in cast.framework.system.DeviceCapabilities.

This example checks if the receiver device is capable of playing HDR and DolbyVision (DV) with the IS_HDR_SUPPORTED and IS_DV_SUPPORTED keys, respectively.

const context = cast.framework.CastReceiverContext.getInstance();
context.addEventListener(cast.framework.system.EventType.READY, () => {
  const deviceCapabilities = context.getDeviceCapabilities();
  if (deviceCapabilities &&
      deviceCapabilities[cast.framework.system.DeviceCapabilities.IS_HDR_SUPPORTED]) {
    // Write your own event handling code, for example
    // using the deviceCapabilities[cast.framework.system.DeviceCapabilities.IS_HDR_SUPPORTED] value
  }
  if (deviceCapabilities &&
      deviceCapabilities[cast.framework.system.DeviceCapabilities.IS_DV_SUPPORTED]) {
    // Write your own event handling code, for example
    // using the deviceCapabilities[cast.framework.system.DeviceCapabilities.IS_DV_SUPPORTED] value
  }
});
context.start();

Voice commands

The following media commands are currently supported in the CAF Receiver SDK for Assistant-enabled devices. The default implementations of these commands are found in cast.framework.PlayerManager.

Command Description
Play Play or resume playback from paused state.
Pause Pause currently playing content.
Previous Skip to the previous media item in your media queue.
Next Skip to the next media item in your media queue.
Stop Stop the currently playing media.
Repeat None Disable repeating of media items in the queue once the last item in the queue is done playing.
Repeat Single Repeat the currently playing media indefinitely.
Repeat All Repeat all items in the queue once the last item in the queue is played.
Repeat All and Shuffle Once the last item in the queue is done playing, shuffle the queue and repeat all items in the queue.
Shuffle Shuffle media items in your media queue.
Closed Captions ON / OFF Enable / Disable Closed Captioning for your media. Enable / Disable is also available by language.
Seek to absolute time Jumps to the specified absolute time.
Seek to time relative to current time Jumps forward or backward by the specified time period relative to the current playback time.
Play Again Restart the currently playing media or play the last played media item if nothing is currently playing.
Set playback rate Vary media playback rate. This should be handled by default in the CAF SDK. You can use the SET_PLAYBACK_RATE message interceptor to override incoming rate requests.

If your application prohibits execution of a command by enforcing the behavior on the sender, you must override the corresponding command in the receiver to ensure the command is not triggered by an Assistant-enabled device.

For example, if you disallow SEEK for your media from your sender applications, you must also intercept the SEEK request provided by the SDK so that SEEK commands issued to Assistant-enabled devices do not trigger a seek in your receiver application.

For commands your application does not support, return an appropriate error reason, such as NOT_SUPPORTED.

Ducking from voice activity

If the Cast platform ducks your application's sound due to Assistant activity such as listening to user speech or talking back, a FocusState message of NOT_IN_FOCUS is sent to the receiver application when the activity starts. Another message with IN_FOCUS is sent when the activity ends. Depending on your application and the media being played, you might want to pause media when the FocusState is NOT_IN_FOCUS.

For example, it's a good user experience to pause audiobook playback if the Assistant is responding to a user query.

Voice-specified caption language

When a user does not explicitly state the language for the captions, the language used for captions is the same language in which the command was spoken. In these scenarios, the isSuggestedLanguage parameter of the incoming message indicates whether the associated language was suggested or explicitly requested by user.

For example, isSuggestedLanguage is set to true for the command "OK Google, turn captions on," because the language was inferred by the language the command was spoken in. If the language is explicitly requested, such as in "OK Google, turn on English captions," isSuggestedLanguage is set to false.

Metadata and voice casting

While voice commands are handled by the CAF Receiver by default, you should ensure the metadata for your content is complete and accurate. This ensures that voice commands are handled properly by the Assistant and that the metadata surfaces properly across new types of interfaces such as the Google Home app and smart displays like the Google Home Hub.

Preserving session state

The CAF Receiver SDK provides a default implementation for receiver apps to preserve session states by taking a snapshot of current media status, converting the status into a load request, and resuming the session with the load request.

The load request generated by the receiver can be overridden in the SESSION_STATE message interceptor if necessary. If you want to add custom data into the load request, we suggest putting them in loadRequestData.customData.

playerManager.setMessageInterceptor(
    cast.framework.messages.MessageType.SESSION_STATE,
    function (sessionState) {
        // Override sessionState.loadRequestData if needed.
        const newCredentials = updateCredentials_(sessionState.loadRequestData.credentials);
        sessionState.loadRequestData.credentials = newCredentials;

        // Add custom data if needed.
        sessionState.loadRequestData.customData = {
            'membership': 'PREMIUM'
        };

        return sessionState;
    });

The custom data can be retrieved from loadRequestData.customData in the RESUME_SESSION message interceptor.

let cred_ = null;
let membership_ = null;

playerManager.setMessageInterceptor(
    cast.framework.messages.MessageType.RESUME_SESSION,
    function (resumeSessionRequest) {
        let sessionState = resumeSessionRequest.sessionState;

        // Modify sessionState.loadRequestData if needed.
        cred_ = sessionState.loadRequestData.credentials;

        // Retrieve custom data.
        membership_ = sessionState.loadRequestData.customData.membership;

        return resumeSessionRequest;
    });

Stream transfer

Preserving session state is the basis of stream transfer, a CAF feature where users can move existing audio and video streams across devices using voice commands, the Home App, or smart displays. Media stops playing on one device (the source) and continues on another (the destination). Any Cast device with the appropriate firmware can serve as sources or destinations in a stream transfer.

The event flow for stream transfer is:

  1. On the source device:
    1. Media stops playing.
    2. The receiver application receives a command to save the current media state.
    3. The receiver application is shut down.
  2. On the destination device:
    1. The receiver application is loaded.
    2. The receiver application receives a command to restore the saved media state.
    3. Media resumes playing.

Elements of media state include:

  • Specific position or timestamp of the song, video, or media item.
  • Its place in a broader queue (such as a playlist or artist radio).
  • The authenticated user.
  • Playback state (for example, playing or paused).

Audio support

Stream transfer for audio is supported by default and no additional development is necessary.

Video support

Stream transfer for video is optional. To implement video support for your CAF Receiver:

  1. Update supportedMediaCommands with the STREAM_TRANSFER command:
    playerManager.addSupportedMediaCommands(
    cast.framework.messages.Command.STREAM_TRANSFER, true);
  2. Optionally override the SESSION_STATE and RESUME_SESSION message interceptors as described in Preserving session state. Only override these if custom data needs to be stored as part of the session snapshot. Otherwise, the default implementation for preserving session states will support stream transfer.

Custom UI data binding

If you want to use your own custom UI element instead of cast-media-player, you can do that; instead of adding cast-media-player element to your HTML, instead use the PlayerDataBinder class to bind the UI to the player state. The binder also supports sending events for data changes, if the app does not support data binding.

const context = cast.framework.CastReceiverContext.getInstance();
const player = context.getPlayerManager();

const playerData = {};
const playerDataBinder = new cast.framework.ui.PlayerDataBinder(playerData);

// Update ui according to player state
playerDataBinder.addEventListener(
    cast.framework.ui.PlayerDataEventType.STATE_CHANGED,
    e => {
      switch (e.value) {
        case cast.framework.ui.State.LAUNCHING:
        case cast.framework.ui.State.IDLE:
          // Write your own event handling code
          break;
        case cast.framework.ui.State.LOADING:
          // Write your own event handling code
          break;
        case cast.framework.ui.State.BUFFERING:
          // Write your own event handling code
          break;
        case cast.framework.ui.State.PAUSED:
          // Write your own event handling code
          break;
        case cast.framework.ui.State.PLAYING:
          // Write your own event handling code
          break;
      }
    });
context.start();

You should add at least one MediaElement to the HTML so that the receiver can use it. If multiple MediaElement objects are available, you should tag the MediaElement that you want the receiver to use. You do this by adding castMediaElement in the video's class list, as shown below; otherwise, the receiver will choose the first MediaElement.

<video class="castMediaElement"></video>

Note, CAF Receiver currently discourages passing a custom MediaElement as you lose many benefits of using CAF Receiver SDK.

Content preload

Cast Application Framework supports preloading of media items after the current playback item in the queue. The preload operation pre-downloads several segments of the upcoming items. The specification is done on the preloadTime value in the QueueItem object (default to 20 seconds if not provided). The time is expressed in seconds, relative to the end of the currently playing item . Only positive values are valid. For example, if the value is 10 seconds, this item will be preloaded 10 seconds before the previous item has finished. If the time to preload is higher than the time left on the currentItem, the preload will just happen as soon as possible. So if a very large value of preload is specified on the queueItem, one could achieve the effect of whenever we are playing the current item we are already preloading the next item. However, we leave the setting and choice of this to developer as this value can affect bandwidth and streaming performance of the current playing item.

Note that preloading will work for HLS and Smooth streaming content by default. For DASH content, preloading works if useLegacyDashSupport is specified in CastReceiverOptions, since Media Player Library (MPL) supports preload while Shaka does not yet. For regular MP4 video and audio files such as MP3, those will not be preloaded, as Cast devices support one media element only and cannot be used to preload while an existing content item is still playing.

Custom messages

Message exchange is the key interaction method for receiver applications.

A sender issues messages to a receiver using the sender APIs for the platform the sender is running (Android, iOS, Chrome). The event object (which is the manifestation of a message) that is passed to the event listeners has a data element (event.data) where the data takes on the properties of the specific event type.

A receiver application may choose to listen for messages on a specified namespace. By virtue of doing so, the receiver application is said to support that namespace protocol. It is then up to any connected senders wishing to communicate on that namespace to use the appropriate protocol.

All namespaces are defined by a string and must begin with "urn:x-cast:" followed by any string. For example, "urn:x-cast:com.example.cast.mynamespace".

Note: The media message namespace "urn:x-cast:com.google.cast.media" is reserved. Media messages sent using the Cast APIs on both the sender and receiver use the media namespace protocol by convention. See Media and players for more about media messages.

Here is a code snippet for the receiver to listen to custom messages from connected senders:

const context = cast.framework.CastReceiverContext.getInstance();

const CUSTOM_CHANNEL = 'urn:x-cast:com.example.cast.mynamespace';
context.addCustomMessageListener(CUSTOM_CHANNEL, function(customEvent) {
  // handle customEvent.
});

context.start();

Similarly, receiver applications can keep senders informed about the state of the receiver by sending messages to connected senders. A receiver application can send messages using sendCustomMessage(namespace, senderId, message) on CastReceiverContext. A receiver can send messages to an individual sender, either in response to a received message or due to an application state change. Beyond point-to-point messaging (with a limit of 64kb), a receiver may also broadcast messages to all connected senders.

Cast for audio devices

See Google Cast for audio devices guide for support on audio only playback.