Smart Home CameraStream Trait Schema

action.devices.traits.CameraStream - This trait covers how to control the camerastream of a device.

This trait belongs to devices which have the capability to stream video feeds to smart displays, Chromecast-enabled devices, or smartphones. By and large, these are security cameras or baby cameras. But this trait also applies to more complex devices which have a camera on them (for example, video-conferencing devices or a vacuum robot with a camera on it).

Device ATTRIBUTES

Devices with this trait may report the following attributes as part of the SYNC operation. To learn more about handling SYNC intents, see Intent fulfillment.

Attributes Type Description
cameraStreamSupportedProtocols Array

Required.

Supported media types for the camera stream, ordered by preference. Typically, the first protocol in this array that is compatible with the target surface is requested.

[item, ...] String

Media type.

Supported values:

hls
HTTP Live Streaming
dash
Dynamic Adaptive Streaming over HTTP
smooth_stream
Smooth Streaming
progressive_mp4
Progressive MP4 (mostly used for clips)
webrtc
WebRTC
cameraStreamNeedAuthToken Boolean

Required.

Whether an auth token will be provided via cameraStreamAuthToken for the target surface to stream the camera feed.

Examples

Camera with multiple protocols support, not requiring an auth token.

{
  "cameraStreamSupportedProtocols": [
    "webrtc",
    "hls",
    "dash",
    "smooth_stream",
    "progressive_mp4"
  ],
  "cameraStreamNeedAuthToken": false
}

Camera supporting a single protocol, requiring an auth token.

{
  "cameraStreamSupportedProtocols": [
    "hls"
  ],
  "cameraStreamNeedAuthToken": true
}

Supported streaming protocols

This trait supports the following streaming protocols:

When streaming hls,dash,smooth_stream, and progressive_mp4 protocols to Cast devices (Chromecasts, smart displays, and Chromecast-enabled smart TVs), a Cast Web Receiver is launched to process the stream and render it on the device. It is recommended that the developer create a Custom Web Receiver to enable access to debugging tools, customize player behavior, customize UI branding, and include analytics. To enable the usage of the custom receiver and opt out of using the default receiver, set the receiver app id that is generated when you register your Cast application in the EXECUTE response's cameraStreamReceiverAppId field.

To learn more about Custom Web Receiver applications, visit the Developer's Site guide.

Device STATES

None.

Device COMMANDS

Devices with this trait may respond to the following commands as part of the EXECUTE operation. To learn more about handling EXECUTE intents, see Intent fulfillment.

action.devices.commands.GetCameraStream

Parameters

Parameters Type Description
StreamToChromecast Boolean

Required.

Whether the stream will be played on a Chromecast device.

SupportedStreamProtocols Array

Required.

Media types/formats supported by the desired destination.

[item, ...] String

Media type.

Supported values:

hls
HTTP Live Streaming
dash
Dynamic Adaptive Streaming over HTTP
smooth_stream
Smooth Streaming
progressive_mp4
Progressive MP4 (mostly used for clips)
webrtc
WebRTC

Results

Results Type Description
cameraStreamAuthToken String

An auth token for the specific receiver to authorize access to the stream. If cameraStreamNeedAuthToken is true and this value is not provided, the user's OAuth credentials will be used as the auth token.

cameraStreamProtocol String

Required.

The media format that the stream url points to. It should be one of the protocols listed in the SupportedStreamProtocols command parameter.

Supported values:

hls
HTTP Live Streaming
dash
Dynamic Adaptive Streaming over HTTP
smooth_stream
Smooth Streaming
progressive_mp4
Progressive MP4 (mostly used for clips)
webrtc
WebRTC

Non-WebRTC results

Results Type Description
cameraStreamAccessUrl String

Required.

URL endpoint for retrieving the real-time stream in the format specified by cameraStreamProtocol.

cameraStreamReceiverAppId String

Cast receiver ID to process the camera stream when the StreamToChromecast parameter is true; default receiver will be used if not provided.

WebRTC results

Results Type Description
cameraStreamSignalingUrl String

Required.

URL endpoint for retrieving and exchanging camera and client session description protocols (SDPs). The client should return the signaling URL which uses the cameraStreamAuthToken as the authentication token in the request header.

cameraStreamOffer String

Offer session description protocol (SDP).

cameraStreamIceServers String

Represents the Interactive Connectivity Establishment (ICE) servers using an encoded JSON string with the description of a RTCIceServer. If you do not specify STUN (Session Traversal Utilities for NAT) servers, the platform defaults to Google's public STUN servers. TURN (Traversal Using Relays around NAT) servers are only required if you cannot guarantee the IPs / ICE candidates provided will be publicly accessible (e.g. via a media server, public host ICE candidate, relay ICE candidate, etc).

Examples

Show the front door camera (default Cast receiver).

{
  "cameraStreamAccessUrl": "https://fluffysheep.com/baaaaa.mp4",
  "cameraStreamProtocol": "progressive_mp4"
}

Show front door camera (custom Cast receiver).

{
  "cameraStreamAccessUrl": "https://fluffysheep.com/baaaaa.mp4",
  "cameraStreamReceiverAppId": "1g2f89213hg",
  "cameraStreamAuthToken": "12657342190192783",
  "cameraStreamProtocol": "progressive_mp4"
}

Show front door camera (WebRTC stream).

{
  "cameraStreamIceServers": "[{\"urls\": \"stun:stun.l.partner.com:19302\"},{\"urls\":\"turn:192.158.29.39:3478?transport=udp\",\"credential\": \"JZEOEt2V3Qb0y27GRntt2u2PAYA=\",\"username\": \"28224511:1379330808\"},{\"urls\": \"turn:192.158.29.39:3478?transport=tcp\",\"credential\": \"JZEOEt2V3Qb0y27GRntt2u2PAYA=\",\"username\": \"28224511:1379330808\"}]",
  "cameraStreamSignalingUrl": "https://example.com/signaling/answer",
  "cameraStreamOffer": "o=- 4611731400430051336 2 IN IP4 127.0.0.1...",
  "cameraStreamProtocol": "webrtc"
}

Device ERRORS

See the full list of errors and exceptions.

WebRTC protocol specifications

The benefits of using WebRTC are low latency and 1-way talk. WebRTC uses a POST method with a POST body and response in JSON format.

WebRTC is currently supported on the Google Nest Smart Display and Chromecast with Google TV devices.

This section describes the requirements when using the WebRTC stream protocol.

Data Type Parameters/Definitions
Signaling request header

The header should meet these requirements:

  • Authentication: The authentication header should use the auth token from the GetCameraStream return value for cameraStreamAuthToken with token type Bearer.
  • Content-Type: application/json.
Signaling request parameters

The request can include these parameters:

  • action: String. The valid values are:
    • offer: Offer SDP message from provider.
    • answer: Answer SDP message from provider.
    • end: Close the current session.
  • deviceId: String. The device ID as reported in a SYNC or EXECUTE request.
  • sdp: String. Contains the Session Description Protocol message for the peer connection. The content is based on the value of the action parameter. If the action is "end", this parameter can be empty.
Signaling response parameters

The response can include these parameters:

  • action: String. Response value must be of type answer.
  • sdp: String. SDP message for the responding answer.

WebRTC Requirements and Recommendations

  • Google currently supports 1-way (half duplex) communication.
  • You must support bundling and rtcp-mux.
  • You must use (D)TLS 1.2 or later.
  • Trickle ICE is not supported. All ICE candidates must be gathered first before sending the SDP.
  • It is strongly recommended that you include UDP/IPv4, TCP/IPv4, UDP/IPv6 and TCP/IPv6 ICE candidates to increase the probability of a successful connection.

Supported video resolutions:

  • Minimum: 480p
  • Maximum: 1080p

Supported video codecs:

  • VP8
  • H.264

Supported audio codecs:

  • Opus (preferred codec)
  • G.711/PCMU
  • G.722

Cross-Origin Resource Sharing

Cross-Origin Resource Sharing (CORS) is a mechanism that uses additional HTTP Headers to tell browsers to allow a web application running at one origin to access selected resources from a different origin. The server hosting cameraStreamSignalingUrl should respond with the following header:

Access-Control-Allow-Origin: https://www.gstatic.com

Sample signaling request and response

The following example shows a request that Google sends to your signaling service and the corresponding response to Google.

Request
Header:

Authentication: Bearer <cameraStreamAuthToken>
Content-Type: application/json

POST body:

// When camera offer SDP is provided in the execution response, Google provides an answer SDP.
{
  "action": "answer",
  "deviceId": "123",
  "sdp": "o=- 4611731400430051336 2 IN IP4 127.0.0.1..."
}

// When camera offer SDP is not provided in execution response, Google generates and provides an offer SDP.
{
  "action": "offer",
  "deviceId": "123",
  "sdp": "o=- 4611731400430051336 2 IN IP4 127.0.0.1..."
}

// Close the current stream session.
{
  "action": "end"
  "deviceId": "123"
}

Response
// Response to accept the answer SDP in the request.
Response Code : 200
{}

// Response to provide the answer SDP from the service provider.
Response Code : 200
{
  // When the camera offer SDP is not provided in the execution response,
  // Google provides the answer SDP via the signaling response.
  "action": "answer"
  "sdp": "o=- 4611731400430051336 2 IN IP4 127.0.0.1..."
}

// Response to close current session.
Response Code : 200
{}