Smart Home CameraStream Trait Schema

action.devices.traits.CameraStream - This trait covers how to control the camerastream of a device.

This trait belongs to devices which have the capability to stream video feeds to smart displays, Chromecast-enabled devices, or smartphones. By and large, these are security cameras or baby cameras. But this trait also applies to more complex devices which have a camera on them (for example, video-conferencing devices or a vacuum robot with a camera on it).

Device ATTRIBUTES

Devices with this trait may report the following attributes as part of the SYNC operation. To learn more about handling SYNC intents, see Intent fulfillment.

Attributes Type Description

Attributes	Type	Description
`cameraStreamSupportedProtocols`	Array	Required. Supported media types for the camera stream, ordered by preference. Typically, the first protocol in this array that is compatible with the target surface is requested.
`[item, ...]`	String	Media type. Supported values: `hls` HTTP Live Streaming `dash` Dynamic Adaptive Streaming over HTTP `smooth_stream` Smooth Streaming `progressive_mp4` Progressive MP4 (mostly used for clips) `webrtc` WebRTC
`cameraStreamNeedAuthToken`	Boolean	Required. Whether an auth token will be provided via `cameraStreamAuthToken` for the target surface to stream the camera feed.

cameraStreamSupportedProtocols

Array

Required.

Supported media types for the camera stream, ordered by preference. Typically, the first protocol in this array that is compatible with the target surface is requested.

[item, ...]

String

Media type.

Supported values:

hls: HTTP Live Streaming
dash: Dynamic Adaptive Streaming over HTTP
smooth_stream: Smooth Streaming
progressive_mp4: Progressive MP4 (mostly used for clips)
webrtc: WebRTC

cameraStreamNeedAuthToken

Boolean

Required.

Whether an auth token will be provided via cameraStreamAuthToken for the target surface to stream the camera feed.

Examples

Camera with multiple protocols support, not requiring an auth token.

{
  "cameraStreamSupportedProtocols": [
    "webrtc",
    "hls",
    "dash",
    "smooth_stream",
    "progressive_mp4"
  ],
  "cameraStreamNeedAuthToken": false
}

Camera supporting a single protocol, requiring an auth token.

{
  "cameraStreamSupportedProtocols": [
    "hls"
  ],
  "cameraStreamNeedAuthToken": true
}

Supported streaming protocols

This trait supports the following streaming protocols:

hls: HTTP Live Streaming
dash: Dynamic Adaptive Streaming over HTTP
smooth_stream: Smooth Streaming
progressive_mp4: Progressive MP4 (mostly used for clips)
webrtc: WebRTC

Device STATES

None.

Device COMMANDS

Devices with this trait may respond to the following commands as part of the EXECUTE operation. To learn more about handling EXECUTE intents, see Intent fulfillment.

`action.devices.commands.GetCameraStream`

Parameters

Parameters Type Description

Parameters	Type	Description
`StreamToChromecast`	Boolean	Required. Whether the stream will be played on a Chromecast device.
`SupportedStreamProtocols`	Array	Required. Media types/formats supported by the desired destination.
`[item, ...]`	String	Media type. Supported values: `hls` HTTP Live Streaming `dash` Dynamic Adaptive Streaming over HTTP `smooth_stream` Smooth Streaming `progressive_mp4` Progressive MP4 (mostly used for clips) `webrtc` WebRTC

StreamToChromecast

Boolean

Required.

Whether the stream will be played on a Chromecast device.

SupportedStreamProtocols

Array

Required.

Media types/formats supported by the desired destination.

[item, ...]

String

Media type.

Supported values:

hls: HTTP Live Streaming
dash: Dynamic Adaptive Streaming over HTTP
smooth_stream: Smooth Streaming
progressive_mp4: Progressive MP4 (mostly used for clips)
webrtc: WebRTC

Results

Results Type Description

Results	Type	Description
`cameraStreamAuthToken`	String	An auth token for the specific receiver to authorize access to the stream. If `cameraStreamNeedAuthToken` is true and this value is not provided, the user's OAuth credentials will be used as the auth token.
`cameraStreamProtocol`	String	Required. The media format that the stream url points to. It should be one of the protocols listed in the `SupportedStreamProtocols` command parameter. Supported values: `hls` HTTP Live Streaming `dash` Dynamic Adaptive Streaming over HTTP `smooth_stream` Smooth Streaming `progressive_mp4` Progressive MP4 (mostly used for clips) `webrtc` WebRTC

cameraStreamAuthToken

String

An auth token for the specific receiver to authorize access to the stream. If cameraStreamNeedAuthToken is true and this value is not provided, the user's OAuth credentials will be used as the auth token.

cameraStreamProtocol

String

Required.

The media format that the stream url points to. It should be one of the protocols listed in the SupportedStreamProtocols command parameter.

Supported values:

hls: HTTP Live Streaming
dash: Dynamic Adaptive Streaming over HTTP
smooth_stream: Smooth Streaming
progressive_mp4: Progressive MP4 (mostly used for clips)
webrtc: WebRTC

Non-WebRTC results

Results Type Description

Results	Type	Description
`cameraStreamAccessUrl`	String	Required. URL endpoint for retrieving the real-time stream in the format specified by `cameraStreamProtocol`.
`cameraStreamReceiverAppId`	String	Cast receiver ID to process the camera stream when the `StreamToChromecast` parameter is true; default receiver will be used if not provided.

cameraStreamAccessUrl

String

Required.

URL endpoint for retrieving the real-time stream in the format specified by cameraStreamProtocol.

cameraStreamReceiverAppId

String

Cast receiver ID to process the camera stream when the StreamToChromecast parameter is true; default receiver will be used if not provided.

WebRTC results

Results Type Description

Results	Type	Description
`cameraStreamSignalingUrl`	String	Required. URL endpoint for retrieving and exchanging camera and client session description protocols (SDPs). The client should return the signaling URL which uses the `cameraStreamAuthToken` as the authentication token in the request header.
`cameraStreamOffer`	String	Offer session description protocol (SDP).
`cameraStreamIceServers`	String	Represents the Interactive Connectivity Establishment (ICE) servers using an encoded JSON string with the description of a RTCIceServer. If you do not specify STUN (Session Traversal Utilities for NAT) servers, the platform defaults to Google's public STUN servers. TURN (Traversal Using Relays around NAT) servers are only required if you cannot guarantee the IPs / ICE candidates provided will be publicly accessible (e.g. via a media server, public host ICE candidate, relay ICE candidate, etc).

cameraStreamSignalingUrl

String

Required.

URL endpoint for retrieving and exchanging camera and client session description protocols (SDPs). The client should return the signaling URL which uses the cameraStreamAuthToken as the authentication token in the request header.

cameraStreamOffer

String

Offer session description protocol (SDP).

cameraStreamIceServers

String

Represents the Interactive Connectivity Establishment (ICE) servers using an encoded JSON string with the description of a RTCIceServer. If you do not specify STUN (Session Traversal Utilities for NAT) servers, the platform defaults to Google's public STUN servers. TURN (Traversal Using Relays around NAT) servers are only required if you cannot guarantee the IPs / ICE candidates provided will be publicly accessible (e.g. via a media server, public host ICE candidate, relay ICE candidate, etc).

Examples

Show the front door camera (default Cast receiver).

{
  "cameraStreamAccessUrl": "https://fluffysheep.com/baaaaa.mp4",
  "cameraStreamProtocol": "progressive_mp4"
}

Show front door camera (custom Cast receiver).

{
  "cameraStreamAccessUrl": "https://fluffysheep.com/baaaaa.mp4",
  "cameraStreamReceiverAppId": "1g2f89213hg",
  "cameraStreamAuthToken": "12657342190192783",
  "cameraStreamProtocol": "progressive_mp4"
}

Show front door camera (WebRTC stream).

{
  "cameraStreamIceServers": "[{\"urls\": \"stun:stun.l.partner.com:19302\"},{\"urls\":\"turn:192.158.29.39:3478?transport=udp\",\"credential\": \"JZEOEt2V3Qb0y27GRntt2u2PAYA=\",\"username\": \"28224511:1379330808\"},{\"urls\": \"turn:192.158.29.39:3478?transport=tcp\",\"credential\": \"JZEOEt2V3Qb0y27GRntt2u2PAYA=\",\"username\": \"28224511:1379330808\"}]",
  "cameraStreamSignalingUrl": "https://example.com/signaling/answer",
  "cameraStreamOffer": "o=- 4611731400430051336 2 IN IP4 127.0.0.1...",
  "cameraStreamProtocol": "webrtc"
}

Device ERRORS

See the full list of errors and exceptions.

WebRTC protocol specifications

The benefits of using WebRTC are low latency and 1-way talk. WebRTC uses a POST method with a POST body and response in JSON format.

This section describes the requirements when using the WebRTC stream protocol.

Data Type Parameters/Definitions

Signaling request header

Data Type	Parameters/Definitions
Signaling request header	The header should meet these requirements: Authentication: The authentication header should use the auth token from the `GetCameraStream` return value for `cameraStreamAuthToken` with token type `Bearer`. Content-Type: `application/json`.
Signaling request parameters	The request can include these parameters: `action`: String. The valid values are: `offer`: Offer SDP message from provider. `answer`: Answer SDP message from provider. `end`: Close the current session. `deviceId`: String. The device ID as reported in a SYNC or EXECUTE request. `sdp`: String. Contains the Session Description Protocol message for the peer connection. The content is based on the value of the `action` parameter. If the `action` is "end", this parameter can be empty.
Signaling response parameters	The response can include these parameters: `action`: String. Response value must be of type `answer`. `sdp`: String. SDP message for the responding answer.

The header should meet these requirements:

Authentication: The authentication header should use the auth token from the GetCameraStream return value for cameraStreamAuthToken with token type Bearer.
Content-Type: application/json.

Signaling request parameters

The request can include these parameters:

action: String. The valid values are:
- offer: Offer SDP message from provider.
- answer: Answer SDP message from provider.
- end: Close the current session.
deviceId: String. The device ID as reported in a SYNC or EXECUTE request.
sdp: String. Contains the Session Description Protocol message for the peer connection. The content is based on the value of the action parameter. If the action is "end", this parameter can be empty.

Signaling response parameters

The response can include these parameters:

action: String. Response value must be of type answer.
sdp: String. SDP message for the responding answer.

WebRTC Requirements and Recommendations

Google currently supports 1-way (half duplex) communication.
You must support bundling and rtcp-mux.
You must use (D)TLS 1.2 or later.
Trickle ICE is not supported. All ICE candidates must be gathered first before sending the SDP.
It is strongly recommended that you include UDP/IPv4, TCP/IPv4, UDP/IPv6 and TCP/IPv6 ICE candidates to increase the probability of a successful connection.

Supported video resolutions:

Minimum: 480p
Maximum: 1080p

Supported video codecs:

VP8
H.264

Supported audio codecs:

Opus (preferred codec)
G.711/PCMU
G.722

Cross-Origin Resource Sharing (CORS) is a mechanism that uses additional HTTP Headers to tell browsers to allow a web application running at one origin to access selected resources from a different origin. The server hosting cameraStreamSignalingUrl should respond with the following header:

Access-Control-Allow-Origin: https://www.gstatic.com

Sample signaling request and response

The following example shows a request that Google sends to your signaling service and the corresponding response to Google.

Request

Header:

Authentication: Bearer <cameraStreamAuthToken>
Content-Type: application/json

POST body:

// When camera offer SDP is provided in the execution response, Google provides an answer SDP.
{
  "action": "answer",
  "deviceId": "123",
  "sdp": "o=- 4611731400430051336 2 IN IP4 127.0.0.1..."
}

// When camera offer SDP is not provided in execution response, Google generates and provides an offer SDP.
{
  "action": "offer",
  "deviceId": "123",
  "sdp": "o=- 4611731400430051336 2 IN IP4 127.0.0.1..."
}

// Close the current stream session.
{
  "action": "end"
  "deviceId": "123"
}

Response

// Response to accept the answer SDP in the request.
Response Code : 200
{}

// Response to provide the answer SDP from the service provider.
Response Code : 200
{
  // When the camera offer SDP is not provided in the execution response,
  // Google provides the answer SDP via the signaling response.
  "action": "answer"
  "sdp": "o=- 4611731400430051336 2 IN IP4 127.0.0.1..."
}

// Response to close current session.
Response Code : 200
{}