Implement Report State

Report State is an important feature which lets the smart home Action proactively report the latest status of the user’s device back to Google’s Home Graph rather than waiting for a QUERY intent.

Report State will report, back to Google, the states of user devices with the specified agentUserId associated with them (sent in the original SYNC request). When the Google Assistant wants to take some action that requires understanding the current state, it can simply look up the state information in the Home Graph instead of issuing a QUERY intent to various third-party clouds prior to issuing the EXECUTE intent.

Without Report State, given lights from multiple providers in a living room, the command Ok Google, brighten my living room will require resolving multiple QUERY intents sent to multiple clouds, as opposed to simply looking up the current brightness values based on what had been previously reported. For the best user experience, the Google Assistant needs to have the current device state, without requiring a round-trip to the device.

Home Graph only stores the state that is sent via Report State. The state that is returned as the response to EXECUTE and QUERY intents is used only for speech responses to the user and are not stored. As a result, Report State should be called even if the new state of the device has already been returned in the response to an EXECUTE or QUERY intent. The ReportState API should also be called right after a SYNC intent. After a SYNC intent, new devices might have been added, and in order to set their inital states, a Report State call should follow.

Home Graph expects complete state data on a per-trait basis, as opposed to all state data for the device. From smart home and Home Graph's perspective, traits have state while devices do not. Home Graph updates states on a per-trait basis and will overwrite all data for a given trait when a request is sent. For example, if you are reporting state for the StartStop trait, send values for both isRunning and isPaused.

If Report State is not implemented, the associated device will not be displayed on visual Assistant surfaces via the QUERY intent. Report State implementation is a requirement for public launch of a smart home agent.

Get started

Enable the API and download credentials

Do the following to implement Report State:

  1. In the Cloud Platform Console, go to the Projects page. Select the project that matches your smart home project ID.
  2. Enable the Google HomeGraph API.
  3. Create a service account. Select APIs & Services > Credentials from the left navbar. Select Create Credentials > Service account key.
    1. Select the role Service Accounts > Service Account Token Creator.
  4. Fill in the form fields to create the service account. Click Create to download the private key in JSON format.

Call the API

You have two options to call Report State:

  • gRPC
  • HTTP POST with a JSON Web Token (JWT)

Select an option from the tabs below:

gRPC

gRPC is a modern open source, high performance RPC framework that can run in any environment. For more information, see the gRPC site.

gRPC is supported on most modern programming languages. For most languages, the gRPC runtime can now be installed in a single step via native package managers such as npm for Node.js, gem for Ruby and pip for Python. Even though our Node, Ruby and Python runtimes are wrapped on gRPC’s C core, users now do not need to explicitly pre-install the C core library as a package in most Linux distributions. For Java, we have simplified the steps needed to add gRPC support to your build tools by providing plugins for Maven and Gradle.

  1. To install gRPC for the language of your choice, see the following external links:
  2. Download and place the proto file according to documentation for your language.
  3. Place the service account key in your project.
  4. Proto Compiler should generate the necessary files for calling the gRPC service. See the generated packages and classes for Java below:
    Generated packages and classes
  5. Implement the code to the gRPC service. See the Authentication page for more information.
  6. Set the same `requestId` you received from the EXECUTE request (if any) and set the states. A Java example is given below:
    private HomeGraphApiServiceGrpc.HomeGraphApiServiceBlockingStub blockingStub;
    public void reportStateWithGrpc(String agentUserId) {
      GoogleCredentials creds;
      try {
        FileInputStream stream = new FileInputStream("key.json");
        creds = GoogleCredentials.fromStream(stream);
    
        ManagedChannel channel = ManagedChannelBuilder.forTarget("homegraph.googleapis.com").build();
    
        blockingStub = HomeGraphApiServiceGrpc.newBlockingStub(channel)
            // See https://grpc.io/docs/guides/auth.html#authenticate-with-google-3.
            .withCallCredentials(MoreCallCredentials.from(creds));
          ReportStateAndNotificationRequest request =
                  ReportStateAndNotificationRequest.newBuilder()
                      .setRequestId(requestId)
                      .setAgentUserId(agentUserId)
                      .setPayload(
                          StateAndNotificationPayload.newBuilder()
                              .setDevices(
                                  ReportStateAndNotificationDevice.newBuilder()
                                      .setStates(getStates())))
                      .build();
    
              System.out.printf("Calling ReportStateAndNotification with request "+ request);
              ReportStateAndNotificationResponse response = blockingStub.reportStateAndNotification(request);
        System.out.println(response.toString());
      } catch (IOException e) {
        e.printStackTrace();
      }
    }
    
    public Builder getStates() {
        // States
        Struct colorTemperature =
            Struct.newBuilder()
                .putFields("name", Value.newBuilder().setStringValue("Yellow").build())
                .putFields("temperature", Value.newBuilder().setNumberValue(24000).build())
                .build();
        Struct colorSpetrum =
            Struct.newBuilder()
                .putFields("name", Value.newBuilder().setStringValue("Red").build())
                .putFields("spectrumRGB", Value.newBuilder().setNumberValue(0xff0000).build())
                .build();
    
        Value device1States =
            Value.newBuilder()
                .setStructValue(
                    Struct.newBuilder()
                        .putFields("color", Value.newBuilder().setStructValue(colorTemperature).build())
                        .build())
                .build();
        Value device2States =
            Value.newBuilder()
                .setStructValue(
                    Struct.newBuilder()
                        .putFields("on", Value.newBuilder().setBoolValue(true).build())
                        .putFields("brightness", Value.newBuilder().setNumberValue(98.0).build())
                        .putFields("color", Value.newBuilder().setStructValue(colorSpetrum).build())
                        .putFields("thermostatMode", Value.newBuilder().setStringValue("heat").build())
                        .putFields(
                            "thermostatTemperatureSetpoint",
                            Value.newBuilder().setNumberValue(78).build())
                        .putFields(
                            "thermostatTemperatureAmbient",
                            Value.newBuilder().setNumberValue(68).build())
                        .putFields(
                            "thermostatTemperatureSetpointHigh",
                            Value.newBuilder().setNumberValue(99).build())
                        .putFields(
                            "thermostatTemperatureSetpointLow",
                            Value.newBuilder().setNumberValue(50).build())
                        .putFields(
                            "thermostatHumidityAmbient", Value.newBuilder().setNumberValue(45).build())
                        .build())
                .build();
        Builder states =
            Struct.newBuilder().putFields(deviceId1, device1States).putFields(deviceId2, device2States);
        return states;
    }
    

HTTP POST

  1. Use the downloaded service account JSON file to create a JSON Web Token. For more information, see Authenticating Using a Service Account.
  2. Construct a JWT payload where the iss field comes from the service account JSON file that you downloaded. The aud, iat and exp fields will be provided by your fulfillment at runtime and the scope needs to include homegraph. See the example below:
    {
      "iss": "<service-account-email>",
      "scope": "https://www.googleapis.com/auth/homegraph",
      "aud": "https://accounts.google.com/o/oauth2/token",
      "iat": <current-time>,
      "exp": <current-time-plus-one-hour>
    }
    
  3. Sign the JWT payload with the private key from your service account.
  4. Use the JWT to request an access token from https://accounts.google.com/o/oauth2/token.
  5. Create the JSON request with the agentUserId and list of devices and their states. Use the same requestId you received from the EXECUTE request (if any). Here's a sample JSON request for Report State:
    {
      "requestId": "ff36a3cc-ec34-11e6-b1a0-64510650abcf",
      "agentUserId": "1234",
      "payload": {
        "devices": {
          "states": {
            "1458765": {
              "on": true
            },
            "4578964": {
              "on": true,
              "isLocked": true
            }
          }
        }
      }
    }
    
  6. Combine the Report State JSON and the token in your HTTP POST request to the Google Home Graph endpoint. Here's an example of how to make the request in the command line using curl, as a test:
    TOKEN=$ACCESS_TOKEN
    DEVICE_STATE=`cat device-state.json`
    curl -i -s -X POST -H "Authorization: Bearer $TOKEN" -H "X-GFE-SSL: yes" \
    -H "Content-Type: application/json" \
    -d "$DEVICE_STATE"  https://homegraph.googleapis.com/v1/devices:reportStateAndNotification
    
Code samples

Java

import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import java.io.UnsupportedEncodingException;
import java.util.HashMap;
import java.util.Map;
import java.util.UUID;

import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.HttpClients;
import org.json.JSONException;
import org.json.JSONObject;

import com.google.auth.oauth2.ServiceAccountCredentials;

import io.jsonwebtoken.JwtBuilder;
import io.jsonwebtoken.Jwts;
import io.jsonwebtoken.SignatureAlgorithm;

public class TestReportStatePost {

    private static final String KEY_JSON = "<PATH_TO_YOUR_SERVICE_ACCOUNT_KEY>";
    private static final String AGENT_USER_ID = "<YOUR_AGENT_USER_ID>";

    public static void main(String[] args) throws IOException, JSONException {

        TestReportStatePost reportState = new TestReportStatePost();
        // create and sign jwt
        String jwt = reportState.getJwt();
        // get access token
        String token = reportState.getAccessToken(jwt);
        // call request sync
        reportState.callRS(token, AGENT_USER_ID);
    }

    private String getJwt() throws IOException {

        FileInputStream stream = new FileInputStream(KEY_JSON);
        ServiceAccountCredentials serviceAccount = ServiceAccountCredentials.fromStream(stream);
        JwtBuilder jwts = Jwts.builder();

        // set claims
        Map claims = new HashMap<>();
        claims.put("exp", System.currentTimeMillis() / 1000 + 3600);
        claims.put("iat", System.currentTimeMillis() / 1000);
        claims.put("iss", serviceAccount.getClientEmail());
        claims.put("aud", "https://accounts.google.com/o/oauth2/token");
        claims.put("scope", "https://www.googleapis.com/auth/homegraph");

        jwts.setClaims(claims).signWith(SignatureAlgorithm.RS256, serviceAccount.getPrivateKey());

        return jwts.compact();
    }

    private String getAccessToken(String jwt) throws JSONException, ClientProtocolException, IOException {

        Map m = new HashMap();
        m.put("grant_type", "urn:ietf:params:oauth:grant-type:jwt-bearer");
        m.put("assertion", jwt);
        // Request parameters and other properties.
        StringEntity params = new StringEntity(formEncode(m));

        HttpEntity entity = httpRequest("https://accounts.google.com/o/oauth2/token", jwt, params,
                "application/x-www-form-urlencoded");

        if (entity != null) {
            final StringBuilder out = readResponse(entity);
            String res = out.toString();
            if (res.indexOf("access_token") > 0) {
                String token = res.split(":")[1];
                token = token.substring(2, token.indexOf(",") - 1);
                return token;
            }
        }
        return "";
    }

    private void callRS(String token, String agentUserId) throws JSONException, ClientProtocolException, IOException {

        JSONObject json = prepareJson(agentUserId);

        StringEntity params = new StringEntity(json.toString());
        HttpEntity entity = httpRequest("https://homegraph.googleapis.com/v1/devices:reportStateAndNotification", token,
                params, "application/json");

        if (entity != null) {
            final StringBuilder out = readResponse(entity);
            System.out.println(out.toString());
        }
    }

    private JSONObject prepareJson(String agentUserId) throws JSONException {
        JSONObject json = new JSONObject();
        //Use the request id from execute request
        json.put("requestId", UUID.randomUUID());
        json.put("agentUserId", agentUserId);//agent_user_id is also ok
        JSONObject payload = new JSONObject();
        JSONObject devices = new JSONObject();
        JSONObject states = new JSONObject();
        JSONObject device1 = new JSONObject();
        device1.put("on", true);
        device1.put("online", true);
        JSONObject device2 = new JSONObject();
        device2.put("on", true);
        device2.put("online", true);
        device2.put("locked", true);
        states.put("1458765", device1);
        states.put("4578964", device2);
        devices.put("states", states);
        payload.put("devices", devices);
        json.put("payload", payload);
        return json;
    }

    /*
     * Helper methods below
     */
    private HttpEntity httpRequest(String url, String token, StringEntity params, String type)
            throws UnsupportedEncodingException, IOException, ClientProtocolException {
        HttpClient httpclient = HttpClients.createDefault();
        HttpPost httppost = new HttpPost(url);
        httppost.setHeader("Authorization", "Bearer " + token);

        // Request parameters and other properties.
        httppost.addHeader("content-type", type);
        httppost.setEntity(params);

        // Execute and get the response.
        HttpResponse response = httpclient.execute(httppost);
        HttpEntity entity = response.getEntity();
        return entity;
    }

    private StringBuilder readResponse(HttpEntity entity) throws IOException, UnsupportedEncodingException {
        InputStream instream = entity.getContent();
        final int bufferSize = 1024;
        final char[] buffer = new char[bufferSize];
        final StringBuilder out = new StringBuilder();

        try {
            Reader in = new InputStreamReader(instream, "UTF-8");
            while (true) {
                int rsz = in.read(buffer, 0, buffer.length);
                if (rsz < 0)
                    break;
                out.append(buffer, 0, rsz);
            }
        } finally {
            instream.close();
        }
        return out;
    }

    private String formEncode(Map<String, String> m) {
        String s = "";
        for (String key : m.keySet()) {
            if (s.length() > 0)
                s += "&";
            s += key + "=" + m.get(key);
        }
        return s;
    }
}

Python

#!/usr/bin/env python

# Copyright 2018 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""A tool for calling HomeGraph API with a JWT signed by a Google API Service Account."""

import argparse
import time
import json
import io

import google.auth.crypt
import google.auth.jwt
import requests
from six.moves import urllib

def generate_jwt(service_account_file):
    """Generates a signed JSON Web Token using a Google API Service Account."""

    # Note: this sample shows how to manually create the JWT for the purposes
    # of showing how the authentication works, but you can use
    # google.auth.jwt.Credentials to automatically create the JWT.
    #   http://google-auth.readthedocs.io/en/latest/reference
    #   /google.auth.jwt.html#google.auth.jwt.Credentials

    signer = google.auth.crypt.RSASigner.from_service_account_file(
        service_account_file)

    now = int(time.time())
    expires = now + 3600  # One hour in seconds

    iss = ''

    with io.open(service_account_file, 'r', encoding='utf-8') as json_file:
        data = json.load(json_file)
        iss = data['client_email']

    payload = {
        'iat': now,
        'exp': expires,
        'aud': 'https://accounts.google.com/o/oauth2/token',
        'iss': iss,
        'scope': 'https://www.googleapis.com/auth/homegraph'
    }

    signed_jwt = google.auth.jwt.encode(signer, payload)

    return signed_jwt


def get_access_token(signed_jwt):
    url = 'https://accounts.google.com/o/oauth2/token'
    headers = {'Content-Type': 'application/x-www-form-urlencoded'}
    data = 'grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Ajwt-bearer&assertion=' + signed_jwt

    response = requests.post(url, headers=headers, data=data)

    if response.status_code == requests.codes.ok:
        token_data = json.loads(response.text)
        return token_data['access_token']

    response.raise_for_status()
    return 'ERROR'

def report_state(access_token, report_state_file):
    url = 'https://homegraph.googleapis.com/v1/devices:reportStateAndNotification'
    headers = {
        'X-GFE-SSL': 'yes',
        'Authorization': 'Bearer ' + access_token
    }
    data = {}

    with io.open(report_state_file, 'r', encoding='utf-8') as json_file:
        data = json.load(json_file)

    response = requests.post(url, headers=headers, json=data)

    print 'Response: ' + response.text

    return response.status_code == requests.codes.ok

def main(service_account_file, report_state_file):
    signed_jwt = generate_jwt(service_account_file)
    print('signed JWT: ' + signed_jwt)

    access_token = get_access_token(signed_jwt)
    print('access token: ' + access_token)

    success = report_state(access_token, report_state_file)
    if success:
        print 'Report State has been done successfully.'
    else:
        print 'Report State failed. Please check the log above.'

if __name__ == '__main__':
    parser = argparse.ArgumentParser(
        description=__doc__,
        formatter_class=argparse.RawDescriptionHelpFormatter)
    parser.add_argument(
        'service_account_file',
        help='The path to your service account json file.')
    parser.add_argument(
        'report_state_file',
        help='The path to the json file containing the states you want to report.')

    args = parser.parse_args()

    main(args.service_account_file, args.report_state_file)

Alternatively, gRPC can be used to call the Report State. The public API and proto is available on https://homegraph.googleapis.com/$discovery/rest.

Handle the Disconnect intent

In the event of unlinking, user data will be removed from Home Graph and Report State calls will fail. When the user unlinks their account, similar to other intents, Google will send an action.devices.DISCONNECT intent to the fulfillment url.

{
     "requestId":"17887805757248734228",
     "inputs":[{"intent": "action.devices.DISCONNECT"}]
}

You should handle the DISCONNECT intent if your fulfillment is set to report state back (either via POST or gRPC). If it is not handled, the Action will continue reporting state and will receive an error back every time.

Error Responses

As you are implementing report state and request sync, there are several possible responses that you will receive from Google. These responses come in the form of HTTP status codes on the response.

  • 200 - Success
  • 400 - Failure: The 400 Bad Request Error is an HTTP response status code that indicates that the server was unable to process the request sent by the client due to invalid syntax. A couple common causes include malformed JSON or using null instead of "" for a string value.
  • 404 - Failure: The requested resource could not be found but may be available in the future. Subsequent requests by the client are permissible. Typically, this means that we cannot find either the user (agent_user_id) or the device. It may also mean that the user has not yet linked with Google, or you didn't send the agent_user_id in the SYNC response.
  • 429 - Failure: The user has sent too many sync requests in a given amount of time. The limit is simply one concurrent sync request per user at a time. We don't allow for concurrent requests to be made for the same user. This is for request sync only, not for report state. Report state does not have this limitation and will accept concurrent requests for the same device.