Add local execution to your smart home Action

This guide walks you through developing and testing local execution for your existing smart home Action.

Tips:

• To familiarize yourself with how local execution works, see the local execution overview.
• For a complete example of a smart home integration using local execution, see the Local Home SDK sample on GitHub.
• For the latest changes in the SDK, see the changelog.

Before you begin

• Familiarize yourself with the basics of creating a smart home Action.
• In the Actions console, make sure you have an existing smart home project and that account linking is configured.
• Make sure that you are logged in with the same Google account in the Actions console and in the Assistant on your test device.
• You'll need a Node.js environment to write your app. For installing Node.js and npm, Node Version Manager is recommended.
• To work with the latest version of the Local Home SDK, you will need to enroll your test devices into the Cast Preview Program.

1. Support device discovery

A local execution path is established when Google matches a locally-controllable device to a device returned in the SYNC response from your cloud fulfillment.

To enable Google to discover your device on the local network and establish the local execution path, you need to add discovery information in the Actions console. You also need to update the SYNC response from your cloud fulfillment to let Google know about the locally-controllable device.

Set up the scan config information

To specify the discovery information, follow these steps:

1. Open your smart home project in the Actions console.
2. In the left navigation, click Actions.
3. Under Configure Local Home SDK (optional) > Device Scan Configuration, click Add Scan Config.
4. Select a scan matching protocol type from the drop-down and enter values for Google to scan.

The following tables show the attributes you can add, based on the protocols you want Google to use to scan for your device:

mDNS

Attribute	Description	Example Value
Service Name	Required. Service name published by the device in the format service.domain.	_http._tcp.local
Name	Required. Filter for a unique service instance in the format instance.service.domain. The platform treats this value as a regular expression and returns any matching devices.	my-device-[0-9]{4}\._http\._tcp\.local

UPnP

Attribute	Description	Example Value
Service Type	Required. Fully qualified identifier of the UPnP service in the format domain:service:type:version.	schemas-upnp-org:service:SwitchPower:1
OUI	Optional. Organizationally Unique Identifier. 24-bit value identifying the device manufacturer. Typically, the first three octets of the device MAC address.	1A:2B:3C

UDP

Attribute	Description	Example Value
Discovery Address	Required. Destination IP address for the UDP broadcast.	255.255.255.255
Discovery Packet	Required. Payload to send in the UDP broadcast. Formatted as a hexadecimal encoded string of bytes.	48454C4C4F
Discovery Port Out	Required. Destination port for the UDP broadcast.	5555
Discovery Port In	Required. Listen port for the UDP discovery response.	5556

Update SYNC response in the cloud fulfillment

The SYNC intent reports to the Assistant what devices the user controls and their capabilities.

To support local execution, the Local Home platform checks the SYNC response from your smart home Action’s cloud fulfillment and tries to match the device IDs in the otherDeviceIds field to the verification ID returned by the IDENTIFY handler.

In the otherDeviceIds field of the SYNC response, you need to set the device IDs of smart home devices that can be locally controlled. The field appears at the device level in the response. Google can establish a local execution path on any device with the given ID.

Note: If the local device you are integrating is a hub (or a bridge), the hub device may not be present in the SYNC response. Use the otherDeviceIds field in your SYNC response to represent the local device IDs for end devices connected to the hub that users can control.

Use the customData field to specify any additional data Google needs to connect to a standalone device, or to target end devices via a hub (for example, the port number and other protocol-specific information).

Example

The following snippet shows how you might create your SYNC handler.

Standalone/Hub

{
  "requestId": "ff36a3cc-ec34-11e6-b1a0-64510650abcf",
  "payload": {
    "agentUserId": "1836.15267389",
    "devices": [{
      "id": "123",
      "type": "action.devices.types.OUTLET",
      "traits": [
        "action.devices.traits.OnOff"
      ],
      "name": {
        "name": "Night light"
      },
      "willReportState": false,
      "otherDeviceIds": [{
        "deviceId": "local-device-id"
      }],
      "customData": {
        "port": 5555,
        "authToken": "..."
      }
    }]
  }
}

2. Implement the local execution app

To support local execution, you need to build an app to handle these smart home intents:

• IDENTIFY: Supports discovery of locally-controllable smart devices. The intent handler extracts data that your smart device returns during discovery and sends this in a response to Google.
• EXECUTE: Supports execution of commands.
• REACHABLE_DEVICES: (Optional) Supports discovery of locally-controllable end devices behind a hub (or bridge) device.

This app runs on user’s Google Home or Google Nest devices and connects your smart device to the Assistant. You can create the app using TypeScript (preferred) or JavaScript.

TypeScript is recommended because you can leverage bindings to statically ensure that the data your app returns match the types that the platform expects.

For more details about the API, see the Local Home SDK API reference.

Example

The following snippets show how you might create the local execution app and attach your handlers.

Standalone

import App = smarthome.App;
const localHomeApp: App = new App("1.0.0");
localHomeApp
  .onIdentify(identifyHandler)
  .onExecute(executeHandler)
  .listen()
  .then(() => {
    console.log("Ready");
  });

Hub

import App = smarthome.App;
const localHomeApp: App = new App("1.0.0");
localHomeApp
  .onIdentify(identifyHandler)
  .onReachableDevices(reachableDevicesHandler)
  .onExecute(executeHandler)
  .listen()
  .then(() => {
    console.log("Ready");
  });

Implement the IDENTIFY handler

The IDENTIFY handler will be triggered when the Google Home or Google Nest device reboots and sees unverified local devices (including end devices connected to a hub). The Local Home platform will scan for local devices using the scan config information you specified earlier and call your IDENTIFY handler with the scan results.

The IdentifyRequest from the Local Home platform contains the scan data of a LocalIdentifiedDevice instance. Only one device instance is populated, based on the scan config that discovered the device.

If the scan results match your device, your IDENTIFY handler should return an IdentifyResponsePayload object, that includes a device object with smart home metadata (such as the types, traits, and report state).

Google establishes a device association if the verificationId from the IDENTIFY response matches one of the otherDeviceIds values returned by the SYNC response.

Note: The verificationId must match one of the otherDeviceIds in your SYNC response, in order for the Local Home platform to know it’s the same device (this process is called deduplication).

Example

The following snippets show how you might create IDENTIFY handlers for standalone device and hub integrations, respectively.

Standalone

const identifyHandler = (request: IntentFlow.IdentifyRequest):
  IntentFlow.IdentifyResponse => {

    // Obtain scan data from protocol defined in your scan config
    const device = request.inputs[0].payload.device;
    if (device.udpScanData === undefined) {
      throw Error("Missing discovery response");
    }
    const scanData = device.udpScanData.data;

    // Decode scan data to obtain metadata about local device
    const verificationId = "local-device-id";

    // Return a response
    const response: IntentFlow.IdentifyResponse = {
      intent: Intents.IDENTIFY,
      requestId: request.requestId,
      payload: {
        device: {
          id: device.id || "",
          verificationId, // Must match otherDeviceIds in SYNC response
        },
      },
    };
    return response;
  };

Hub

const identifyHandler = (request: IntentFlow.IdentifyRequest):
  IntentFlow.IdentifyResponse => {

    // Obtain scan data from protocol defined in your scan config
    const device = request.inputs[0].payload.device;
    if (device.udpScanData === undefined) {
      throw Error("Missing discovery response");
    }
    const scanData = device.udpScanData.data;

    // Decode scan data to obtain metadata about local device
    const proxyDeviceId = "local-hub-id";

    // Return a response
    const response: IntentFlow.IdentifyResponse = {
      intent: Intents.IDENTIFY,
      requestId: request.requestId,
      payload: {
        device: {
          id: proxyDeviceId,
          isProxy: true,     // Device can control other local devices
          isLocalOnly: true, // Device not present in `SYNC` response
        },
      },
    };
    return response;
  };

Identify devices behind a hub

If Google identifies a hub device, it will treat the hub as the conduit to the hub's connected end devices and attempt to verify those end devices.

To enable Google to confirm that a hub device is present, follow these instructions for your IDENTIFY handler:

• If your SYNC response reports the IDs of local end devices connected to the hub, set isProxy as truein the IdentifyResponsePayload.
• If your SYNC response does not report your hub device, set isLocalOnly as true in the IdentifyResponsePayload.
• The device.id field contains the local device ID for the hub device itself.

Implement the REACHABLE_DEVICES handler (hub integrations only)

The REACHABLE_DEVICES intent is sent by Google to confirm which end devices can be locally controlled. This intent is triggered every time Google runs a discovery scan (roughly once every minute), as long as the hub is detected to be online.

You implement the REACHABLE_DEVICES handler similarly to the IDENTIFY handler, except that your handler needs to gather additional device IDs reachable by the local proxy (that is, the hub) device. The device.verificationId field contains the local device ID for an end device that is connected to the hub.

The ReachableDevicesRequest from the Local Home platform contains an instance of LocalIdentifiedDevice. Through this instance, you can get the proxy device ID as well as data from the scan results.

Your REACHABLE_DEVICES handler should return a ReachableDevicesPayload object that includes a devices object that contains an array of verificationId values representing the end devices that the hub controls. The verificationId values must match one of the otherDeviceIds from the SYNC response.

The following snippet shows how you might create your REACHABLE_DEVICES handler.

Hub

const reachableDevicesHandler = (request: IntentFlow.ReachableDevicesRequest):
  IntentFlow.ReachableDevicesResponse => {

    // Reference to the local proxy device
    const proxyDeviceId = request.inputs[0].payload.device.id;

    // Gather additional device ids reachable by local proxy device
    // ...

    const reachableDevices = [
      // Each verificationId must match one of the otherDeviceIds
      // in the SYNC response
      { verificationId: "local-device-id-1" },
      { verificationId: "local-device-id-2" },
    ];

    // Return a response
    const response: IntentFlow.ReachableDevicesResponse = {
      intent: Intents.REACHABLE_DEVICES,
      requestId: request.requestId,
      payload: {
        devices: reachableDevices,
      },
    };
    return response;
  };

Implement the EXECUTE handler

Your EXECUTE handler in the app processes user commands and uses the Local Home SDK to access your smart devices through an existing protocol.

The Local Home platform passes the same input payload to the EXECUTE handler function as for the EXECUTE intent to your cloud fulfillment. Likewise, your EXECUTE handler returns output data in the same format as from processing the EXECUTE intent. To simplify the response creation, you can use the Execute.Response.Builder class that the Local Home SDK provides.

Your app does not have direct access to the IP address of the device. Instead, use the CommandRequest interface to create commands based on one of these protocols: UDP, TCP, or HTTP. Then, call the deviceManager.send() function to send the commands.

When targeting commands to devices, use the device ID (and parameters from the customData field, if included) from the SYNC response to communicate with the device.

Example

The following code snippet shows how you might create your EXECUTE handler.

Standalone/Hub

const executeHandler = (request: IntentFlow.ExecuteRequest):
  Promise<IntentFlow.ExecuteResponse> => {

    // Extract command(s) and device target(s) from request
    const command = request.inputs[0].payload.commands[0];
    const execution = command.execution[0];

    const response = new Execute.Response.Builder()
      .setRequestId(request.requestId);

    const result = command.devices.map((device) => {
      // Target id of the device provided in the SYNC response
      const deviceId = device.id;
      // Metadata for the device provided in the SYNC response
      // Use customData to provide additional required execution parameters
      const customData: any = device.customData;

      // Convert execution command into payload for local device
      let devicePayload: string;
      // ...

      // Construct a local device command over TCP
      const deviceCommand = new DataFlow.TcpRequestData();
      deviceCommand.requestId = request.requestId;
      deviceCommand.deviceId = deviceId;
      deviceCommand.data = devicePayload;
      deviceCommand.port = customData.port;
      deviceCommand.operation = Constants.TcpOperation.WRITE;

      // Send command to the local device
      return localHomeApp.getDeviceManager()
        .send(deviceCommand)
        .then((result) => {
          response.setSuccessState(result.deviceId, state);
        })
        .catch((err: IntentFlow.HandlerError) => {
          err.errorCode = err.errorCode || IntentFlow.ErrorCode.INVALID_REQUEST;
          response.setErrorState(device.id, err.errorCode);
        });
    });

    // Respond once all commands complete
    return Promise.all(result)
      .then(() => response.build());
  };

Sending commands to devices behind a hub

To control end devices behind a hub, you may need to provide extra information in the protocol-specific command payload sent to the hub in order for the hub to identify which device the command is aimed for. In some cases, this can be directly inferred from the device.id value, but when this is not the case, you should include this extra data as part of the customData field.

If you created your app using TypeScript, remember to compile your app to JavaScript. You can use the module system of your choice to write your code. Make sure your target is supported by the Chrome browser.

3. Test and debug your app

We recommend that you build your local execution app using the steps described earlier, then test your smart home integration on your own hosting environment using the following steps:

1. In your own hosting environment, serve the HTML page that runs your local execution app. The following snippet shows an example of a static HTML file that runs your local execution app.

   <html>
     <head>
       <!-- Local Home SDK -->
       <script src="//www.gstatic.com/eureka/smarthome/smarthome_sdk.js"></script>
       <!-- Local app under development -->
       <script src="local_execution.js"></script>
     </head>
   
   </html>
   
   

2. Test device control.

3. Debug from Chrome. Use breakpoints and logs to troubleshoot your integration.

4. Modify and compile your TypeScript code, then repeat these steps.

By repeating this build-and-test process, you can see your changes in action quickly and more easily catch and debug issues with your code.

Test device control

In the Action console, you need to specify the URL of your web app, which serves the HTML that gets loaded on the Google Home or Google Nest device during local execution.

Note: In addition to manual testing, we strongly recommend that you test your smart home Action using the test suite for smart home. Doing so helps to ensure that your Action supports all queries that are valid for your device type and traits.

To test device control with local execution, follow these steps:

1. Open your Smart Home project in the Actions console.

2. In the left navigation, select Testing > On device testing.

   

   Figure 2. Specify the URL of your deployment server.

3. Specify the development server URL that serves the HTML that runs your local execution app.

4. Click Save. It may take up to 30 minutes for Google to propagate your console changes.

5. Reboot your test Google Home or Google Nest device.

6. Issue a command to your smart device. For example, if your device implements OnOff trait, you could say "Hey Google, turn on the lights."

Debugging from Chrome

You can use Chrome DevTools to debug your app, since the HTML runs in a browser tab. Follow these steps:

1. In your local development machine, install and launch the Google Chrome browser.
2. Reboot the Google Home or Google Nest device you are testing.
3. Check for the following to ensure your debugging environment is correctly set up:
   • You have set your development URL in the console to a URL reachable by the Google Home or Google Nest device (either on the local area network or via the internet),
   • Your machine is connected to the same local area network as the Google Home or Google Nest device you are testing.
   • Your network doesn’t block packets between devices.
   • You are logged in with the same Google account on the Actions console and on the Google Home or Google Nest device.
   • You have updated the SYNC response in your cloud fulfillment. It should return at least one valid value in the otherDeviceIds field.
   • You have entered the correct scan config information in the Actions console.
4. In the address field of your Chrome browser, launch the Chrome inspector by entering: chrome://inspect#devices. You should see a list of devices on the page, and your HTML file should be listed under the name of your test Google Home or Google Nest device.
5. Click the blue inspect link under your HTML to launch Chrome DevTool. Switch to the Console tab. If you explicitly added a console.log statement in your handler, you should see the logging statement from your handler. If you see the log, it means that Google has loaded your app successfully, and is able to talk to your app. It not, reboot your Google Home or Google Nest device again.

Debugging tips

Some additional things to check during debugging include:

• Check that your JavaScript app loads without errors. To do this, check the console section of the DevTools page. If there is a problem, you will see a message like this: Uncaught TypeError: Cannot read property ‘open’ of null.
• The verificationId from the IDENTIFY response must match one of the otherDeviceIds from the SYNC response.
• For the EXECUTE handler, make sure your HTTP/S, TCP, or UDP commands can be received by your device and work as expected.
• Make sure to return a Promise from the handlers.