Add local fulfillment to your smart home Action

This guide walks you through developing and testing local fulfillment for your existing smart home Action.

Tips:

• To familiarize yourself with how local fulfillment works, see the local fulfillment overview.
• For a complete example of a smart home integration using local fulfillment, see the Local Home SDK sample on GitHub.
• For the latest changes in the SDK, see the changelog.
• Local fulfillment does not currently support traits requiring two-factor authentication.

Before you begin

• Familiarize yourself with the basics of creating a smart home Action.
• In the Actions console, make sure you have an existing smart home project and that account linking is configured.
• Make sure that you are logged in with the same Google account in the Actions console and in Assistant on your test device.
• You'll need a Node.js environment to write your app. For installing Node.js and npm, Node Version Manager is recommended.
• To work with the latest version of the Local Home SDK, you will need to enroll your test devices into the Cast Preview Program.

1. Support device discovery

A local fulfillment path is established when Google matches a locally-controllable device to a device returned in the SYNC response from your cloud fulfillment.

To enable Google to discover your device on the local network and establish the local fulfillment path, you need to add discovery information in the Actions console. You also need to update the SYNC response from your cloud fulfillment to let Google know about the locally-controllable device.

Set up the scan config information

To specify the discovery information, follow these steps:

1. Open your smart home project in the Actions console.
2. In the left navigation, click Actions.
3. Under Configure local home SDK (optional) > Add device scan configuration, click New scan config.
4. Select a scan matching protocol type from the drop-down and enter values for Google to scan.

The following tables show the attributes you can add, based on the protocols you want Google to use to scan for your device:

mDNS

Attribute	Description	Example Value
Service Name	Required. Service name published by the device in the format service.domain.	_http._tcp.local
Name	Required. Filter for a unique service instance in the format instance.service.domain. The platform treats this value as a regular expression and returns any matching devices.	my-device-[0-9]{4}\._http\._tcp\.local

UPnP

Attribute	Description	Example Value
Service Type	Required. Fully qualified identifier of the UPnP service in the format domain:service:type:version.	schemas-upnp-org:service:SwitchPower:1
OUI	Optional. Organizationally Unique Identifier. 24-bit value identifying the device manufacturer. Typically, the first three octets of the device MAC address.	1A:2B:3C

UDP

Attribute	Description	Example Value
Broadcast Address	Required. Destination IP address for the UDP broadcast.	255.255.255.255
Broadcast Port	Required. Destination port for the UDP broadcast.	5555
Listen Port	Required. Listen port for the UDP discovery response.	5556
Discovery Packet	Required. Payload to send in the UDP broadcast. Formatted as a hexadecimal encoded string of bytes.	48454C4C4F

Update SYNC response in the cloud fulfillment

The SYNC intent reports to Assistant what devices the user controls and their capabilities.

To support local fulfillment, the Local Home platform checks the SYNC response from your smart home Action’s cloud fulfillment and tries to match the device IDs in the otherDeviceIds field to the verification ID returned by the IDENTIFY handler. Device entries without an otherDeviceIds field are excluded from local fulfillment.

In the otherDeviceIds field of the SYNC response, you need to set the device IDs of smart home devices that can be locally controlled. The field appears at the device level in the response. Google can establish a local fulfillment path on any device with the given ID.

Note: If the local device you are integrating is a hub (or a bridge), the hub device may not be present in the SYNC response. Use the otherDeviceIds field in your SYNC response to represent the local device IDs for end devices connected to the hub that users can control.

Use the customData field to specify any additional data Google needs to connect to a standalone device, or to target end devices via a hub (for example, the port number and other protocol-specific information).

Note: You should always use the customData field whenever you need to send device-specific data to a local fulfillment handler. This may include authorization tokens, encryption keys, or unique identifiers that the handler needs to successfully communicate with the device.

Example

The following snippet shows how you might create your SYNC handler.

Standalone/Hub

{
  "requestId": "ff36a3cc-ec34-11e6-b1a0-64510650abcf",
  "payload": {
    "agentUserId": "1836.15267389",
    "devices": [{
      "id": "123",
      "type": "action.devices.types.OUTLET",
      "traits": [
        "action.devices.traits.OnOff"
      ],
      "name": {
        "name": "Night light"
      },
      "willReportState": false,
      "otherDeviceIds": [{
        "deviceId": "local-device-id"
      }],
      "customData": {
        "port": 5555,
        "authToken": "..."
      }
    }]
  }
}

2. Implement the local fulfillment app

To support local fulfillment, you need to build an app to handle these smart home intents:

• IDENTIFY: Supports discovery of locally-controllable smart devices. The intent handler extracts data that your smart device returns during discovery and sends this in a response to Google.
• EXECUTE: Supports execution of commands.
• REACHABLE_DEVICES: (Optional) Supports discovery of locally-controllable end devices behind a hub (or bridge) device.

This app runs on user’s Google Home or Google Nest devices and connects your smart device to Assistant. You can create the app using TypeScript (preferred) or JavaScript.

TypeScript is recommended because you can leverage bindings to statically ensure that the data your app returns match the types that the platform expects.

For more details about the API, see the Local Home SDK API reference.

Example

The following snippets show how you might initialize the local fulfillment app and attach your handlers.

Standalone

import App = smarthome.App;
const localHomeApp: App = new App("1.0.0");
localHomeApp
  .onIdentify(identifyHandler)
  .onExecute(executeHandler)
  .listen()
  .then(() => {
    console.log("Ready");
  });

Hub

import App = smarthome.App;
const localHomeApp: App = new App("1.0.0");
localHomeApp
  .onIdentify(identifyHandler)
  .onReachableDevices(reachableDevicesHandler)
  .onExecute(executeHandler)
  .listen()
  .then(() => {
    console.log("Ready");
  });

Create your project

In order to deploy your local fulfillment app, you need to build a JavaScript bundle for your code and all its dependencies.

Use the local fulfillment app project initializer to bootstrap the appropriate project structure with your preferred bundler configuration.

Project templates

To select your bundler configuration, run the npm init command as shown in the following examples:

None

TypeScript with no bundler configuration:

npm init @google/local-home-app project-directory/ --bundler none

Project structure:

project-directory/
├── node_modules/
├── package.json
├── .gitignore
├── index.ts
├── test.ts
├── tsconfig.json
├── tslint.json
└── serve.js

Replace project-directory with a new directory that will contain the local fulfillment app project.

Webpack

TypeScript with webpack bundler configuration:

npm init @google/local-home-app project-directory/ --bundler webpack

Project structure:

project-directory/
├── node_modules/
├── package.json
├── .gitignore
├── index.ts
├── test.ts
├── tsconfig.json
├── tslint.json
├── webpack.config.web.js
├── webpack.config.node.js
└── serve.js

Replace project-directory with a new directory that will contain the local fulfillment app project.

Rollup

TypeScript with Rollup bundler configuration:

npm init @google/local-home-app project-directory/ --bundler rollup

Project structure:

project-directory/
├── node_modules/
├── package.json
├── .gitignore
├── index.ts
├── test.ts
├── tsconfig.json
├── tslint.json
├── rollup.config.js
└── serve.js

Replace project-directory with a new directory that will contain the local fulfillment app project.

Parcel

TypeScript with Parcel bundler configuration:

npm init @google/local-home-app project-directory/ --bundler parcel

Project structure:

project-directory/
├── node_modules/
├── package.json
├── .gitignore
├── index.ts
├── test.ts
├── tsconfig.json
├── tslint.json
└── serve.js

Replace project-directory with a new directory that will contain the local fulfillment app project.

Perform common project-level tasks

The generated project supports the following npm scripts:

Bundle

cd project-directory/
npm run build

This script compiles TypeScript source, and bundles your app with its dependencies for the Chrome runtime environment in the dist/web subdirectory and the Node.js runtime environment in the dist/node subdirectory.

Verify

cd project-directory/
npm run lint
npm run compile
npm test

This script verifies the syntax of your TypeScript code, compiles it without producing any output in the dist/ subdirectory, and runs automated tests from test.ts.

Serve

cd project-directory/
npm run start

During development, this script serves your app bundles for the Chrome and Node.js runtime environments locally.

Implement the IDENTIFY handler

The IDENTIFY handler will be triggered when the Google Home or Google Nest device reboots and sees unverified local devices (including end devices connected to a hub). The Local Home platform will scan for local devices using the scan config information you specified earlier and call your IDENTIFY handler with the scan results.

The IdentifyRequest from the Local Home platform contains the scan data of a LocalIdentifiedDevice instance. Only one device instance is populated, based on the scan config that discovered the device.

If the scan results match your device, your IDENTIFY handler should return an IdentifyResponsePayload object, that includes a device object with smart home metadata (such as the types, traits, and report state).

Google establishes a device association if the verificationId from the IDENTIFY response matches one of the otherDeviceIds values returned by the SYNC response.

Note: The verificationId must match one of the otherDeviceIds in your SYNC response, in order for the Local Home platform to know it’s the same device (this process is called deduplication).

Example

The following snippets show how you might create IDENTIFY handlers for standalone device and hub integrations, respectively.

Standalone

const identifyHandler = (request: IntentFlow.IdentifyRequest):
  IntentFlow.IdentifyResponse => {

    // Obtain scan data from protocol defined in your scan config
    const device = request.inputs[0].payload.device;
    if (device.udpScanData === undefined) {
      throw Error("Missing discovery response");
    }
    const scanData = device.udpScanData.data;

    // Decode scan data to obtain metadata about local device
    const verificationId = "local-device-id";

    // Return a response
    const response: IntentFlow.IdentifyResponse = {
      intent: Intents.IDENTIFY,
      requestId: request.requestId,
      payload: {
        device: {
          id: device.id || "",
          verificationId, // Must match otherDeviceIds in SYNC response
        },
      },
    };
    return response;
  };

Hub

const identifyHandler = (request: IntentFlow.IdentifyRequest):
  IntentFlow.IdentifyResponse => {

    // Obtain scan data from protocol defined in your scan config
    const device = request.inputs[0].payload.device;
    if (device.udpScanData === undefined) {
      throw Error("Missing discovery response");
    }
    const scanData = device.udpScanData.data;

    // Decode scan data to obtain metadata about local device
    const proxyDeviceId = "local-hub-id";

    // Return a response
    const response: IntentFlow.IdentifyResponse = {
      intent: Intents.IDENTIFY,
      requestId: request.requestId,
      payload: {
        device: {
          id: proxyDeviceId,
          isProxy: true,     // Device can control other local devices
          isLocalOnly: true, // Device not present in `SYNC` response
        },
      },
    };
    return response;
  };

Identify devices behind a hub

If Google identifies a hub device, it will treat the hub as the conduit to the hub's connected end devices and attempt to verify those end devices.

To enable Google to confirm that a hub device is present, follow these instructions for your IDENTIFY handler:

• If your SYNC response reports the IDs of local end devices connected to the hub, set isProxy as truein the IdentifyResponsePayload.
• If your SYNC response does not report your hub device, set isLocalOnly as true in the IdentifyResponsePayload.
• The device.id field contains the local device ID for the hub device itself.

Implement the REACHABLE_DEVICES handler (hub integrations only)

The REACHABLE_DEVICES intent is sent by Google to confirm which end devices can be locally controlled. This intent is triggered every time Google runs a discovery scan (roughly once every minute), as long as the hub is detected to be online.

You implement the REACHABLE_DEVICES handler similarly to the IDENTIFY handler, except that your handler needs to gather additional device IDs reachable by the local proxy (that is, the hub) device. The device.verificationId field contains the local device ID for an end device that is connected to the hub.

The ReachableDevicesRequest from the Local Home platform contains an instance of LocalIdentifiedDevice. Through this instance, you can get the proxy device ID as well as data from the scan results.

Your REACHABLE_DEVICES handler should return a ReachableDevicesPayload object that includes a devices object that contains an array of verificationId values representing the end devices that the hub controls. The verificationId values must match one of the otherDeviceIds from the SYNC response.

The following snippet shows how you might create your REACHABLE_DEVICES handler.

Hub

const reachableDevicesHandler = (request: IntentFlow.ReachableDevicesRequest):
  IntentFlow.ReachableDevicesResponse => {

    // Reference to the local proxy device
    const proxyDeviceId = request.inputs[0].payload.device.id;

    // Gather additional device ids reachable by local proxy device
    // ...

    const reachableDevices = [
      // Each verificationId must match one of the otherDeviceIds
      // in the SYNC response
      { verificationId: "local-device-id-1" },
      { verificationId: "local-device-id-2" },
    ];

    // Return a response
    const response: IntentFlow.ReachableDevicesResponse = {
      intent: Intents.REACHABLE_DEVICES,
      requestId: request.requestId,
      payload: {
        devices: reachableDevices,
      },
    };
    return response;
  };

Implement the EXECUTE handler

Your EXECUTE handler in the app processes user commands and uses the Local Home SDK to access your smart devices through an existing protocol.

The Local Home platform passes the same input payload to the EXECUTE handler function as for the EXECUTE intent to your cloud fulfillment. Likewise, your EXECUTE handler returns output data in the same format as from processing the EXECUTE intent. To simplify the response creation, you can use the Execute.Response.Builder class that the Local Home SDK provides.

Your app does not have direct access to the IP address of the device. Instead, use the CommandRequest interface to create commands based on one of these protocols: UDP, TCP, or HTTP. Then, call the deviceManager.send() function to send the commands.

When targeting commands to devices, use the device ID (and parameters from the customData field, if included) from the SYNC response to communicate with the device.

Example

The following code snippet shows how you might create your EXECUTE handler.

Standalone/Hub

const executeHandler = (request: IntentFlow.ExecuteRequest):
  Promise<IntentFlow.ExecuteResponse> => {

    // Extract command(s) and device target(s) from request
    const command = request.inputs[0].payload.commands[0];
    const execution = command.execution[0];

    const response = new Execute.Response.Builder()
      .setRequestId(request.requestId);

    const result = command.devices.map((device) => {
      // Target id of the device provided in the SYNC response
      const deviceId = device.id;
      // Metadata for the device provided in the SYNC response
      // Use customData to provide additional required execution parameters
      const customData: any = device.customData;

      // Convert execution command into payload for local device
      let devicePayload: string;
      // ...

      // Construct a local device command over TCP
      const deviceCommand = new DataFlow.TcpRequestData();
      deviceCommand.requestId = request.requestId;
      deviceCommand.deviceId = deviceId;
      deviceCommand.data = devicePayload;
      deviceCommand.port = customData.port;
      deviceCommand.operation = Constants.TcpOperation.WRITE;

      // Send command to the local device
      return localHomeApp.getDeviceManager()
        .send(deviceCommand)
        .then((result) => {
          response.setSuccessState(result.deviceId, state);
        })
        .catch((err: IntentFlow.HandlerError) => {
          err.errorCode = err.errorCode || IntentFlow.ErrorCode.INVALID_REQUEST;
          response.setErrorState(device.id, err.errorCode);
        });
    });

    // Respond once all commands complete
    return Promise.all(result)
      .then(() => response.build());
  };

Sending commands to devices behind a hub

To control end devices behind a hub, you may need to provide extra information in the protocol-specific command payload sent to the hub in order for the hub to identify which device the command is aimed for. In some cases, this can be directly inferred from the device.id value, but when this is not the case, you should include this extra data as part of the customData field.

If you created your app using TypeScript, remember to compile your app to JavaScript. You can use the module system of your choice to write your code. Make sure your target is supported by the Chrome browser.

3. Test and debug your app

We recommend that you build your local fulfillment app using the steps described earlier, then test your smart home integration on your own hosting environment using the following steps:

1. In your own hosting environment, serve the HTML page that runs your local fulfillment app. The following snippet shows an example of a static HTML file that runs your local fulfillment app.

   <html>
     <head>
       <!-- Local Home SDK -->
       <script src="//www.gstatic.com/eureka/smarthome/smarthome_sdk.js"></script>
       <!-- Local app under development -->
       <script src="local_execution.js"></script>
     </head>
   
   </html>
   
   

2. Test device control.

3. Debug from Chrome. Use breakpoints and logs to troubleshoot your integration.

4. Modify and compile your TypeScript code, then repeat these steps.

By repeating this build-and-test process, you can see your changes in action quickly and more easily catch and debug issues with your code.

Test device control

In the Action console, you need to specify the URL of your web app, which serves the HTML that gets loaded on the Google Home or Google Nest device during local fulfillment.

Note: In addition to manual testing, we strongly recommend that you test your smart home Action using the test suite for smart home. Doing so helps to ensure that your Action supports all queries that are valid for your device type and traits.

To test device control with local fulfillment, follow these steps:

Chrome

1. Open your Smart Home project in the Actions console.
2. In the left navigation, click Actions.
3. Under Configure local home SDK (optional) > Enter your testing URL for Chrome, specify the development server URL that serves the HTML that runs your local fulfillment app.
4. Click Save. It may take up to 30 minutes for Google to propagate your console changes.
5. Reboot your test Google Home or Google Nest device.
6. Issue a command to your smart device. For example, if your device implements the OnOff trait, you could say "Hey Google, turn on the lights."

Node.js

1. Open your Smart Home project in the Actions console.
2. In the left navigation, click Actions.
3. Under Configure local home SDK (optional) > Enter your testing URL for Node, specify the development server URL that serves the JavaScript that runs your local fulfillment app.
4. Click Save. It may take up to 30 minutes for Google to propagate your console changes.
5. Reboot your test Google Home or Google Nest device.
6. Issue a command to your smart device. For example, if your device implements the OnOff trait, you could say "Hey Google, turn on the lights."

Note: For more details on which runtime is used by your device, see Supported devices.

Note: You can force your test Google Home or Google Nest device to use a local fulfillment path with the command "Hey Google, force local." In this mode, intents will not fall back to cloud fulfillment if an error occurs. When you are done testing, you should reset your device to normal operation using the command "Hey Google, force default."

Debugging from Chrome

You can debug your local fulfillment app using Chrome DevTools. Before you can debug, make sure that your environment is correctly set up:

• You have set your development URL in the console to a URL reachable by the Google Home or Google Nest device (either on the local area network or via the internet),
• Your machine is connected to the same local area network as the Google Home or Google Nest device you are testing.
• Your network doesn’t block packets between devices.
• You are logged in with the same Google account on the Actions console and on the Google Home or Google Nest device.
• You have updated the SYNC response in your cloud fulfillment. It should return at least one valid value in the otherDeviceIds field.
• You have entered the correct scan config information in the Actions console.

To connect your local fulfillment app to the Chrome DevTools debugger, follow these steps:

Chrome

1. In your local development machine, install and launch the Google Chrome browser.
2. In the address field of your Chrome browser, launch the Chrome inspector by entering: chrome://inspect#devices. You should see a list of devices on the page, and your HTML file should be listed under the name of your test Google Home or Google Nest device.
3. Click the blue inspect link under your HTML to launch Chrome DevTools. Switch to the Console tab. The Local Home platform outputs your app version and the Local Home SDK version in the console log. If you see the log, it means that Google has loaded your app successfully, and is able to connect to it. If not, reboot your Google Home or Google Nest device.



Figure 1. Local fulfillment app in chrome://inspect.

Node.js

Note: The following steps require Chrome M84 or later. This release is currently available in the Chrome beta channel.

1. In your local development machine, install and launch the Google Chrome browser.
2. Determine the local IP address of your test device.
3. In the address field of your Chrome browser, launch the Chrome inspector by entering: chrome://inspect#devices.
4. Select Configure... to open the Target discovery settings.



Figure 2. Target discovery settings in chrome://inspect.

5. Enter DEVICE_IP_ADDRESS:9222 in the list and click Done.
6. Click the blue inspect link under your script to launch Chrome DevTools. Switch to the Console tab. The Local Home platform outputs your app version and the Local Home SDK version in the console log. If you see the log, it means that Google has loaded your app successfully, and is able to connect to it. If not, reboot your Google Home or Google Nest device.

Note: For more details on which runtime is used by your device, see Supported Devices.

Debugging tips

Some additional things to keep in mind during debugging include:

• Do not link multiple Google Home or Google Nest devices to your test account on the same local network. You will not be able to control which Google Home or Google Nest device is targeted with the local fulfillment commands.
• Refresh the page in Chrome DevTools to reload your local fulfillment app container with the latest code from your development URL. This does not reset the Local Home platform, which may be necessary to re-trigger platform intents (such as IDENTIFY) in your local fulfillment app. To reset the Local Home platform, reboot your Google Home or Google Nest device.
• Check that your JavaScript app loads without errors. To do this, check the console section of the DevTools page. If there is a problem, you will see a message like this: Uncaught TypeError: Cannot read property ‘open’ of null.
• The verificationId from the IDENTIFY response must match one of the otherDeviceIds from the SYNC response.
• For the EXECUTE handler, make sure your HTTP, TCP, or UDP commands can be received by your device and work as expected.
• Make sure to return a Promise from the handlers.
• Avoid maintaining global state in memory. See Application lifecycle.
• Errors thrown by your local fulfillment app will appear in your project error logs.

4. Prepare and launch to production

When you are ready to launch your smart home Action, follow these steps:

1. Open a terminal. In your project directory, run the npm run build command. This command generates the following JavaScript bundles for your app under the dist directory:

   project-directory/
   └── dist
    ├── web
    │    └── bundle.js
    └── node
         └── bundle.js
   

2. In the console, upload your JavaScript app by clicking Develop > Actions. In the Configure Local Home SDK section, click Upload JavaScript files.

   

   Figure 3. Upload your JavaScript app.

3. In the Upload files dialog, upload the bundle files that you previously generated. Make sure to upload both versions of the bundle files (Node, Web) so that your Action is configured to work correctly across all runtime environments that local fulfillment supports.
   1. Upload your JavaScript targeting Node.js: Upload the bundle.js file from the dist/node directory.
   2. Upload your JavaScript targeting Chrome (browser): Upload the bundle.js file from the dist/web directory.
4. Test your Action on an Assistant-enabled device, to verify that it behaves as expected in a production environment. To learn more, see Test and share your smart home Action.
5. When you are satisfied with how your Action works, submit it to Google for production deployment by following the instructions in Launch your smart home Action. This includes completing the self-test and certification request steps.