Google is committed to advancing racial equity for Black communities. See how.

Connect smart home devices to the Google Assistant

As an Internet of Things (IoT) developer, you can build smart home Actions that give your users the ability to control their devices through touch controls in the Google Home app and voice commands with the Assistant.

Smart home Actions rely on Home Graph to provide contextual data about the home and its devices, creating a logical map of the home. That context gives the Assistant a more natural understanding of the user's requests relative to his or her location in the home. For example, Home Graph can store the concept of a living room that contains multiple types of devices from different manufacturers, such as a thermostat, lamp, fan, and vacuum.

Prerequisites

What you'll build

In this codelab, you'll publish a cloud service that manages a virtual smart washing machine, then build a smart home Action and connect it to the Assistant.

What you'll learn

  • How to deploy a smart home cloud service
  • How to connect your service to the Assistant
  • How to publish device state changes to Google

What you'll need

  • A web browser, such as Google Chrome
  • An iOS or Android device with the Google Home app
  • Node.js version 8 or later

Enable Activity controls

Enable the following Activity controls in the Google Account you plan to use with the Assistant:

  • Web & App Activity
  • Device Information
  • Voice & Audio Activity

Create an Actions project

  1. Go to the Actions on Google Developer Console.
  2. Click New Project, enter a name for the project, and click CREATE PROJECT.

Select the Smart Home App

On the Overview screen in the Actions console, select Smart home.

Choose the Smart home experience card, and you will then be directed to your project console.

Install the Firebase CLI

The Firebase Command Line Interface (CLI) will allow you to serve your web apps locally and deploy your web app to Firebase hosting.

To install the CLI, run the following npm command from the terminal:

npm install -g firebase-tools

To verify that the CLI has been installed correctly, run:

firebase --version

Authorize the Firebase CLI with your Google account by running:

firebase login

Now that you set up your development environment, you can deploy the starter project to verify everything is configured properly.

Get the source code

Click the following link to download the sample for this codelab on your development machine:

Download source code

...or you can clone the GitHub repository from the command line:

git clone https://github.com/googlecodelabs/smarthome-washer.git

About the project

The starter project contains the following subdirectories:

  • public: A frontend UI to easily control and monitor the state of the smart washer.
  • functions: A fully implemented cloud service that manages the smart washer with Cloud Functions for Firebase and Firebase Realtime Database.

Connect to Firebase

Navigate to the washer-start directory, then set up the Firebase CLI with your Actions project:

cd washer-start
firebase use <project-id>

Deploy to Firebase

Navigate to the functions folder and install all the necessary dependencies using npm.

cd functions
npm install

Now that you have installed the dependencies and configured your project, you are ready to run the app for the first time.

firebase deploy

This is the console output you should see:

...

✔ Deploy complete!

Project Console: https://console.firebase.google.com/project/<project-id>/overview
Hosting URL: https://<project-id>.firebaseapp.com

This command deploys a web app, along with several Cloud Functions for Firebase.

Open the Hosting URL in your browser (https://<project-id>.firebaseapp.com) to view the web app. You will see the following interface:

This web UI represents a third-party platform to view or modify device states. To begin populating your database with device information, click UPDATE. You won't see any changes on the page, but the current state of your washer will be stored in the database.

Now it's time to connect the cloud service you've deployed to the Google Assistant using the Actions console.

Configure your Actions console project

Under Overview > Build your Action, select Add Action(s). Enter the URL for your cloud function that provides fulfillment for the smart home intents and click Save.

https://us-central1-<project-id>.cloudfunctions.net/smarthome

On the Develop > Invocation tab, add a Display Name for your Action, and click Save. This name will appear in the Google Home app.

To enable Account linking, select the Develop > Account linking option in the left navigation. Use these account linking settings:

Client ID

ABC123

Client secret

DEF456

Authorization URL

https://us-central1-<project-id>.cloudfunctions.net/fakeauth

Token URL

https://us-central1-<project-id>.cloudfunctions.net/faketoken

Click Save to save your account linking configuration, then click Test to enable testing on your project.

You will be redirected to the Simulator. Verify that testing has been enabled for your project by moving your mouse over the Testing on Device ( ) icon.

Now you can begin implementing the webhooks necessary to connect the device state with the Assistant.

Now that you configured your Action, you can add devices and send data. Your cloud service needs to handle the following three intents:

  • A SYNC intent occurs when the Assistant wants to know what devices the user has connected. This is sent to your service when the user links an account. You should respond with a JSON payload of all the user's devices and their capabilities.
  • A QUERY intent occurs when the Assistant wants to know the current state or status of a device. You should respond with a JSON payload with the state of each requested device.
  • An EXECUTE intent occurs when the Assistant wants to control a device on a user's behalf. You should respond with a JSON payload with the execution status of each requested device.

You will update the functions that you previously deployed to handle these intents in the following sections.

Update SYNC response

Open functions/index.js, which contains the code to respond to requests from the Assistant.

You will need to handle a SYNC intent by returning the device metadata and capabilities. Update the JSON in the onSync array to include the device information and recommended traits for a clothes washer.

index.js

app.onSync((body) => {
  return {
    requestId: body.requestId,
    payload: {
      agentUserId: '123',
      devices: [{
        id: 'washer',
        type: 'action.devices.types.WASHER',
        traits: [
          'action.devices.traits.OnOff',
          'action.devices.traits.StartStop',
          'action.devices.traits.RunCycle',
        ],
        name: {
          defaultNames: ['My Washer'],
          name: 'Washer',
          nicknames: ['Washer'],
        },
        deviceInfo: {
          manufacturer: 'Acme Co',
          model: 'acme-washer',
          hwVersion: '1.0',
          swVersion: '1.0.1',
        },
        willReportState: true,
        attributes: {
          pausable: true,
        },
      }],
    },
  };
});

Deploy to Firebase

Deploy the updated cloud fulfillment using the Firebase CLI:

firebase deploy --only functions

In order to test your smart home Action, you need to link your project with a Google account. This enables testing through Google Assistant surfaces and the Google Home app that are signed in to the same account.

  1. On your phone, open the Google Assistant settings. Note that you should be logged in as the same account as in the console.
  2. Navigate to Google Assistant > Settings > Home Control (under Assistant).
  3. Select the plus (+) icon in the bottom right corner
  4. You should see your test app with the [test] prefix and the display name you set.
  5. Select that item. The Google Assistant will then authenticate with your service and send a SYNC request, asking your service to provide a list of devices for the user.

Open the Google Home app and verify that you can see your washer device.

Now that your cloud service properly reports the washer device to Google, you need to add the ability to request the device state and send commands.

Handle QUERY intent

A QUERY intent includes a set of devices. For each device, you should respond with its current state.

In functions/index.js, edit the QUERY handler to process the list of target devices contained in the intent request.

index.js

app.onQuery(async (body) => {
  const {requestId} = body;
  const payload = {
    devices: {},
  };
  const queryPromises = [];
  const intent = body.inputs[0];
  for (const device of intent.payload.devices) {
    const deviceId = device.id;
    queryPromises.push(queryDevice(deviceId)
      .then((data) => {
        // Add response to device payload
        payload.devices[deviceId] = data;
      }
    ));
  }
  // Wait for all promises to resolve
  await Promise.all(queryPromises)
  return {
    requestId: requestId,
    payload: payload,
  };
});

For each device contained in the request, return the current state stored in the Realtime Database. Update the queryFirebase and queryDevice functions to return the state data of the washer.

index.js

const queryFirebase = async (deviceId) => {
  const snapshot = await firebaseRef.child(deviceId).once('value');
  const snapshotVal = snapshot.val();
  return {
    on: snapshotVal.OnOff.on,
    isPaused: snapshotVal.StartStop.isPaused,
    isRunning: snapshotVal.StartStop.isRunning,
  };
}

const queryDevice = async (deviceId) => {
  const data = await queryFirebase(deviceId);
  return {
    on: data.on,
    isPaused: data.isPaused,
    isRunning: data.isRunning,
    currentRunCycle: [{
      currentCycle: 'rinse',
      nextCycle: 'spin',
      lang: 'en',
    }],
    currentTotalRemainingTime: 1212,
    currentCycleRemainingTime: 301,
  };
}

Handle EXECUTE intent

The EXECUTE intent handles commands to update device state. The response returns the status of each command—for example, SUCCESS, ERROR, or PENDING—and the new device state.

In functions/index.js, edit the EXECUTE handler to process the list of traits that need updates and the set of target devices for each command:

index.js

app.onExecute(async (body) => {
  const {requestId} = body;
  // Execution results are grouped by status
  const result = {
    ids: [],
    status: 'SUCCESS',
    states: {
      online: true,
    },
  };
  const executePromises = [];
  const intent = body.inputs[0];
  for (const command of intent.payload.commands) {
    for (const device of command.devices) {
      for (const execution of command.execution) {
        executePromises.push(
          updateDevice(execution,device.id)
            .then((data) => {
              result.ids.push(device.id);
              Object.assign(result.states, data);
            })
            .catch(() => console.error(`Unable to update ${device.id}`))
        );
      }
    }
  }
  await Promise.all(executePromises)
  return {
    requestId: requestId,
    payload: {
      commands: [result],
    },
  };
});

For each command and target device, update the values in the Realtime Database that correspond to the requested trait. Modify the updateDevice function to update the appropriate Firebase reference and return the updated device state.

index.js

const updateDevice = async (execution,deviceId) => {
  const {params,command} = execution;
  let state, ref;
  switch (command) {
    case 'action.devices.commands.OnOff':
      state = {on: params.on};
      ref = firebaseRef.child(deviceId).child('OnOff');
      break;
    case 'action.devices.commands.StartStop':
      state = {isRunning: params.start};
      ref = firebaseRef.child(deviceId).child('StartStop');
      break;
    case 'action.devices.commands.PauseUnpause':
      state = {isPaused: params.pause};
      ref = firebaseRef.child(deviceId).child('StartStop');
      break;
  }
  return ref.update(state)
    .then(() => state);
};

After you implement all three intents, you can test that your Action controls the washer.

Deploy to Firebase

Deploy the updated cloud fulfillment using the Firebase CLI:

firebase deploy --only functions

Test the washer

Now you can see the value change when you try any of the following voice commands through your phone:

"Hey Google, turn on my washer."

"Hey Google, pause my washer."

"Hey Google, stop my washer."

You can also see the current state of your washer by asking questions.

"Hey Google, is my washer on?"

"Hey Google, is my washer running?"

"Hey Google, what cycle is my washer on?"

You can also see these queries and commands in your Firebase console logs by clicking Develop > Functions > Logs in the navigation menu.

You have fully integrated your cloud service with the smart home intents, enabling users to control and query the current state of their devices. However, the implementation still lacks a way for your service to proactively send event information—such as changes to device presence or state—to the Assistant.

With Request Sync, you can trigger a new sync request when users add or remove devices, or when their device capabilities change. With Report State, your cloud service can proactively send a device's state to Home Graph when users physically change a device state—for example, turning on a light switch—or change the state using another service.

In this section, you will add code to call these methods from the frontend web app.

Enable the HomeGraph API

The HomeGraph API enables the storage and querying of devices and their states within a user's Home Graph. To use this API, you must first open the Google Cloud console and enable the HomeGraph API.

In the Google Cloud console, make sure to select the project that matches your Actions <project-id>. Then, in the API Library screen for the HomeGraph API, click Enable.

Enable Report State

Writes to the Realtime Database trigger the reportstate function in the starter project. Update the reportstate function in functions/index.js to capture the data written to the database and post it to Home Graph via Report State.

index.js

exports.reportstate =
    functions.database.ref('{deviceId}').onWrite(async (change, context) => {
  console.info('Firebase write event triggered this cloud function');
  const snapshot = change.after.val();

  const requestBody = {
    requestId: 'ff36a3cc', /* Any unique ID */
    agentUserId: '123', /* Hardcoded user ID */
    payload: {
      devices: {
        states: {
          /* Report the current state of our washer */
          [context.params.deviceId]: {
            on: snapshot.OnOff.on,
            isPaused: snapshot.StartStop.isPaused,
            isRunning: snapshot.StartStop.isRunning,
          },
        },
      },
    },
  };

  const res = await homegraph.devices.reportStateAndNotification({
    requestBody
  });
  console.info('Report state response:', res.status, res.data);

});

Enable Request Sync

Refreshing the icon in the frontend web UI triggers the requestsync function in the starter project. Implement the requestsync function in functions/index.js to call the HomeGraph API.

index.js

exports.requestsync =
    functions.https.onRequest(async (request, response) => {
  response.set('Access-Control-Allow-Origin', '*');
  console.info('Request SYNC for user 123');
  try {
    const res = await homegraph.devices.requestSync({
      requestBody: {
        agentUserId: '123'
      }
    });
    console.info('Request sync response:', res.status, res.data);
    response.json(res.data);
  } catch (err) {
    console.error(err);
    response.status(500).send(`Error requesting sync: ${err}`)
  }

});

Deploy to Firebase

Deploy the updated code using the Firebase CLI:

firebase deploy --only functions

Test your implementation

Click the Refresh button in the web UI and verify that you see a sync request in the Firebase console log.

Next, adjust the attributes of the washer device in the frontend web UI and click Update. Verify that you can see the state change reported to Google in your Firebase console logs.

Congratulations! You successfully integrated the Assistant with a device cloud service using smart home Actions.

Learn more

Here are some ideas you can implement to go deeper:

You can also learn more about testing and submitting an Action for review, including the certification process to publish your Action to users.