Register Custom Device Actions

Your device may have special abilities not covered by the current set of traits. For example, there isn't a "dance" or a "blink my light" trait...yet. That's ok, you can define your own Custom Device Actions. A Custom Device Action specifies the command that is sent to your device to trigger its special ability.

To define a Custom Device Action, you need four pieces of information:

  1. A pattern to try to match against the user query
  2. A Custom Device Action to associate with a matched query
  3. Text that is spoken back to the user if the device supports the action
  4. A command name that is sent back to the device, along with any parameters

You create the Custom Device Action by putting this information into an Actions on Google Action Package. Action Packages define the format for Google Assistant Server responses. Unlike the Actions SDK, Custom Device Actions are fulfilled locally; you do not specify an endpoint to process requests and provide responses. Custom Device Actions are not conversational in nature.

Create an Action Package

Create a file (e.g., actions.json) that defines a test command - blinking an LED. You can use the example below; copy it from the sample code you downloaded in an earlier step:

cd assistant-sdk-python/google-assistant-sdk/googlesamples/assistant/grpc/
cp ~/assistant-sdk-python/google-assistant-sdk/actions.json .
Example
{
    "manifest": {
        "displayName": "Blinky light",
        "invocationName": "Blinky light",
        "category": "PRODUCTIVITY"
    },
    "actions": [
        {
            "name": "com.example.actions.BlinkLight",
            "availability": {
                "deviceClasses": [
                    {
                        "assistantSdkDevice": {}
                    }
                ]
            },
            "intent": {
                "name": "com.example.intents.BlinkLight",
                "parameters": [
                    {
                        "name": "number",
                        "type": "SchemaOrg_Number"
                    },
                    {
                        "name": "speed",
                        "type": "Speed"
                    }
                ],
                "trigger": {
                    "queryPatterns": [
                        "blink ($Speed:speed)? $SchemaOrg_Number:number times",
                        "blink $SchemaOrg_Number:number times ($Speed:speed)?"
                    ]
                }
            },
            "fulfillment": {
                "staticFulfillment": {
                    "templatedResponse": {
                        "items": [
                            {
                                "simpleResponse": {
                                    "textToSpeech": "Blinking $number times"
                                }
                            },
                            {
                                "deviceExecution": {
                                    "command": "com.example.commands.BlinkLight",
                                    "params": {
                                        "speed": "$speed",
                                        "number": "$number"
                                    }
                                }
                            }
                        ]
                    }
                }
            }
        }
    ],
    "types": [
        {
            "name": "$Speed",
            "entities": [
                {
                    "key": "slowly",
                    "synonyms": [
                        "slowly",
                        "slow"
                    ]
                },
                {
                    "key": "normally",
                    "synonyms": [
                        "normally",
                        "regular"
                    ]
                },
                {
                    "key": "quickly",
                    "synonyms": [
                        "quickly",
                        "fast",
                        "quick"
                    ]
                }
            ]
        }
    ]
}

This example uses the following information to define the Custom Device Action:

  1. A pattern to try to match against the user query (blink N times)
  2. The Custom Device Action to associate with a matched query (com.example.actions.BlinkLight) for organizational purposes
  3. Text that is spoken back to the user if the device supports the action (Blinking N times)
  4. A command name (com.example.commands.BlinkLight) that is sent back to the device, along with any parameters (a number)

Note the following:

  • The types [...] array defines the list of custom types (for example, $Speed).
  • Custom types can be used in the query pattern. Any of the synonyms in that type can be spoken by the user to match the query pattern.
  • When a synonym does match, the type instance (speed) would return the normalized key (slowly). There can be multiple entities in case, for example, there are different lights that support different speeds of blinking.
  • Parts of the request TTS pattern can be optional. For example, use ($Speed:speed)? in the query pattern to make this part optional.
  • $type.raw (for example, $speed.raw) in the response TTS is replaced by the word(s) the user actually spoke.

Descriptions for many of these fields are available in the Actions Package reference documentation.

Deploy the Action Package

Now make the Action Package accessible to the Google Assistant Server.

While you can do the steps in this section on the device, it may be easier to do them on your development machine. These commands do not require a virtual environment to run.

  1. Download the gactions command line tool.

  2. Remove any existing credentials from the same directory as the gactions tool.

    rm creds.data

  3. Save your Action Package to Google by using the gactions CLI. Replace project_id with your Actions Console project ID.

    ./gactions update --action_package actions.json --project project_id

  4. The first time you run this command you will be given a URL and be asked to sign in. Copy the URL and paste it into a browser (this can be done on any machine). The page will ask you to sign in to your Google account. Sign into the Google account that created the project in a previous step.

  5. After you approve the permission request from the API, a code will appear in your browser, such as "4/XXXX". Copy and paste this code into the terminal:

    Enter the authorization code:

    If authorization was successful, you will see a response similar to the following:

    Your app for the Assistant for project my-devices-project was successfully updated with your actions.

  6. Deploy your action package into test mode by using the gactions CLI. You must have saved your Action Package to Google at least once before running this command. Test mode enables the action package on your user account only.

    ./gactions test --action_package actions.json --project project_id

    Currently, you cannot test the project using the Actions Web Simulator.

  7. When you are finished building and testing your action package, update your action package using gactions update. Contact us for more information about distribution. Approved action packages can be used on any user account.

  8. Navigate to the console to submit your Action Package. For more details, refer to the Action Package reference documentation.

Modify the sample

Do the steps in this section on the device.

nano pushtotalk.py

Add a handler for your custom action. Note that the handler below has already been added to the sample code for the sample Action Package above.

...

device_handler = device_helpers.DeviceRequestHandler(device_id)

@device_handler.command('com.example.commands.BlinkLight')
def blink(speed, number):
    logging.info('Blinking device %s times.' % number)
    delay = 1
    if speed == "slowly":
        delay = 2
    elif speed == "quickly":
        delay = 0.5
    for i in range(int(number)):
        logging.info('Device is blinking.')
        # GPIO.output(25, 1)
        time.sleep(delay)
        # GPIO.output(25, 0)
        time.sleep(1)

Run the sample

Run the source code.

python pushtotalk.py

Try a query. For the example above, try the following:

Blink 5 times.

Note that the query needs to match the query pattern in the Action Package.