Join the Actions on Google Developer Challenge to win a trip to Google I/O 2018 and more than 20 other prizes.

Helpers

Helpers tell the Assistant to momentarily take over the conversation to obtain common data such as a user's full name, a date and time, or a delivery address. When you request a helper, the Assistant presents a standard, consistent UI to users to obtain this information, so you don't have to design your own.

The general process for using a helper is:

  1. Specify the helper's intent in the possibleIntents object when responding to the Assistant. When the Assistant receves the response, it knows that it should carry out the dialog for the helper.
  2. In the subsequent request to your fulfillment, the Assistant provides the helper intent and the result of the helper.
  3. Parse any information that you need and continue on with your conversation.

Here's the following intents that you can requess:

Built-in intents

Intent name API.AI Event name Usage
actions.intent.PERMISSION actions_intent_PERMISSION Obtain the user's full name, coarse location, or precise location, or all 3.
actions.intent.OPTION actions_intent_OPTION Receive the selected item from a list or carousel UI
actions.intent.DATETIME actions_intent_DATETIME Obtain a date and time input from the user.
actions.intent.SIGN_IN actions_intent_SIGN_IN Requests an account linking flow to link a user's account.
actions.intent.DELIVERY_ADDRESS actions_intent_DELIVERY_ADDRESS Obtain a delivery address input from the user
actions.intent.CONFIRMATION actions_intent_CONFIRMATION Obtain a confirmation from the user (for example, an answer to a yes or no question)

The following sections describe the available helpers and the associated intent that you must request to use the helper.

User information

You can obtain the following user information with this helper:

  • Display name
  • Given name
  • Family name
  • Coarse device location (zip code and city)
  • Precise device location (coordinates and street address)

Calling the helper

Call the helper with the Node.js client library:

Node.js
// Choose one or more supported permissions to request:
// app.SupportedPermissions.NAME
// app.SupportedPermissions.DEVICE_PRECISE_LOCATION
// app.SupportedPermissions.DEVICE_COARSE_LOCATION

let namePermission = app.SupportedPermissions.NAME;
let preciseLocationPermission = app.SupportedPermissions.DEVICE_PRECISE_LOCATION

// Ask for one permission
app.askForPermission('To address you by name', [namePermission]);
// Ask for more than one permission. User can authorize all or none.
app.askForPermissions('To address you by name and know your location',
    [namePermission, preciseLocationPermission]);
JSON
{
  "conversationToken": "{\"state\":null,\"data\":{}}",
  "expectUserResponse": true,
  "expectedInputs": [
    {
      "inputPrompt": {
        "initialPrompts": [
          {
            "textToSpeech": "PLACEHOLDER_FOR_PERMISSION"
          }
        ],
        "noInputPrompts": []
      },
      "possibleIntents": [
        {
          "intent": "actions.intent.PERMISSION",
          "inputValueData": {
            "@type": "type.googleapis.com/google.actions.v2.PermissionValueSpec",
            "optContext": "To deliver your order",
            "permissions": [
              "NAME",
              "DEVICE_PRECISE_LOCATION"
            ]
          }
        }
      ]
    }
  ]
}

Getting the results of the helper

After the user responds to the helper, you receive a request to your fulfillment and can check to see if the user granted you the information by calling isPermissionGranted() and then access the data with getUserName() or getDeviceLocation().

Node.js
if (app.isPermissionGranted()) {
  let displayName = app.getUserName().displayName;
  let deviceCoordinates = app.getDeviceLocation().coordinates;
}
JSON
{
  "user": {
    "userId": "user123",
    "profile": {
      "displayName": "Jane Smith",
      "givenName": "Jane",
      "familyName": "Smith"
    }
  },
  "conversation": {
    "conversationId": "1494884577894",
    "type": "ACTIVE",
    "conversationToken": "{\"state\":null,\"data\":{}}"
  },
  "inputs": [
    {
      "intent": "actions.intent.PERMISSION",
      "rawInputs": [
        {
          "inputType": "KEYBOARD",
          "query": "yes"
        }
      ],
      "arguments": [
        {
          "name": "PERMISSION",
          "rawText": "yes",
          "textValue": "true"
        }
      ]
    }
  ],
  "surface": {
    "capabilities": [
      {
        "name": "actions.capability.AUDIO_OUTPUT"
      },
      {
        "name": "actions.capability.SCREEN_OUTPUT"
      }
    ]
  },
  "device": {
    "location": {
      "coordinates": {
        "latitude": 37.422366,
        "longitude": -122.084406
      },
      "formattedAddress": "1600 Amphitheatre Parkway, Mountain View, CA 94043, United States",
      "zipCode": "94043",
      "city": "Mountain View"
    },
    "locale": "en-US"
  },
  "isInSandbox": false
}

Once you obtain the user's information, we recommend that you persist the information after receiving it, so you don't have to ask again. If you store the user's information, remember to disclose it in your action's privacy policy when submitting your action for review).

You can display a list or carousel UI and obtain the selected item from the user with the actions.intent.OPTION.

Node.js
function list (app) {
  app.askWithList(app.buildRichResponse()
    .addSimpleResponse('Alright')
    .addSuggestions(
      ['Basic Card', 'List', 'Carousel', 'Suggestions']),
    // Build a list
    app.buildList('Things to learn about')
    // Add the first item to the list
    .addItems(app.buildOptionItem('MATH_AND_PRIME',
      ['math', 'math and prime', 'prime numbers', 'prime'])
      .setTitle('Math & prime numbers')
      .setDescription('42 is an abundant number because the sum of its ' +
        'proper divisors 54 is greater…')
      .setImage('http://example.com/math_and_prime.jpg', 'Math & prime numbers'))
    // Add the second item to the list
    .addItems(app.buildOptionItem('EGYPT',
      ['religion', 'egpyt', 'ancient egyptian'])
      .setTitle('Ancient Egyptian religion')
      .setDescription('42 gods who ruled on the fate of the dead in the ' +
        'afterworld. Throughout the under…')
      .setImage('http://example.com/egypt', 'Egypt')
    )
    // Add third item to the list
    .addItems(app.buildOptionItem('RECIPES',
      ['recipes', 'recipe', '42 recipes'])
      .setTitle('42 recipes with 42 ingredients')
      .setDescription('Here\'s a beautifully simple recipe that\'s full ' +
        'of flavor! All you need is some ginger and…')
      .setImage('http://example.com/recipe', 'Recipe')
    )
  );
}
JSON
{
    "conversationToken": "{\"state\":null,\"data\":{}}",
    "expectUserResponse": true,
    "expectedInputs": [
        {
            "inputPrompt": {
                "richInitialPrompt": {
                    "items": [
                        {
                            "simpleResponse": {
                                "textToSpeech": "Alright! Here are a few things you can learn. Which sounds interesting?"
                            }
                        }
                    ],
                    "suggestions": [
                        {
                            "title": "Basic Card"
                        },
                        {
                            "title": "List"
                        },
                        {
                            "title": "Carousel"
                        },
                        {
                            "title": "Suggestions"
                        }
                    ]
                }
            },
            "possibleIntents": [
                {
                    "intent": "actions.intent.OPTION",
                    "inputValueData": {
                        "@type": "type.googleapis.com/google.actions.v2.OptionValueSpec",
                        "listSelect": {
                            "title": "Things to learn about",
                            "items": [
                                {
                                    "optionInfo": {
                                        "key": "MATH_AND_PRIME",
                                        "synonyms": [
                                            "math",
                                            "math and prime",
                                            "prime numbers",
                                            "prime"
                                        ]
                                    },
                                    "title": "Math & prime numbers",
                                    "description": "42 is an abundant number because the sum of its proper divisors 54 is greater…",
                                    "image": {
                                        "url": "http://example.com/math_and_prime.jpg",
                                        "accessibilityText": "Math & prime numbers"
                                    }
                                },
                                {
                                    "optionInfo": {
                                        "key": "EGYPT",
                                        "synonyms": [
                                            "religion",
                                            "egpyt",
                                            "ancient egyptian"
                                        ]
                                    },
                                    "title": "Ancient Egyptian religion",
                                    "description": "42 gods who ruled on the fate of the dead in the afterworld. Throughout the under…",
                                    "image": {
                                        "url": "http://example.com/egypt",
                                        "accessibilityText": "Egypt"
                                    }
                                },
                                {
                                    "optionInfo": {
                                        "key": "RECIPES",
                                        "synonyms": [
                                            "recipes",
                                            "recipe",
                                            "42 recipes"
                                        ]
                                    },
                                    "title": "42 recipes with 42 ingredients",
                                    "description": "Here's a beautifully simple recipe that's full of flavor! All you need is some ginger and…",
                                    "image": {
                                        "url": "http://example.com/recipe",
                                        "accessibilityText": "Recipe"
                                    }
                                }
                            ]
                        }
                    }
                }
            ]
        }
    ]
}

Getting the results of the helper

When users select an item, the selected item value is passed to you as an argument. You can use the client library to read the value by calling app.getContextArgument(). In the returned value, you will get the key identifier for the selected item:

function itemSelected (app) {
  // Get the user's selection
  const param = app.getContextArgument('actions_intent_option',
    'OPTION').value;

  // Compare the user's selections to each of the item's keys
  if (!param) {
    app.ask('You did not select any item from the list or carousel');
  } else if (param === 'MATH_AND_PRIME') {
    app.ask('42 is an abundant number because the sum of its…');
  } else if (param === 'EGYPT') {
    app.ask('42 gods who ruled on the fate of the dead in the ');
  } else if (param === 'RECIPES') {
    app.ask('Here\'s a beautifully simple recipe that\'s full ');
  } else {
    app.ask('You selected an unknown item from the list or carousel');
  }
}

Date and Time

You can obtain a date and time from users by requesting the actions.intent.DATETIME.

Request the helper

You can provide a custom prompt or use the Assistant's prompt when asking the user for a data and time.

Node.js
function datetime (app) {
  app.askForDateTime('When do you want to come in?',
    'What is the best date to schedule your appointment?',
    'What time of day works best for you?');
}

function datetimeWithoutPrompt (app) {
  app.askForDateTime();
}
JSON
{
  "conversationToken": "{\"state\":null,\"data\":{}}",
  "expectUserResponse": true,
  "expectedInputs": [
    {
      "inputPrompt": {
        "initialPrompts": [
          {
            "textToSpeech": "PLACEHOLDER_FOR_DATETIME"
          }
        ],
        "noInputPrompts": []
      },
      "possibleIntents": [
        {
          "intent": "actions.intent.DATETIME",
          "inputValueData": {
            "@type": "type.googleapis.com/google.actions.v2.DateTimeValueSpec",
            "dialogSpec": {
              "requestDatetimeText": "When do you want to come in?",
              "requestDateText": "What is the best date to schedule your appointment?",
              "requestTimeText": "What time of day works best for you?"
            }
          }
        }
      ]
    }
  ]
}

Getting the results of the helper

After the user responds to the helper, you receive a request to your fulfillment and can check to see if the user granted you the information by calling getDateTime().

Node.js
  function dateTimeHandler (app) {
    if (app.getDateTime()) {
      app.tell({speech: 'Great, see you at your appointment!',
        displayText: 'Great, we'll see you on ' + app.getDateTime().date.month
        + '/' + app.getDateTime().date.day
        + ' at ' + app.getDateTime().time.hours
        + (app.getDateTime().time.minutes || '')});
    } else {
      app.tell('I\'m having a hard time finding an appointment');
    }
  }
JSON
{
  "user": {
    "userId": "user123"
  },
  "conversation": {
    "conversationId": "1494884466160",
    "type": "ACTIVE",
    "conversationToken": "{\"state\":null,\"data\":{}}"
  },
  "inputs": [
    {
      "intent": "actions.intent.DATETIME",
      "rawInputs": [
        {
          "inputType": "VOICE",
          "query": "tuesday at 9"
        }
      ],
      "arguments": [
        {
          "name": "DATETIME",
          "datetimeValue": {
            "date": {
              "year": 2017,
              "month": 5,
              "day": 16
            },
            "time": {
              "hours": 9
            }
          }
        }
      ]
    }
  ],
  "surface": {
    "capabilities": [
      {
        "name": "actions.capability.AUDIO_OUTPUT"
      },
      {
        "name": "actions.capability.SCREEN_OUTPUT"
      }
    ]
  },
  "device": {
    "locale": "en-US"
  },
  "isInSandbox": false
}

Account Signin

You can have users sign-in to their accounts that are associated with your service by requesting the actions.intent.SIGN_IN intent

Request the helper

Node.js
function signIn (app) {
app.askForSignIn();
}
JSON
{
  "conversationToken": "{\"state\":null,\"data\":{}}",
  "expectUserResponse": true,
  "expectedInputs": [
    {
      "inputPrompt": {
        "initialPrompts": [
          {
            "textToSpeech": "PLACEHOLDER_FOR_SIGN_IN"
          }
        ],
        "noInputPrompts": []
      },
      "possibleIntents": [
        {
          "intent": "actions.intent.SIGN_IN",
          "inputValueData": {}
        }
      ]
    }
  ]
}

Getting the results of the helper

After the user responds to the helper, you receive a request to your fulfillment and can check to see if the user granted you the information by calling getDateTime().

Node.js
function signInHandler (app) {
  if (app.getSignInStatus() === app.SignInStatus.OK) {
    let accessToken = app.getUser().accessToken;
    // access account data with token
  } else {
    app.tell('You need to sign-in before using the app.);
  }
}
JSON
{
  "isInSandbox'": false,
  "surface": {
    "capabilities": [
      {
        "name": "actions.capability.AUDIO_OUTPUT"
      },
      {
        "name": "actions.capability.SCREEN_OUTPUT"
      }
    ]
  },
  "inputs": [
    {
      "rawInputs": [
        {
          "query": "i think so",
          "inputType": "VOICE"
        }
      ],
      "arguments": [
        {
          "name": "SIGN_IN",
          'extension': {
            "@type": "type.googleapis.com/google.actions.v2.SignInValue",
            "status": "OK"
          }
        }
      ],
      "intent': "actions.intent.SIGN_IN"
    }
  ],
  "user": {
    "userId": "user123"
  },
  "device": {
    "locale": "en-US"
  },
  "conversation": {
    "conversationId": "1494606917128",
    "type": "ACTIVE",
    "conversationToken": "[\"_actions_on_google_\"]"
  }
};

Confirmation

You can ask a generic confirmation from the user (yes/no question) and get the resulting answer. The grammar for "yes" and "no" naturally expands to things like "Yea" or "Nope", making it usable in many situations.

Request the helper

You can provide a custom prompt or use the Assistant's prompt when asking the user for confirmation.

Node.js
  function confirmation (app) {
    app.askForConfirmation('Will you marry me?');
  }

  function confirmationWithoutPrompt (app) {
    app.askForConfirmation();
  }
JSON
{
  "conversationToken": "{\"state\":null,\"data\":{}}",
  "expectUserResponse": true,
  "expectedInputs": [
    {
      "inputPrompt": {
        "initialPrompts": [
          {
            "textToSpeech": "PLACEHOLDER_FOR_CONFIRMATION"
          }
        ],
        "noInputPrompts": []
      },
      "possibleIntents": [
        {
          "intent": "actions.intent.CONFIRMATION",
          "inputValueData": {
            "@type": "type.googleapis.com/google.actions.v2.ConfirmationValueSpec",
            "dialogSpec": {
              "requestConfirmationText": "Will you marry me?"
            }
          }
        }
      ]
    }
  ]
}

Getting the results of the helper

After the user responds to the helper, you receive a request to your fulfillment and can check whether the user confirmed or not.

Node.js
function confirmationHandler (app) {
    if (app.getUserConfirmation()) {
      app.tell('Yes!');
    } else {
      app.tell('Bye!');
    }
  }
JSON
{
  "user": {
    "userId": "user123"
  },
  "conversation": {
    "conversationId": "1494884269122",
    "type": "ACTIVE",
    "conversationToken": "{\"state\":null,\"data\":{}}"
  },
  "inputs": [
    {
      "intent": "actions.intent.CONFIRMATION",
      "rawInputs": [
        {
          "inputType": "VOICE",
          "query": "yes"
        }
      ],
      "arguments": [
        {
          "name": "CONFIRMATION",
          "boolValue": true
        }
      ]
    }
  ],
  "surface": {
    "capabilities": [
      {
        "name": "actions.capability.AUDIO_OUTPUT"
      },
      {
        "name": "actions.capability.SCREEN_OUTPUT"
      }
    ]
  },
  "device": {
    "locale": "en-US"
  },
  "isInSandbox": false
}