Dialogs and Fulfillment

After the Google Assistant invokes your Conversation Action, you handle the user experience from beginning to end, by defining the user interface with dialogs and satisfying the user's request with fulfillment logic.

API.AI lets you build simple actions without having to write fulfillment logic (referred to as webhooks in API.AI). However, you will want to use a webhook most of the time if you want to do processing of input before returning a response. For the most part, this document assumes that you are going to be using a webhook.

Overview

API.AI provides a web GUI for declaring your Conversation Action for the Google Assistant. You communicate with API.AI in it's own webhook format and API.AI handles the communication with the Google Assistant via the Conversation API.

Your fulfillment webhook therefore receives requests from API.AI when users trigger intents, your webhook then processes the request, responds to API.AI, and then API.AI responds to the Google Assistant.

You define the dialogs that comprise your Conversation Action in the following ways:

  • Request dialogs: Define these dialogs in the User says portion of an intent. Users trigger the intent by speaking something that matches the grammar specified in the User says phrases. When this happens, your fulfillment endpoint receives a request to process it with the information in the Action area of the intent.
  • Response dialogs: Define this in your fulfillment logic and return responses back to users. API.AI also lets you define responses in the Responses area of the intent, if you're not using a webhook for fulfillment. Responses contain the actual text to speak back to the user.

The general process for building dialogs and fulfillment is:

  1. Enable webhook fulfillment for each intent that you want to use fulfillment for.
  2. Define request dialogs by creating intents and specifying User says phrases, which define the user request portion of your dialogs.
  3. Build response dialogs:
    1. Initialize the ApiAiAssistant object.
    2. Create functions to handle requests.
    3. Build and initialize an action map that maps intents to functions. When your endpoint receives a request, the client library checks the intent associated with the request and automatically calls the appropriate function.

Enable webhook fulfillment

Before you begin, you need to enable fulfillment for your API.AI agent. This lets you specify which intents to use webhooks for (most likely all of the ones in your agent).

  1. Click Fulfillment in the left navigation and enable it for your action. You'll later need to provide the URL for the webhook after you deploy it.
  2. Enable fulfillment for any desired intents. This tells API.AI to send a request to your fulfillment webhook when the intent is triggered:
    1. Click Intents in the left navigation
    2. Click the intent you want to enable fulfillment for.
    3. Select Enable webhook for this intent at the bottom of the screen.

Defining request dialogs

Intents let you specify the two-way dialog between users and your action. You specify the request portion of the dialogs in the User says portion of the intent. The Actions portion of the intent specifies how the User says phrases are parsed and the data to pass on to your fulfillment webhook.

To define request dialogs:

  1. Enter phrases in the User says field phrases. These phrases trigger the intent when users speak them.
  2. In the Action area for the intent, specify the action name and any parameters that you want to parse from the User says phrases. When the intent is triggered your webhook receives a request with the action name, parameters, and other information as part of the request. See API.AI webhook format for more information about what your webhook receives. See the Actions and Parameters in the API.AI documentation for more information about defining actions.

Building response dialogs

The most common scenario for building responses is to hand off the logic to a webhook to handle requests from API.AI. You can handle the requests by constructing your own responses as described in the Webhook Request/Response Format, but we highly recommend using the Node.js client library.

Initialize the ApiAiAssistant object

The following code instantiates ApiAiAssistant and does some boilerplate Node.js setup for Google Cloud Functions:

'use strict';

const ApiAiAssistant = require('actions-on-google').ApiAiAssistant;

exports.sillyNameMaker = (req, res) => {
  const assistant = new ApiAiAssistant({request: req, response: res});

  // Create functions to handle requests here

}

Create functions to handle requests

When users speak a phrase that triggers an intent, you receive a request from API.AI. To handle the requests, create functions to handle the triggered intents.

  1. Carry out any logic required to process the user input. You'll typically call the following get functions to get user input from the request.

    • getArgument() - Gets values from query pattern arguments by name
    • getRawInput() - Gets the user's raw input as a string.
  2. Call the ask() function with the text to respond with. The Google Assistant speaks the input text to the user as a response.

The following code shows how to build a simple response that echoes back a number that a user has spoken.

const assistant = new ApiAiAssistant({request: request, response: response});

const WELCOME_INTENT = 'input.welcome';  // the action name from the API.AI intent
const NUMBER_INTENT = 'input.number';  // the action name from the API.AI intent
const NUMBER_ARGUMENT = 'input.mynum'; // the action name from the API.AI intent

function welcomeIntent (assistant) {
  assistant.ask('Welcome to action snippets! Say a number.');
}

function numberIntent (assistant) {
  let number = assistant.getArgument(NUMBER_ARGUMENT);
  assistant.tell('You said ' + number);
}

Build and initialize an action map

Once you have all your functions to handle triggered intents, build an action map to map intents to functions. For example, this builds an action map for our previous example:

let actionMap = new Map();
actionMap.set(WELCOME_INTENT, welcomeIntent);
actionMap.set(NUMBER_INTENT, numberIntent);
assistant.handleRequest(actionMap);

If you have only one handler function, then you don't need to do this and just pass in the function name to handleRequest. In this case, all triggered intents will call this function, so you have to check for the intent that is triggered to do the appropriate thing.

const WELCOME_INTENT = 'input.welcome';
const NUMBER_INTENT = 'input.number';
const NUMBER_ARGUMENT = 'input.mynum';

function responseHandler (assistant) {
  // intent contains the name of the intent you defined in the Actions area of API.AI
  let intent = assistant.getIntent();
  switch (intent) {
    case WELCOME_INTENT:
      assistant.ask('Welcome! Say a number.');
      break;

    case NUMBER_INTENT:
      let number = assistant.getArgument(NUMBER_ARGUMENT);
      assistant.tell('You said ' + number);
      break;
  }
}
// you can add the function name instead of an action map
assistant.handleRequest(responseHandler);

Ending conversations

You can specify the intents that you want to set as end intents in API.AI at the bottom of the intents screen. When these intents are triggered, your Conversation Action ends.

No-match reprompting

When API.AI cannot match any of the input grammars defined in the User says portion of your intents, it triggers a fallback intent. Fallback intents typically reprompt the user to provide the necessary input for your action. You can provide reprompt phrases by specifying them in the Response area of a fallback intent or you can use a webhook to provide responses.

To create fallback intents:

  1. Click Intents in the left navigation of API.AI.
  2. Click the menu icon to the right of the Create Intent button and select Create Fallback Intent. Alternatively, click the Default Fallback Intent to edit it.

  3. Specify reprompt phrases to speak back to users. These phrases should be conversational and be as useful as possible to the user's current context.

    To do this without fulfillment:

    Specify phrases in the Response area of the intent. API.AI randomly chooses phrases from this list to speak back to users until a more specific intent is triggered.

    To do this with fulfillment:

    1. Select the Use webhook checkbox in the Fulfillment area of the intent.
    2. In your fulfillment logic, handle the fallback intent that gets triggered like with any other intent, as described in Building response dialogs.

    For example, the following function uses the assistant.data object (an arbitrary data payload that you can use to maintain state) to store a counter that tracks how many times a fallback intent is triggered. If it's triggered more than once, the action quits. You can then reset the counter when any other intent is triggered.

    function defaultFallback (assistant) {
        assistant.data.fallbackCount++;
        // Provide two prompts before ending game
        if (assistant.data.fallbackCount === 1) {
          assistant.setContext(DONE_YES_NO_CONTEXT);
          assistant.ask('Are you done playing Number Genie?');
        } else {
          assistant.tell('Since I\'m still having trouble, so I\'ll stop here. Let’s play again soon.');
        }
      }
    

    See the Number Genie sample for details on how to implement this.

Using contexts

Use contexts if you want API.AI to trigger falllback intents only in certain situations. This is helpful if you want to have different fallback intents for different no-match scenarios.

  • If you don't set contexts on a fallback intent, it's considered to be a global fallback intent that API.AI triggers when no other intent is matched. You should only have one of these defined if you choose to use one.
  • If you set input contexts on a fallback intent, API.AI triggers this fallback intent when the following is true:

    • The user's current contexts are a superset of the contexts defined in the intent.
    • No other intent matches.

    This lets you use multiple fallback intents with different input contexts to customize no-match reprompting to specific scenarios.

  • If you set an output context on a fallback intent, you keep the user in the same context after the fallback intent is triggered and processed.

See API.AI Contexts for more information.

No-input reprompting

When you call the ask() method to return responses to users, you can specify an array of prompts that is spoken to users if the Google Assistant can't detect any input. The Google Assistant speaks these prompts in order, and you can specify up to three prompts. For example:

assistant.ask(`Guess a number`,
  ['I didn\'t hear a number', 'If you\'re still there, what\'s your guess?',
    'We can stop here. Let\'s play again soon.']);

SSML

SSML lets you make your dialogs sound more life-like. The following code snippet shows you how to use the client library in your fulfillment code to create a response that uses SSML:

function saySSML(assistant) {
  let text_to_speech = '<speak>'
    + 'Here are <say-as interpret-as="characters">SSML</say-as> samples. '
    + 'I can pause <break time="3"/>. '
    + 'I can play a sound <audio src="https://www.example.com/MY_WAVE_FILE.wav">your wave file</audio>. '
    + 'I can speak in cardinals. Your position is <say-as interpret-as="cardinal">10</say-as> in line. '
    + 'Or I can speak in ordinals. You are <say-as interpret-as="ordinal">10</say-as> in line. '
    + 'Or I can even speak in digits. Your position in line is <say-as interpret-as="digits">10</say-as>. '
    + 'I can also substitute phrases, like the <sub alias="World Wide Web Consortium">W3C</sub>. '
    + 'Finally, I can speak a paragraph with two sentences. '
    + '<p><s>This is sentence one.</s><s>This is sentence two.</s></p>'
    + '</speak>'
  assistant.tell(text_to_speech);
};

See the SSML reference documentation for more information.

Deploying fulfillment

You can deploy your fulfillment logic to any webhost platform that supports Node.js. If you don't already have a preferred platform, see fulfillment hosting for more information on how to deploy your app on a variety of hosting options.