Custom intents

Every app is different, and not all app functionality matches an available App Actions built-in intent. For cases where there isn't a built-in intent (BII) for your app functionality, you can instead use a custom intent to extend your app with App Actions.

Like BIIs, custom intents follow the shortcuts.xml schema and act as connection points between Assistant and your defined fulfillments. Custom intents also have intent parameters, which you can map to parameters in your corresponding fulfillment.

Unlike BIIs, custom intents require query patterns to describe example queries that a user might say. This approach differs from built-in intents, which each model common ways that users express that intent.

Limitations

Custom intents have the following limitations:

  • The name of a custom intent must not begin with actions.intent.
  • The name of a custom intent must be unique among custom intent names for your app.
  • Only certain data types are available for parameter extraction by Google Assistant. See Supported types for the list of supported types.
  • A maximum of two text parameters are supported per query. This limit does not apply to other data types.
  • Only the en-US locale is supported for custom intents (device and Assistant language settings must both match).

Supported types

Custom intents support the following schema.org types for parameter extraction:

  • https://schema.org/Text

    Custom intents support both open-ended and inventory-constrained text parameters. Open-ended parameters pass text values found in a user query directly to your fulfillment. Inventory-constrained parameters only pass values from a user query that match expected values your app defines using an inline inventory.

    Inventory-constrained parameters provide faster, more accurate results and should be preferred when possible. For details, see Use inventory sets for text parameters.

  • https://schema.org/Date

  • https://schema.org/Time

  • https://schema.org/Number

Defining App Actions with custom intents

shortcuts.xml

As with other App Actions that use built-in intents, you define a custom intent in the <capability> element in shortcuts.xml.

Capabilities are defined in the <shortcuts> root element. When you define your <shortcuts> element, you must include the namespaces of the attributes you want to access.

<shortcuts
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto">
  ...
</shortcuts>

Provide the name of the custom intent in the android:name attribute, and reference a query patterns resource file in the queryPatterns attribute.

<shortcuts
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto">
  <capability
      android:name="custom.actions.intent.EXAMPLE_INTENT"
      app:queryPatterns="@array/ExampleQueries">
    <intent ...>
      <url-template
          android:value="http://custom.com{?number_of_items,item_name}" />
      <parameter
          android:name="number_of_items"
          android:key="number_of_items"
          android:mimeType="https://schema.org/Number" />
      <parameter
          android:name="item_name"
          android:key="item_name"
          android:mimeType="https://schema.org/Text" />
    </intent>
  </capability>
  ...
</shortcuts>

When naming your custom intents, we recommend using the prefix custom.actions.intent to distinguish your custom intents from both built-in intents and Android intents (which function differently). Custom intent names must not begin with actions.intent, because that namespace is reserved for built-in intents.

For each parameter, provide the supported schema.org type that best describes the meaning of the parameter. For example, you can use https://schema.org/Date to describe a date you expect to receive:

...
<intent>
  <url-template android:value="https://example.com/appt{?apptType,date,time}" />
  <parameter
      android:name="date"
      android:key="date"
      android:mimeType="https://schema.org/Date" />
  ...
</intent>
...

Shortcut definition in shortcuts.xml uses the same format with custom intents as it does with built-in intents. The following code describes an App Action that uses the referenced query patterns to trigger the SCHEDULE_APPOINTMENT custom intent and a defined set of values, DRIVERS_LICENSE and VEHICLE_REGISTRATION, for the apptType parameter.

<shortcuts
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto">
  <capability
      android:name="custom.actions.intent.SCHEDULE_APPOINTMENT"
      app:queryPatterns="@array/scheduleApptQueries">
    <intent ...>
      <url-template android:value="https://example.com/appt{?apptTypeUrlKey,dateUrlKey,timeUrlKey}" />
      <parameter
          android:name="date"
          android:key="dateUrlKey"
          android:mimeType="https://schema.org/Date" />
      <parameter
          android:name="time"
          android:key="timeUrlKey"
          android:mimeType="https://schema.org/Time" />
      <!-- The following parameter has no type because the shortcuts are bound to it -->
      <parameter android:name="apptType" android:key="apptTypeUrlKey" />
    </intent>
  </capability>

  <shortcut
      android:shortcutShortLabel="@string/driversLicenseLabel"
      android:shortcutId="DRIVERS_LICENSE">
    <capability-binding android:key="custom.actions.intent.SCHEDULE_APPOINTMENT">
      <parameter-binding
          android:key="apptType"
          android:value="@string/driversLicense" />
    </capability-binding>
  </shortcut>

  <shortcut
      android:shortcutsShortLabel="@string/vehicleRegistrationLabel"
      android:shortcutId="VEHICLE_REGISTRATION">
    <capability-binding android:key="custom.actions.intent.SCHEDULE_APPOINTMENT">
      <parameter-binding
          android:key="apptType"
          android:value="@string/vehicleRegistration" />
    </capability-binding>
  </shortcut>
</shortcuts>

You can configure custom intent parameters with inline inventory, which you can use to guide entity extraction to a set of supported entities specified in shortcuts.xml.

actions.xml

As with other App Actions that use built-in intents, you define a custom intent in the <action> element in actions.xml. Provide the name of the custom intent in the intentName attribute, and reference a query patterns resource file in the queryPatterns attribute:

<action intentName="custom.actions.intent.EXAMPLE_INTENT" queryPatterns="@array/ExampleQueries">
    <!-- Define parameters -->
    <!-- Define fulfillment -->
</action>

We recommend naming your custom intents using the prefix "custom.actions.intent" to distinguish them from both built-in intents and Android intents (which function differently). Custom intent names must not begin with "actions.intent", because that namespace is reserved for built-in intents.

For each parameter, provide the supported schema.org type that best describes the meaning of the parameter. For example, you can use https://schema.org/Number to describe a number you expect to receive:

<action intentName="custom.actions.intent.EXAMPLE_INTENT" queryPatterns="@array/ExampleQueries">
    <parameter name="number1" type="https://schema.org/Number" />
    <parameter name="text1" type="https://schema.org/Text" />

    <!-- Define fulfillment -->
</action>

Fulfillment definition in actions.xml uses the same format with custom intents as it does with built-in intents. The following code describes an App Action that uses the referenced query patterns to trigger the "EXAMPLE_INTENT" custom intent and its corresponding fulfillment:

<action intentName="custom.actions.intent.EXAMPLE_INTENT" queryPatterns="@array/ExampleQueries">
    <parameter name="number1" type="https://schema.org/Number" />
    <parameter name="text1" type="https://schema.org/Text" />

    <fulfillment urlTemplate="https://example.com/custom-requests/{?number_of_items,item_name}">
        <parameter-mapping intentParameter="number1" urlParameter="number_of_items" />
        <parameter-mapping intentParameter="text1" urlParameter="item_name" />
    </fulfillment>
</action>

You can configure custom intent parameters with inline inventory, which you can use to guide entity extraction to a set of supported entities specified in actions.xml.

The following code describes an App Action that defines an entity-set-reference for the "apptType" parameter, referencing an entity-set legacy of eligible values for fulfillment:

<action intentName="custom.actions.intent.SCHEDULE_APPOINTMENT" queryPatterns="@array/scheduleApptQueries">
  <parameter name="date" type="https://schema.org/Date" />
  <parameter name="time" type="https://schema.org/Time" />
  <parameter name="apptType">
    <entity-set-reference entitySetId="ApptTypeEntitySet" />
  </parameter>
  <fulfillment urlTemplate="https://example.com/appt{?apptType,date,time}">
    <parameter-mapping intentParameter="date" urlParameter="date" />
    <parameter-mapping intentParameter="time" urlParameter="time" />
    <parameter-mapping intentParameter="apptType" urlParameter="apptType" />
  </fulfillment>
</action>
<entity-set entitySetId="ApptTypeEntitySet">
  <entity identifier="DRIVERS_LICENSE" name="@string/driversLicense" alternateName="@array/driversLicenseSynonyms" />
  <entity identifier="VEHICLE_REGISTRATION" name="@string/vehicleRegistration" alternateName="@array/vehicleRegistrationSynonyms" />
</entity-set>

Query patterns

Each custom intent you use requires a set of queries expected from the user for that intent. This approach is unlike built-in intents, where queries are already modeled for common ways that users express tasks they're trying to do or information they seek.

In an Android resource file (usually /res/values/strings.xml), specify query patterns as items in a string array. When your App Action is invoked, Google Assistant checks the user query against your query patterns as part of matching the user's intent for fulfillment. Each query pattern you provide represents a phrase that you consider valid for the corresponding custom intent.

When providing query patterns for custom intents, expect each pattern to follow an explicit invocation like "open ExampleApp and" or "start ExampleApp and". For example, consider the following user queries:

  • "Hey Google, open ExampleGameApp and start making a cake."
  • "Hey Google, open ExampleGameApp and start making an apple pie."
  • "Hey Google, start ExampleGameApp and craft 5 cake items."
  • "Hey Google, use ExampleGameApp to produce cake 5 times."

To match user queries, provide query patterns that contain the portion of the query after the invocation phrase. For information you want to extract from the query (like text or a number provided by the user), you assign values to the corresponding intent parameter with placeholders in the query pattern.

To reference a parameter in a query pattern, add $ to the name of the parameter in your pattern. For example, to create a placeholder value for a parameter such as <parameter name="date1" ... (actions.xml) or <parameter android:name="date1" ... (shortcuts.xml), you would use $date1.

The following code describes query patterns that match the above user queries and extract values for item names and the number of items to be made:

<resources>
  <string-array name="ExampleQueries">
    <item>start making a $text1</item>
    <item>start making an $text1</item>
    <item>craft $number1 $text1 items</item>
    <item>produce $text1 $number1 times</item>
  </string-array>
</resources>

Query patterns support Conditionals. For example, "set (an)? appointment $date $time". In this case both "set appointment today at noon" and "set an appointment today at noon" are valid queries.

Best practices

You can train Assistant to more effectively recognize requests for your custom intents by providing details of the different ways users might ask for them. To provide a robust set of training context to Assistant, you should:

  • Define a set of effective query patterns for each intent.
  • Optimize the performance of your intent parameters.

Accomplish these steps by following the guidelines in this section.

Define effective query patterns

Assistant uses query patterns to train its natural language understanding (NLU) service to match user voice queries to custom intents. Match accuracy can be improved by providing query patterns that follow these guidelines:

Provide multiple query patterns

Each query pattern represents a variation in how a user might invoke your custom intent. As a general rule, provide at least ten patterns for each intent. Complex intents, such as those including multiple parameters, should include a higher number of patterns.

Vary your query patterns

Effective query patterns consider a broad variety of ways a user might phrase the request. You should define enough patterns to cover variations of questions, commands, verbs, and synonyms for common nouns. For example, these sample patterns describe various ways a user could phrase a request to start preparing a food item:

<resources>
  <string-array name="ExampleQueries">
    <item>start making a $foodItem</item>
    <item>help me prepare a $foodItem</item>
    <item>i want to make a $foodItem</item>
    <item>let’s make a $foodItem</item>
    <item>let’s do a $foodItem recipe tonight</item>
    <item>show me a popular $foodItem recipe?</item>
    <item>what should I do to make a $foodItem?</item>
    <item>show me a $foodItem recipe</item>
    <item>show steps for $foodItem</item>
    <item>$foodItem recipe</item>
  </string-array>
</resources>

Do not use similar query patterns across different intents.

Assistant may not accurately match intents that share similar query phrases. Vary your query patterns across intents to help the NLU service determine the distinct context for each intent.

Optimize intent parameters

The number and complexity of your custom intent parameters can impact how effectively Assistant can extract them from a user query. Follow these guidelines to improve Assistant’s ability to effectively detect and extract these parameters.

Provide several query patterns for each parameter.

In general, each intent parameter should include at least five query patterns which mention it. This enables the NLU service to understand the contexts where a user might use each parameter. For example, a parameter allowing users to remotely adjust their car’s air conditioning might have the following sample patterns:

<resources>
  <string-array name="TempQueries">
    <item>set the car to $temperature degrees</item>
    <item>heat the car up to $temperature degrees</item>
    <item>cool the car down to $temperature degrees</item>
    <item>lower the temperature to $temperature</item>
    <item>raise the temperature to $temperature</item>
    …
  </string-array>
</resources>

Use inventory sets for text parameters

If your custom intent uses https://schema.org/Text type parameters, you can optimize their efficacy by providing the set of values your app expects from that parameter. For example, say you define a custom intent that allows users to open a specific cooking recipe using following example pattern:

<resources>
  <string-array name="ExampleQueries">
    <item>find a `$foodItem` recipe</item>
  </string-array>
</resources>

In shortcuts.xml, the $foodItem parameter included in the pattern is defined as a https://schema.org/Text type:

If your app supports a limited set of values for a text parameter, like the names of supported food recipes, you can provide a reference to a resource containing an array of strings that define an inline inventory of expected parameter values. The following sample demonstrates this:

<shortcut android:shortcutId="FOOD_PIZZA">
  <capability-binding android:key="custom.actions.intent.EXAMPLE_INTENT">
    <parameter-binding
      android:key="foodItem"
      android:value="@array/pizza_names" />
    </capability-binding>
</shortcut>

The preceding sample returns the parameter value FOOD_PIZZA for the following expected values defined in a string array resource:

<!-- Inventory items for "FOOD_PIZZA" -->
<resources>
  <string-array name="pizza_names">
    <item>Cheese pizza</item>
    <item>Veggie pizza</item>
    <item>Pepperoni pizza</item>
    <item>Margherita pizza</item>
    <item>Detroit pizza</item>
    <item>Neopolitan pizza</item>
  </string-array>
</resources>