Method: indexing.datasources.items.poll

Polls for unreserved items from the indexing queue and marks a set as reserved, starting with items that have the oldest timestamp from the highest priority ItemStatus. The priority order is as follows:

ERROR

MODIFIED

NEW_ITEM

ACCEPTED

Reserving items ensures that polling from other threads cannot create overlapping sets.

After handling the reserved items, the client should put items back into the unreserved state, either by calling index, or by calling push with the type REQUEUE.

Items automatically become available (unreserved) after 4 hours even if no update or push method is called.

This API requires an admin or service account to execute. The service account used is the one whitelisted in the corresponding data source.

HTTP request

POST https://cloudsearch.googleapis.com/v1/indexing/{name=datasources/*}/items:poll

The URL uses gRPC Transcoding syntax.

Path parameters

Parameters
name

string

The name of the Data Source to poll items. Format: datasources/{sourceId}

Request body

The request body contains data with the following structure:

JSON representation 
{
  "connectorName": string,
  "statusCodes": [
    enum (ItemStatus.Code)
  ],
  "limit": integer,
  "queue": string,
  "debugOptions": {
    object (DebugOptions)
  }
}
Fields
connectorName

string

The name of connector making this call.

Format: datasources/{sourceId}/connectors/{ID}
statusCodes[]

enum (ItemStatus.Code)

Limit the items polled to the ones with these statuses.
limit

integer

Maximum number of items to return.

The maximum value is 100 and the default value is 20.
queue

string

Queue name to fetch items from. If unspecified, items.poll will fetch from 'default' queue. The maximum length is 100 characters.
debugOptions

object (DebugOptions)

Common debug options.

Response body

If successful, the response body contains data with the following structure:

JSON representation 
{
  "items": [
    {
      object (Item)
    }
  ]
}
Fields
items[]

object (Item)

Set of items from the queue available for connector to process.

These items have the following subset of fields populated:

version

metadata.hash

structuredData.hash

content.hash

payload

status

queue

Authorization Scopes

Requires one of the following OAuth scopes:

  • https://www.googleapis.com/auth/cloud_search.indexing
  • https://www.googleapis.com/auth/cloud_search

For more information, see the Authorization guide.