Custom Search

Creating Custom Search Engines programatically

This page documents request methods for programmers who want to write client applications that dynamically create custom search engines (CSE) or interact with Custom Search. While pages in the Developer's Guide do not presume any programming knowledge, this page assumes that you know how to program and are familiar with Hypertext Transfer Protocol (HTTP).

Contents

Overview

The Custom Search API lets you skip the control panel altogether and use a client application of your choice to retrieve a list of custom search engines (CSE) under an account, create and delete custom search engines, and retrieve code for a search box, among other things.

The API uses standard HTTP requests, such as: GET, POST, and DELETE. All requests must have an Authorization header that contains an authentication token (described in the next section). In response, the Custom Search server sends an HTTP status code (such as a 200-type status for success or 400-type status for failure) that reflects the result of each request. You can use any programming language that lets you issue HTTP requests and parse XML-based responses.

The process of programmatically creating search engines involves the following steps:

  1. Create the custom search engine specification or context XML. To learn more, see Context: Defining a Search Engine.
  2. Create the annotations XML to let Custom Search know which sites to search or ignore, and how to rank the search results. To learn more, see Annotations: Selecting Sites.
  3. Get the authentication token from the Account Authentication API. To learn more, see Authentication.
  4. Submit the context XML by sending an authenticated request to the Custom Search server. To learn more, see Creating and Updating the Search Engine Specification.
  5. Send another authenticated request to the Custom Search server to submit the annotations XML. To learn more, see Creating and Updating Annotations.

Once you've created your search engine, you have two options for displaying it on your site:

  • Google Custom Search element. To display the search box and search results on your website, you can copy and paste a snippet of code into your pages. To learn more, see Retrieving the Code for the Search Box and Retrieving the Code for the Search Results. The Custom Search element provides three methods for designing the look and feel of search results. You can:
    • Create a custom result structure. Using Custom Search element Data Rendering, you can write HTML to include rich snippets and other custom data attributes. More information..
    • Modify result CSS properties. The Custom Search element uses Search Control result styles, which you can modify to change basic look and feel.
    • Use a predefined theme. Google offers several predefined themes so you can customize your look and feel without having to change any code or markup.
  • Use the Google WebSearch service to retrieve results in XML format. (Note: This service is available only to customers of Google Site Search.) The WebSearch service uses a simple HTTP-based protocol to serve search results. Search administrators have complete control over the way they request search results and the way they present those results to the end user. To retrieve custom search reuslts, your application sends Google a simple HTTP request. Google then returns search results in XML format. XML-formatted results give you the ability to customize the way search results are displayed. More information.

 

Back to top

Authentication

Custom Search Engine IDs

Each Custom Search engine is identified by a unique ID created by combining a user ID with a Custom Search engine ID, separated by a colon, like this:

011737558837375720776:mbfrjmyam1g

In this case, the user ID is 011737558837375720776, and the search engine ID is mbfrjmyam.

Although you can set HTTP requests to the URL http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>, we recommend using the following URL instead:
http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>

You can find your Custom Search engine ID on the Basics tab of the Custom Search control panel.

We recommend that you always explicitly specify the USERID. If the authenticating user is the same as the CSE owner, you can replace USERID with default. However, if the authenticating user is not the same as the CSE owner, you must explicitly specify the USERID.

Authenticating

Like many Google services, Custom Search is protected by Google Accounts. This means that your client application must have an authentication token in order to interact with Custom Search. You can get authentication tokens from the Google Accounts Authentication Service, which is an API that processes requests for access and issues authentication tokens.

Although the Accounts Authentication API supports multiple authentication methods, Custom Search only supports ClientLogin, which is the interface that lets you incorporate programmatic login into your desktop or mobile applications. It also supports CAPTCHA for greater security against password trollers. When you make a request to ClientLogin, provide the service name for Custom Search, which is cprose.

If your client application is accessing Custom Search on behalf of other users, you must first get their explicit authorization. They can provide their usernames and passwords directly to your client or to Custom Search. The Accounts Authentication Service returns authentication tokens that your client can then send to the Custom Search service, and you must use the token with every subsequent request on behalf of your users. The tokens do expire after a while.

For an overview of authenticating with Google, see Google Data APIs Authentication Overview; for more detailed documentation, see the Account Authentication API; for general FAQ, see the Google Data APIs Frequently Asked Questions; for user discussions on the Google Accounts API, visit the Google Accounts API Group.

Back to top

Once you have acquired an authentication token, you can use it to create the Authorization header for each subsequent API request, as in the following:

Authorization: GoogleLogin auth=yourAuthToken

Here is an example of an authenticated GET request:

GET http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>
Authorization: GoogleLogin auth=IM6F7Cx2fo0TAiwlhNVdSE8Ov8hw6aHV

Back to top

You can create a script to authenticate call the Custom Search API. Here's an example of a script (cse-api.sh) that authenticates the user and then submits a URL for on-demand indexing.

HOST="www.google.com"

EMAIL="$1"
PASSWORD="$2"
USERID="$3"
CSEID="$4"

URL="http://$HOST/cse/api/$USERID/index/$CSEID"
ISPOST=1

AUTHTOKEN='curl -k https://www.google.com/accounts/ClientLogin -d Email=$EMAIL \
 -d Passwd=$PASSWORD -d accountType=GOOGLE -d service=cprose | grep -x "Auth=.*"'
AUTHTOKEN='echo $AUTHTOKEN | cut -d = -f 2'
AUTHHEADER="Authorization: GoogleLogin auth=$AUTHTOKEN"

TMP="post.xml"
echo "" > $TMP

XML="<OnDemandIndex>
<Pages>
 <Page url=\"http://www.example.com/\">
 </Page>
</Pages>
</OnDemandIndex>"

echo $XML >> $TMP;

echo "Posting to $URL"
echo "Header: $AUTHHEADER"
echo "Content:"
cat $TMP

curl -v -X POST -d @$TMP -H "$AUTHHEADER" -H "Content-Type:text/xml" $URL

Call this shell script as follows:

cse-api.sh myemail@example.com mypassword 006545903547994973528 ddkzxleq6ca

We recommend that you always explicitly specify the USERID. The USERID is the first part of the full CSE ID. For example, if your full CSE ID is 006545903547994973528:ddkzxleq6ca, your USERID is 006545903547994973528.

If the authenticating user is the same as the CSE owner, you can replace USERID with default, as shown in examples in this document. However, if the authenticating user is not the same as the CSE owner, you must explicitly specify the USERID.

Request Methods

The section describes the different request methods you can use to create and manage your search engines.

To create the search engine, you need to submit a context XML file and an annotations XML file.

If you do not know the ID for the custom search engine, you can retrieve the list of search engines and extract the CustomSearchEngine id value.

Back to top

Creating and Updating the Search Engine Specification

To create or update a context or search engine specification, send an authenticated POST request with the context XML in the message body to the following URL:

http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>

Note: If the search engine already exists, sending the request will replace the stored context XML with the submitted context XML.

The following is an example of a POST request that creates the search engine specification.

POST http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>my_first_cse
Content-Type: text/xml
Authorization: GoogleLogin auth="IM6F7Cx2fo0TAiwlhNVdSE8Ov8hw6aHV"
 <CustomSearchEngine keywords="cars" language="en">
   <Title>My First CSE</Title>
   <Description>Car Search</Description>
   <Context>
      <BackgroundLabels>
       <Label name="_cse_my_first_cse" mode="FILTER" />
       <Label name="_cse_exclude_my_first_cse" mode="ELIMINATE" />
     </BackgroundLabels>
   </Context>
   <LookAndFeel nonprofit="false" />
 </CustomSearchEngine>

If the request includes a valid XML, the Custom Search server responds with the Custom Search context XML that it stored. In some cases, it might be slightly different from the context XML that you sent.

<?xml version="1.0" encoding="UTF-8" ?>
  <CustomSearchEngine id="my_first_cse" keywords="cars" language="en">
    <Title>My First CSE</Title>
    <Description>Car Search</Description>
    <Context>
     <BackgroundLabels>
       <Label name="_cse_my_first_cse" mode="FILTER" />
       <Label name="_cse_exclude_my_first_cse" mode="ELIMINATE" />
     </BackgroundLabels>
    </Context>
    <LookAndFeel nonprofit="false" />
  </CustomSearchEngine>

Back to top

Creating and Updating Annotations

To add or remove annotations from the annotations file stored by the Custom Search server, send an authenticated POST request with the annotations XML in the message body to the following URL:

http://www.google.com/cse/api/<USER_ID>/annotations/<CSE_ID>

Note: Custom Search has a file size limit for annotations. To learn more about the limits, see the Annotations: Selecting Sites page.

You end only the annotations that you want to add or remove from the search engines. To modify an existing annotation, you can remove the annotation and add the modified version in the same message. Because Custom Search combines all the annotations for all search engines associated with your account into a single XML annotations file, you can add and remove annotations for multiple search engines in a single message.

Using the Batch Update XML

To uploading changes to the annotations file using HTTP methods, you must use the Batch root element and child elements in addition to the standard Annotations XML. The additional elements tell Custom Search whether your annotations should be added or removed.

The Batch root element can have two child sections, Add and Remove. The Batch element can contain either or both sections in the same message and has the following hierarchy:

  • Batch
    • Add
      • Annotations
        • Annotation
          • Label
          • Comment (optional)
    • Remove
      • Annotations
        • Annotation href

You can have only one Add section and one Remove section, each of which can have a single Annotations subsection with multiple Annotation elements.

Back to top

Adding Annotations

To add annotations to your account, use the Add element and populate it with annotations in the standard annotations XML format, as shown in the following:

<Annotations>
  <Annotation about="http://www.example.com/*">
    <Label name="_cse_example"/>
  </Annotation>
  <Annotation about="http://www.sample.net/*">
    <Label name="_cse_example"/>
  </Annotation>
</Annotations>

Let's say that you created another search engine for an account that already has two search engines, and you want to upload the annotations just for that search engine. You can send the following request.

POST http://www.google.com/cse/api/<USER_ID>/annotations/<CSE_ID>
Content-Type: text/xml
Authorization: GoogleLogin auth="IM6F7Cx2fo0TAiwlhNVdSE8Ov8hw6aHV"

<Batch>
  <Add>
    <Annotations>
      <Annotation about="http://www.solarenergy.org/*">
        <Label name="_cse_solar_example"/>
      </Annotation>
      <Annotation about="http://www.solarfacts.net/*">
        <Label name="_cse_solar_example"/>
      </Annotation>
      <Annotation about="http://en.wikipedia.org/wiki/*">
        <Label name="_cse_exclude_solar_example"/>
      </Annotation>

   </Annotations>
  </Add>
</Batch>

Back to top

Removing Annotations

The XML for removing annotations is slightly different. Instead of using th full annotation format with URL and label names, you just need to use the annotations ID or href id. Custom Search tags each annotation with an href id. To get this ID, retrieve all the annotations and extract the href id of the annotations you want to delete.

The following is a code snippet that removes annotations.

<Remove>
  <Annotations>
    <Annotation href="Cg93d3cuZ29vZ2xlLmNvbS8Qrsq3gv456gRI" />
    <Annotation href="Cg93d3cuZ29vZ2xlLmNvbS8Qzsq3gvWd23" />
  </Annotations>
</Remove>

Back to top

Making Multiple Changes

You can also send a batch update with both Add and Remove elements, as in the following example.

POST http://www.google.com/cse/api/<USER_ID>/annotations/<CSE_ID>
Content-Type: text/xml
Authorization: GoogleLogin auth="IM6F7Cx2fo0TAiwlhNVdSE8Ov8hw6aHV"
<Batch>
  <Add>
    <Annotations>
      <Annotation about="www.cnn.com/*" score="1.0">
        <Label name="test" />
        <Comment>News</Comment>
      </Annotation>
      <Annotation about="www.youtube.com/*" score="0.5">
        <Label name="test" />
        <Comment>Videos</Comment>
      </Annotation>
    </Annotations>
  </Add>
  <Remove>
     <Annotations>
       <Annotation href="Cg93d3cuZ29vZ2xlLmNvbS8Qrsq3gv456gRI" />
       <Annotation href="Cg93d3cuZ29vZ2xlLmNvbS8Qzsq3gvWd23" />
     </Annotations>
  </Remove>
</Batch>

If the request includes valid XML, the Custom Search server responds with the annotations XML it will store.

<Batch>
  <Add>
    <Annotations>
      <Annotation about="www.cnn.com/*" score="1" comment="News"
                  timestamp="0x00045d3da6b1d7fd" href="Cg13d3cuY25uLmNvbS8qEP2vx7Xap5cC">
      <Label name="test" />
      </Annotation>
      <Annotation about="www.youtube.com/*" score="0.5" comment="Videos"
                  timestamp="0x00045d3da6b1d81a" href="ChF3d3cueW91dHViZS5jb20vKhCasMe12qeXAg">
        <Label name="test" />
      </Annotation>
    </Annotations>
  </Add>
  <Remove>
    <Annotations>
      <Annotation href="Cg93d3cuZ29vZ2xlLmNvbS8Qrsq3gv456gRI" />
      <Annotation href="Cg93d3cuZ29vZ2xlLmNvbS8Qzsq3gvWd23" />
    </Annotations>
  </Remove>
</Batch>

Back to top

Creating and Updating Promotions

To selectively add or remove promotions from the promotions file stored by the Custom Search server, send an authenticated POST request with the promotions XML in the message body to the following URL:

http://www.google.com/cse/api/<USER_ID>/promotions/<CSE_ID>

Note: Custom Search has a file size limit for promotions. To learn more about the limits, see the Promotions page.

You send only the promotions that you want to add or remove from the search engine. To modify an existing promotion, remove it and add the modified version in the same message.

Using the Batch Update XML

To upload changes to the promotions file using HTTP methods, you must use the Batch root element and child elements in addition to the standard Promotions XML. The additional elements tell Custom Search whether the promotions should be added or removed.

The Batch element can have two child sections: Add and Remove. The Batch element can contain either or both sections in the same message and has the following hierarchy:

  • Batch
    • Add
      • Promotions
        • Promotion
    • Remove
      • Promotions
        • Promotion

You can have only one Add section and one Remove section, each of which can have a single Promotions subsection with multiple Promotion elements. More information about the Promotions XML format.

Back to top

Adding Promotions

To add promotions to your account, use the Add element and populate it with promotions in the standard promotions XML format, as shown in the following:

<Promotions>
  <Promotion id="0001"
             queries="american born chinese, American Born Chinese, abc, ABC"
             title="American Born Chinese"
             url="http://books.google.com/books?id=vawdZyrDw64C"
             description="Graphic novel. First-person account of growing up Asian American by Gene Luen Yang."
             image_url="http://146.74.224.231/archives/Gene%20Yang.jpg" />
</Promotions>

If you do not define the ID of the promotion, Custom Search automatically assigns an ID. Let's say that you want to add promotions with and without IDs. You can send the following request.

POST http://www.google.com/cse/api/<USER_ID>/promotions/<CSE_ID>
Content-Type: text/xml
Authorization: GoogleLogin auth="IM6F7Cx2fo0TAiwlhNVdSE8Ov8hw6aHV"
<Batch>
  <Add>
    <Promotions>
      <Promotion id="0001"
                 queries="american born chinese, American Born Chinese, abc, ABC"
                 title="American Born Chinese"
                 url="http://books.google.com/books?id=vawdZyrDw64C"
                 description="Graphic novel. First-person account of growing up Asian American by Gene Luen Yang."
                 image_url="http://146.74.224.231/archives/Gene%20Yang.jpg" />
      <Promotion queries="wanderer, the wanderer"
                 title="Groo the Wanderer"
                 url="http://www.groo.com/"
                 description="Comedy. American series illustrated by Sergio Aragonés."
                 image_url="http://www.newsfromme.com/images5/groo11.jpg" />
    </Promotions>
  </Add>
</Batch>

Back to top

Removing Promotions

The XML for removing promotions is slightly different. Instead of using the full promotion format with queries and results, you just need to use the promotions ID. Custom Search tags each promotion with an Promotions id. If you did not create an ID for each promotion, Custom Search automatically assigns one to them. To get this ID, retrieve the promotions and extract the Promotion id of the promotions you want to delete.

The following is a code snippet that removes promotions.

<Batch>
  <Remove>
     <Promotions>
       <Promotion id="comics00001"/>
       <Promotion id="comics00103"/>
     </Promotions>
  </Remove>
</Batch>

Back to top

Making Multiple Changes

You can also send a batch update with both Add and Remove elements, as in the following example.

POST http://www.google.com/cse/api/<USER_ID>/promotions/<CSE_ID>
Content-Type: text/xml
Authorization: GoogleLogin auth="IM6F7Cx2fo0TAiwlhNVdSE8Ov8hw6aHV"

<Batch>
  <Add>
     <Promotions>
       <Promotion id="0001"
                  queries="american born chinese, American Born Chinese, abc, ABC"
                  title="American Born Chinese"
                  url="http://books.google.com/books?id=vawdZyrDw64C"
                  description="Graphic novel. First-person account of growing up Asian American by Gene Luen Yang"
                  image_url="http://146.74.224.231/archives/Gene%20Yang.jpg" />
      <Promotion id="0003"
                 queries="wanderer, the wanderer"
                 title="Groo the Wanderer"
                 url="http://www.groo.com/"
                 description="Comedy. American series illustrated by Sergio Aragonés."
                 image_url="http://www.newsfromme.com/images5/groo11.jpg" />
    </Promotions>
  </Add>
  <Remove>
     <Promotions>
         <Promotion id="comics00001"/>
         <Promotion id="comics00103"/>
     </Promotions>
   </Remove>
</Batch>

If the request includes valid XML, the Custom Search server will respond with the promotions XML it will store. {# including the The response is the list of batch update operations including the href ids for the new annotations.#}

<Batch>
  <Add>
    <Promotions>
      <Promotion id="0001"
                 queries="american born chinese, American Born Chinese, abc, ABC"
                 title="American Born Chinese"
                 url="http://books.google.com/books?id=vawdZyrDw64C"
                 description="Graphic novel. First-person account of growing up Asian American by Gene Luen Yang."
                 image_url="http://146.74.224.231/archives/Gene%20Yang.jpg" /></Promotions>
      <Promotion queries="wanderer, the wanderer"
                 title="Groo the Wanderer"
                 url="http://www.groo.com/"
                 description="Comedy. American series illustrated by Sergio Aragonés."
                 image_url="http://www.newsfromme.com/images5/groo11.jpg" />
     </Promotions>
  </Add>
  <Remove>
     <Promotions>
       <Promotion id="comics00001"/>
       <Promotion id="comics00103"/>
     </Promotions>
  </Remove>
</Batch>

Back to top

Creating and Updating Synonyms

To selectively add or remove synonyms from the synonyms file stored by the Custom Search server, send an authenticated POST request with the synonyms XML in the message body to the following URL:

http://www.google.com/cse/api/<USER_ID>/synonyms/<CSE_ID>

Note: Custom Search has a file size limit for synonyms. To learn more about the limits, see the Improving User Queries for More Relevant Results page.

You send only the synonyms that you want to add or remove from the search engine. If you want to modify an existing synonym, you can remove it and add the modified version in the same message.

Using the Synonym Batch Update XML

When you are uploading changes to the synonyms file using HTTP methods instead of the control panel, you need to use the Batch root element and child elements in addition to the standard synonyms XML. The additional elements tell Custom Search what to do with the synonyms— whether they should be added or removed.

The Batch element can have two child sections: Add and Remove. The Batch element can contain either or both sections in the same message and has the following hierarchy:

  • Batch
    • Add
      • Synonyms
        • Synonym
    • Remove
      • Synonyms
        • Synonym

You can have only one Add section and one Remove section, each of which can have a single Synonyms subsection with multiple Synonym child elements. For a more detailed discussion of the synonyms XML format, see the Improving User Queries for More Relevant Results page.

Back to top

Adding and Updating Synonyms

To add synonyms to your account, use the Add element and populate it with synonyms in the standard synonyms XML format, as shown in the following:

<Batch>
   <Add>
     <Synonyms>
       <Synonym term="American football">
         <Variant>grid iron</Variant>
         <Variant>tackle football</Variant>
       </Synonym>
       <Synonym term="racquet sport">
         <Variant>badminton</Variant>
         <Variant>squash</Variant>
         <Variant>racquetball</Variant>
       </Synonym>
    </Synonyms>
  </Add>
</Batch>

Back to top

Updating Synonyms

If you want to add synonym variants to a term, include the existing variants and the additional variants, not just the new variants. When you add a synonym term to a search engine that already has that term, Custom Search replaces the old synonym-term definitions with a new one. It does not merge the content. The following code shows how you can add more variants to the synonym term, racquet sport, which was added in the previous example.

<Batch>
   <Add>
     <Synonyms>
       <Synonym term="racquet sport">
         <Variant>badminton</Variant>
         <Variant>squash</Variant>
         <Variant>racquetball</Variant>
         <Variant>tennis</Variant>
         <Variant>speedminton</Variant>
       </Synonym>
    </Synonyms>
  </Add>
</Batch>

Removing Synonyms

The XML for removing synonyms is slightly different. Instead of using the full synonym format with the search term and its search variants, you just need to list the search term, that is, just the Synonym word element.

The following shows code than removes synonyms.

<Batch>
  <Remove>
    <Synonyms>
      <Synonym term="clown" />
      <Synonym term="ninja"/>
      <Synonym term="monkey"/>
    </Synonyms>
  </Remove>
</Batch>

Back to top

Making Multiple Changes

You can also send a batch update with both Add and Remove elements, as in the following example.

POST http://www.google.com/cse/api/<USER_ID>/synonyms/<CSE_ID>
Content-Type: text/xml
Authorization: GoogleLogin auth="IM6F7Cx2fo0TAiwlhNVdSE8Ov8hw6aHV"

<Batch>
  <Add>
    <Synonyms>
      <Synonym term="cootie">
        <Variant>bacteria</Variant>
        <Variant>virus</Variant>
        <Variant>germs</Variant>
        <Variant>bugs</Variant>
      <Synonym>
    </Synonyms>
  </Add>
  <Remove>
    <Synonyms>
      <Synonym term="dingdong ditch" />
    </Synonyms>
  </Remove>
</Batch>

If the request includes valid XML, the Custom Search server will respond with the synonyms XML it will store.

Back to top

Deleting a Search Engine

To delete a search engine, send an authenticated DELETE request to the following URL:

http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>

Warning: Verify that you have the correct CSE ID because you cannot undo this operation.

The following is an example of a DELETE request.

DELETE http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>/my_first_cse
Authorization: GoogleLogin auth="IM6F7Cx2fo0TAiwlhNVdSE8Ov8hw6aHV"

Back to top

Retrieving Components of the Search Engines

You can retrieve the following:

Retrieving a List of Search Engines

An account can have more than one search engine. To retrieve the complete list of custom search engines for the account, send an authenticated GET request to the following URL:

http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>

The following is an example of a GET request that retrieves the entire list of search engines.

GET http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>
Authorization: GoogleLogin auth="IM6F7Cx2fo0TAiwlhNVdSE8Ov8hw6aHV"

The Custom Search server responds with a list of of custom search engine IDs and titles. The IDs are unique across all accounts, but the title is not. No other account owner has the same creator ID, but multiple account owners can have search engines with the same title.

<?xml version="1.0" encoding="UTF-8" ?>
  <CustomSearchEngines>
    <CustomSearchEngine id="my_first_cse" creator="0123456789" title="My First CSE" />
    <CustomSearchEngine id="my_second_cse" creator="0123456789" title="My Second CSE" />
  </CustomSearchEngines>

Back to top

Retrieving the Specifications of a Search Engine

To retrieve the context XML of a specific search engine, send an authenticated GET request to the following URL:

http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>

The following is an example of a GET request that retrieves the context XML of a particular search engine.

GET http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>/my_first_cse
Authorization: GoogleLogin auth="IM6F7Cx2fo0TAiwlhNVdSE8Ov8hw6aHV"

The Custom Search server responds with the context XML.

<?xml version="1.0" encoding="UTF-8" ?>
  <CustomSearchEngine id="my_first_cse" creator="0123456789" keywords="cars" language="en">
    <Title>My First CSE</Title>
    <Description>Car Search</Description>
    <Context>
      <BackgroundLabels>
        <Label name="_cse_my_first_cse" mode="FILTER" />
        <Label name="_cse_exclude_my_first_cse" mode="ELIMINATE" />
      </BackgroundLabels>
    </Context>
    <LookAndFeel nonprofit="false" />
  </CustomSearchEngine>

Back to top

Retrieving All Annotations

To retrieve all annotations for all search engines in an account, send an authenticated GET request to the following URL:

http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>/annotations/

The following is an example of a GET request that retrieves the annotations XML in its entirety.

GET http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>/annotations/
Authorization: GoogleLogin auth="IM6F7Cx2fo0TAiwlhNVdSE8Ov8hw6aHV"

The Custom Search server responds with the entire list of annotations.

<?xml version="1.0" encoding="UTF-8" ?>
  <Annotations start="0" num="2">
    <Annotation about="*.cnn.com/*" score="1" timestamp="0x0004545f2e9f36cf" href="CgsqLmNubi5jb20vKhDP7fz08ouVAg">
      <Label name="_cse_hg1shdcqrjs"/>
      <AdditionalData attribute="original_url" value="cnn.com/*"/>
    </Annotation>
    <Annotation about="*.google.com/*" score="1" timestamp="0x0004585ba0c61ce4" href="Cg4qLmdvb2dsZS5jb20vKhDkuZiGuouWAg">
      <Label name="_cse_qcfwlffvvrg"/>
      <AdditionalData attribute="original_url" value="google.com/*"/>
    </Annotation>
  </Annotations>

Back to top

Retrieving XML Feeds of Search Results (Site Search only)

If you are Site Search customer or a Custom Search partner, you can retrieve the XML feeds for your search results by sending a GET request to the following URL:

http://www.google.com/cse?<CSE ID>&q=<query term>&output=xml

Note: This is not available to regular Custom Search accounts.

The following is an example of a GET request that retrieves the results XML for the query "Einstein."

GET https://www.google.com/cse?cx=999999999999999999999:srraaaaaaaa&q=einstein&output=xml

The Custom Search server responds with the results XML. If the webpage has structured data, the data is included.

<GSP VER="3.2">
  <TM>0.405150</TM>
  <Q>einstein</Q>
  <PARAM name="cx" value="cx=999999999999999999999:srraaaaaaaa" original_value="cx=999999999999999999999:srraaaaaaaa"/>
  <PARAM name="q" value="einstein" original_value="einstein"/>
  <PARAM name="output" value="xml" original_value="xml"/>
  <PARAM name="adkw" value="AELymgUxINzkwsr2ClnvOZ4bodqf4MPdK634hY
    kFSIGs9-bHCGP_obrUjCfugy_8JPaHjXINX6NJt_fOotOsBkrtj9OxtZAWHgc2a2jRPl3OfKVbQgAluMA"
    original_value="AELymgUxINzkwsr2ClnvOZ4bodqf4MPdK634hYkFSIGs9-bHCGP_obrUjCfugy_8JPaHjXINX6NJt_fOotOsBkrtj9OxtZAWHgc2a2jRPl3OfKVbQgAluMA"/>
  <PARAM name="hl" value="en" original_value="en"/>
  <PARAM name="oe" value="UTF-8" original_value="UTF-8"/>
  <PARAM name="ie" value="UTF-8" original_value="UTF-8"/>
  <PARAM name="client" value="google-csbe" original_value="google-csbe"/>
  <PARAM name="boostcse" value="0" original_value="0"/>
  −
  <Context>
    <title>Genius Search Engine</title>
  </Context>
  −
    <RES SN="1" EN="10">
    <M>569000</M>
    −
    <NB>
    −
      <NU>
       /custom?q=einstein&hl=en&client=google-csbe&cx=cx=999999999999999999999:srraaaaaaaa
       &boostcse=0&output=xml&ie=UTF-8&
       oe=UTF-8&start=10&sa=N
      </NU>
    </NB>
    <RG START="1" SIZE="10"/>
    <RG START="1" SIZE="1"/>
    −
    <R N="1">
    −
      <U>
        http://www.scribd.com/doc/48868/Albert-Einstein-The-World-as-I-See-It
      </U>
      −
      <UE>
      http://www.scribd.com/doc/48868/Albert-Einstein-The-World-as-I-See-It
      </UE>
      <T>Albert <b>Einstein</b> - The World as I See It</T>
      <RK>0</RK>
      −
      <S>
      May 10, 2007 <b>...</b> The collection of thoughts on various
      scenarios by Albert Eienstein Religion <br>  <b>Einstein</b>
      Albert Religion-Christianity Business-Asian.
      </S>
      <LANG>en</LANG>
      <Label>_cse_srraacmywxi</Label>
      −
      <PageMap>
      −
        <DataObject type="document">
            <Attribute name="title">Albert Einstein &ndash;The World as I See It</Attribute>
            <Attribute name="author">pablasarabjot</Attribute>
            <Attribute name="description">
            The collection of thoughts on various scenarios by Albert Eienstein
            </Attribute>
            ...
            <Attribute name="thumbnail_url">
            http://i5.scribdassets.com/profiles/images/dbw8y5a0etg2a-thumb.jpg
            </Attribute>
            <Attribute name="filetype_image">
            http://www.scribd.comhttp://s7.scribdassets.com/images/filetypes/pdf_16x16.gif?1256049874
            </Attribute>
        </DataObject>
      </PageMap>
....

Back to top

Retrieving Promotions

To retrieve promotions of a search engine, send an authenticated GET request to the following URL:

http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>/promotions/

The following is an example of a GET request that retrieves the promotions XML

GET http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>/promotions/
Authorization: GoogleLogin auth="IM6F7Cx2fo0TAiwlhNVdSE8Ov8hw6aHV"

The Custom Search server responds with the entire list of promotions.

<?xml version="1.0"?>
  <Promotions cse_id="2b-oaog9aqg" start="0" num="3"
        total="3">
    <Promotion id="p1" queries="chocolate" title="Best Chocolate
        for People with Taste" url="http://www.example.com/best/chocolate"/>
    <Promotion id="7c3481ba" queries="chocolate" title="The Dangerous
        Crimes of Chocolate Addicts" url="http://www.example.com/books/chocolate/danger.html"/>
    <Promotion id="5139e65a" queries="candy cane" title="Christmas Candies in Summer"
        url="http://www.example.com/candycane.asp"
        image_url="http://images.example.com/candycane.jpg"/>
  </Promotions>

You can use parameters to retrieve a range of promotions. If you do not use a paramater, Custom Search API uses the default values.

The following is an example of a GET request with parameters.

GET http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>/promotions/?start=2&num=5
Parameter Description Value Default value
start

Indicates the first promotion that you want to retrieve from a list.

 

The index of the first element of the range. The order begins from 0. For example, to retrieve a list of promotions starting from the second promotion in the XML, send the following request:

GET http://.../promotions/<cse_ID>?start=1
          

 

0. The API returns the first promotion listed in the XML.
num

Indicates the number of promotions that you want to retrieve from a list.

 

The number of promotions in the range. For example, to get eight promotions, including the starting promotion, send the following request:

GET http://.../promotions/<cse_ID>?num=8

20. The API returns up to 20 promotions, beginning from the first element you defined in start. If you did not define start either, Custom Search retrieves the first 20 promotions.

Back to top

Retrieving Synonyms

To retrieve synonyms of a search engine, send an authenticated GET request to the following URL:

http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>/synonyms/

The following is an example of a GET request that retrieves the synonyms XML.

GET http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>/synonyms/
Authorization: GoogleLogin auth="IM6F7Cx2fo0TAiwlhNVdSE8Ov8hw6aHV"

The Custom Search server responds with the entire list of synonyms.

<?xml version="1.0"?>
  <Synonyms>
    <Synonym term="cootie">
      <Variant>bacteria</Variant>
      <Variant>virus</Variant>
      <Variant>germs</Variant>
      <Variant>bugs</Variant>
    </Synonym>
    ...
  </Synonyms>

Retrieving the Code for the Search Box

If you are using a Search element with a full-width or compact layout, you only need to retrieve the code for the search box. The search results are displayed directly under the search box, in the same webpage, as soon as your users send their queries. However, if you are using a Search element with a two-column layout, you need to retrieve the code for both the search box and the search results. For more information about Search element layouts, see Look and Feel .

If you are using a Google-hosted search engine, you only need to retrieve the code for the search box. The search results page is hosted by Google, so you do not need to handle it.

To get the code for the search box that you can insert into an HTML page, send an authenticated GET request to the following URL:

http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>/code/searchbox

The following is an example of a GET request that retrieves the search box code for a particular search engine.

GET http://www.google.com/cse/api/<IM6F7Cx2fo0TAiwlhNVdSE8Ov8hw6aHV>/cse/<tubigakawal>/
/code/searchbox
Authorization: GoogleLogin auth="IM6F7Cx2fo0TAiwlhNVdSE8Ov8hw6aHV"

If the search engine is a Search element, the Custom Search server responds with the something like the following:

<div id="cse" style="width: 100%;">Loading</div>
<script src="http://www.google.com/jsapi" type="text/javascript"></script>
<script type="text/javascript">
  google.load('search', '1', {language : 'en'});
  google.setOnLoadCallback(function(){
    var customSearchControl = new google.search.CustomSearchControl('012345678999012345678:tubigakawal');
    customSearchControl.setResultSetSize(google.search.Search.SMALL_RESULTSET);
    customSearchControl.draw('cse');
  }, true);
</script>

<link rel="stylesheet" href="http://www.google.com/cse/style/look/greensky.css" type="text/css" />
<style type="text/css">
  .gsc-control-cse {
    font-family: &quot;Trebuchet MS&quot;, sans-serif;
    border-color: #E1F3DA;
    background-color: #E1F3DA;
  }
  input.gsc-input {
    border-color: #94CC7A;
  }
  input.gsc-search-button {
    border-color: #94CC7A;
    background-color: #AADA92;
  }
  .gsc-tabHeader.gsc-tabhInactive {
    border-color: #A9DA92;
    background-color: #FFFFFF;
  }
  .gsc-tabHeader.gsc-tabhActive {
    border-color: #A9DA92;
    background-color: #A9DA92;
  }
  .gsc-tabsArea {
    border-color: #A9DA92;
  }
  .gsc-webResult.gsc-result {
    border-color: #A9DA92;
    background-color: #FFFFFF;
  }
  .gsc-webResult.gsc-result:hover {
    border-color: #A9DA92;
    background-color: #FFFFFF;
  }
  .gs-webResult.gs-result a.gs-title:link,
  .gs-webResult.gs-result a.gs-title:link b {
    color: #0066CC;
  }


...

}</style>

If your search engine is not a Search element, the Custom Search server returns something more like the following:

<form action="http://www.google.com/cse" id="tubigakawal" target="_blank">
  <div>
    <input type="hidden" name="cx" value="0123456789:my_first_cse" />
    <input type="hidden" name="ie" value="UTF-8" />
    <input type="text" name="q" size="31" />
    <input type="submit" name="sa" value="Search" />
  </div>
</form>

<script type="text/javascript" src="http://www.google.com/cse/cse/brand?form=cse-search-box&amp;lang=en"></script>

Back to top

Retrieving the Code for the Search Results

You only need to retrieve the code for search engines using the Custom Search Element. All other search engines use only the code for the searchbox.

To get the code for the search results that you can insert into an HTML page, send an authenticated GET request to the following URL:

http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>/code/searchresults

The following is an example of a GET request that retrieves the search results code for a particular search engine.

GET http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>my_first_cse/code/searchresults
Authorization: GoogleLogin auth="IM6F7Cx2fo0TAiwlhNVdSE8Ov8hw6aHV"

The Custom Search server responds with the following:

<div id="cse" style="width:100%;"></div>

Back to top

Associating a Search Engine with an AdSense Account

You can associate a search engine with an AdSense account by adding the AdSense element in the context XML specification, as described in Making Money. Send the authenticated POST request to http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>. For a sample of code, see Creating and Updating the Search Engine Specification section.

To retrieve the publisher ID and channel ID of the AdSense account, you could use the AdSense API, but it requires you to apply for review. You must meet the main requirements for using the AdSense API, which includes the following:

  • You must have at least 100,000 daily page views.
  • You must have an approved AdSense account.

To learn more, see the AdSense API review process.

Back to top

Submitting on-demand indexing requests

To submit an on-demand indexing request, send an authenticated POST request with the context XML in the message body to the following URL:

http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>/index/

The following is an example of a POST request that submits a request for instant indexing of URLs included in a Sitemap:

POST http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>/index/
Content-Type: text/xml
Authorization: GoogleLogin auth="IM6F7Cx2fo0TAiwlhNVdSE8Ov8hw6aHV"

<?xml version="1.0" encoding="UTF-8" ?>
<OnDemandIndex>
  <Sitemap url="http://www.example.com/sitemap.xml" />
</OnDemandIndex>

The following is an example of a POST request that submits a request for instant indexing of individual URLs:

POST http://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>/index/
Content-Type: text/xml
Authorization: GoogleLogin auth="IM6F7Cx2fo0TAiwlhNVdSE8Ov8hw6aHV"

<?xml version="1.0" encoding="UTF-8" ?>
<OnDemandIndex>
  <Pages>
    <Page url="http://www.example.com/new_page1.html" />
    <Page url="http://www.example.com/new_page2.html" />
    <Page url="http://www.example.com/outdated.html" removal="true" />
  </Pages>
</OnDemandIndex>

The following is an example of a POST request that submits request for instant indexing of individual URLs and includes PageMap data for each URL:

POST hhttp://www.google.com/cse/api/<USER_ID>/cse/<CSE_ID>/index/
Content-Type: text/xml
Authorization: GoogleLogin auth="IM6F7Cx2fo0TAiwlhNVdSE8Ov8hw6aHV"

<?xml version="1.0" encoding="UTF-8" ?>
<OnDemandIndex>
<Pages>
  <Page url="http://www.example.com/monkeys/">
    <pagemap>
      <DataObject type="document">
        <Attribute name="title">Monkeys</Attribute>
        <Attribute name="review">4.0/5.0</Attribute>
      </DataObject>
      <DataObject type="stats">
        <AccessKey>12345</AccessKey>
        <Attribute name="installs">1000</Attribute>
        <Attribute name="comments">100</Attribute>
      </DataObject>
    </pagemap>
  </Page>
  <Page url="http://www.example.com/parrots">
    <pagemap>
      <DataObject type="document">
        <Attribute name="title">Parrots</Attribute>
        <Attribute name="review">4.5/5.0</Attribute>
      </DataObject>
      <DataObject type="stats">
        <AccessKey>12345</AccessKey>
        <Attribute name="installs">2000</Attribute>
        <Attribute name="comments">200</Attribute>
      </DataObject>
    </pagemap>
  </Page>
</Pages>
</OnDemandIndex>

You may receive a reply like this if your request was submitted successfully:

<?xml version="1.0" encoding="UTF-8" ?>
<OnDemandIndex indexing_status="PENDING" quota_available="9992" refresh_status="SUCCESS"
  quota="10000" removal_quota_available="994" removal_quota="1000">
  <Sitemap url="http://www.example.com/sitemap.xml" />
</OnDemandIndex>

Pay special attention to refresh_status. If it is not SUCCESS, it means your submission was not successful. All other fields are about your previous on-demand indexing status that you can ignore.

Here is a list of common refresh_status error codes:

refresh_status Description
SUCCESS Your request has been submitted.
RATE_ERROR Your on-demand indexing requests were too frequent. We currently only allow one request per day.
SITEMAP_ERROR You have submitted a bad Sitemap URL.
SITEMAP_UNVERIFIED_ERROR You have submitted a Sitemap URL that does not belong to a website you have verified through Webmaster Tools.
PAGES_ERROR Your submitted individual URLs have some invalid URLs. You get a list of them in Pages element. Note in this case the whole submission fails, not just those invalid URLs.

Example reply when Sitemap submission failed:

<?xml version="1.0" encoding="UTF-8" ?>
<OnDemandIndex indexing_status="PENDING" quota_available="9992"
  refresh_status="SITEMAP_UNVERIFIED_ERROR" quota="10000" removal_quota_available="994" removal_quota="1000">
  <Sitemap url="http://www.example.com/sitemap.xml" />
</OnDemandIndex>

Example reply when you have invalid URLs in your submission:

<?xml version="1.0" encoding="UTF-8" ?>
<OnDemandIndex indexing_status="PENDING" quota_available="9992"
  refresh_status="PAGES_ERROR" quota="10000" removal_quota_available="994" removal_quota="1000">
  <Sitemap url="http://www.example.com/sitemap.xml" />
  <invalid_url>http://www.SiteYouDontOwn.com/new_page1.html</invalid_url>
  <invalid_url>Bad url</invalid_url>
</OnDemandIndex>

A few things to note:

  • Your submitted Sitemap or individual URLs must belong to a website that you have verified through Google Webmaster Tools. e.g., if you have verified site www.example.com through webmaster tool, then you can submit Sitemaps like
    • http://www.example.com/sitemap.xml
    • http://www.example.com/subpath/sitemap.xml
    but not
    • http://www.google.com/sitemap.xml
    • http://example.com/sitemap.xml

    To learn more, see about site verification.

  • There is a delay between the time when your Sitemap is submitted and the time when it is processed. Therefore, on-demand indexing API returns SUCCESS once a Sitemap URL from one of your verified sites is submitted. You must make sure that the Sitemap URL exists and can be crawled by Google.

Back to top

Status Codes

When you send a request to the Custom Search API, it will return an HTTP status code that reflects the result of the request. The following tables describes the status codes.

Status Code for Success

Status Code Description
200 - Success Your request has succeeded.
204 - No content Your delete request has succeeded.

Back to top

Status Code for Failure

Status Code Issue Solution
400 - Bad request The request has syntax error.

Check the URL in the request. Make sure that the CSE ID is correct.

If the URL is correct, review your code and make changes to the request before resending it.

If you are submitting context or annotations XML, make sure that the XML is valid.

401 - Authorization failure The request does not have an authentication token or has an incorrect authentication token for the user (for example, using an authentication token for another Google service).

All your requests must be authenticated with an authentication token (as discussed in the Authentication section).

The authentication token must be for the correct Google service, cprose, and for the right user.

403 - Forbidden You cannot add more annotations to the account because you have reached the limit for annotations.

Remove annotations or, if possible, combine some annotations by using broader URL patterns. To learn more about URL patterns, see the Help Center.

The annotations limit is documented on the Selecting Sites to Search page.

404 - Resource not found The request is not being sent to the right URL.

Check the URL in the request. Make sure that the CSE ID is correct.

If you are certain that the URL is correct, the Custom Search server might be temporarily unavailable. You can try again later.

405 - Method not allowed Custom Search API does not support the method. The API supports only the following methods: GET, POST, and DELETE.
413 - File too large The XML that is being submitted exceeds the Custom Search limit.

Verify that your file is not exceeding the size limit, which is documented on the Selecting Sites to Search page and the Promotions page.

If you are certain that the file size doesn't exceed the limit, try again later.

500 - Server error The Custom Search server is down. Wait 60 seconds, and then resubmit your request.

Back to top

Authentication required

You need to be signed in with Google+ to do that.

Signing you in...

Google Developers needs your permission to do that.