Content API for Shopping

Developer's Guide: Java

Google provides the Google API Client Library for Java for connecting to the Content API for Shopping server and interpreting the results. This client library is by no means the only way of connecting to the Content API for Shopping server. You can achieve the same result using a command-line tool, such as curl coupled with an XML parser.

The following document is intended to demonstrate how you can use the Google API Client Library for Java for accessing and editing your data.

Contents

  1. Audience
  2. Scope
  3. Introduction
  4. Setup
  5. Modelling data representation classes
    1. Summary
    2. Advanced topics and troubleshooting
  6. Performing authentication
    1. ClientLogin
    2. OAuth
  7. The HttpTransport object
  8. Managing items
    1. Inserting items
    2. Updating items
    3. Deleting items
    4. Retrieving items
  9. Handling errors
  10. Batching
    1. Creating a list of entries to be modified
    2. Sending the list of entries to be modified
    3. Error handling
    4. Comments
  11. Multi-threading
  12. The generic projection
  13. Downloading the complete example

Audience

This document assumes that you know Java programming, and that you are familiar with the basic concepts of the Google Content API for Shopping.

Scope

After reading this document you will know how to setup the Google API Client Library for Java, authenticate using ClientLogin or OAuth, and insert, update, delete and retrieve product items.

Introduction

The Google API Client Library for Java is a generic client library that can be used with almost all of Google's APIs. For each API you want to use it with, you will need to provide appropriate classes that represent the data relevant to the API.

As part of this documentation we provide an example of how you could do this. Bear in mind that this example is only a teaching example that is meant to show how the Client Library is used, but is not guaranteed to contain all data fields you might wish to access, nor is it guaranteed not to contain deprecated data fields.

Setup

This documentation and the downloadable examples have been written for and tested with version 1.5.0 of the Client Library. The latest version is not guaranteed to be backward compatible with this one. You can include the Client Library in your project by first downloading the .zip file. Then include the following .jar files in your Java project:

google-api-client-1.5.0-beta.jar
google-http-client-1.5.0-beta.jar
dependencies/xpp3-1.1.4c.jar
dependencies/guava-r09.jar
    

The Client Library setup instructions contain directions for Maven users.

You can now import the classes you will need using the following import statements:

import com.google.api.client.googleapis.*;
import com.google.api.client.http.*;
import com.google.api.client.xml.*;
import com.google.api.client.util.Key;
    

For a sample implementation, please refer to the downloadable examples.

Modelling data representation classes

One of the main tasks of the Google API Client Library is translating XML to Java classes and back. You need to provide these Java classes yourself and explain to the Client Library how to use them. The following chapter will show you how to do this.

For demonstration purposes we will model a Product class that represents one product entry. Let's have a look at what a simple product entry looks like in XML:

<entry>
  <title>Camera</title>
  <content type="text">A great compact body to make it easy to get a great shot everytime.</content>
  <sc:id>123456</sc:id>
  <link rel="alternate" type="text/html" href="http://www.replace-with-your-homepage.com/item1-info-page.html"/>
  <sc:image_link>http://www.example.com/image1.jpg</sc:image_link>
  <sc:target_country>US</sc:target_country>
  <sc:content_language>en</sc:content_language>
  <scp:brand>Acme</scp:brand>
  <scp:condition>new</scp:condition>
  <scp:price unit="usd">25</scp:price>
  <scp:quantity>3</scp:quantity>
  <scp:color>red</scp:color>
  <scp:color>blue</scp:color>
  <scp:tax>
    <scp:tax_country>US</scp:tax_country>
    <scp:tax_region>CA</scp:tax_region>
    <scp:tax_rate>8.25</scp:tax_rate>
    <scp:tax_ship>true</scp:tax_ship>
  </scp:tax>
  <scp:tax>
    <scp:tax_country>US</scp:tax_country>
    <scp:tax_region>926*</scp:tax_region>
    <scp:tax_rate>8.75</scp:tax_rate>
    <scp:tax_ship>false</scp:tax_ship>
  </scp:tax>
</entry>
    

In your Product class, you will therefore want to have fields for title, content, sc:id, link, sc:target_country, content_language, brand, condition, price, quantity, color and tax.

Some of these fields are simple data types like strings (title, sc:target_country, content_language, brand, condition) or numbers (sc:id, quantity), others are more complex like for example price, containing a number and a unit, or tax, which contains named fields itself (scp:tax_country, scp:tax_region, scp:tax_rate and scp:tax_ship).

Moreover, some fields can appear multiple times (like for example tax and color), so instead of a data field of the appropriate type you will want a List of that data type.

A very simple (but not yet usable) implementation of such a Product class would look like this:

// WARNING: not usable with Client Library because of missing @Key annotation
public class Product {
  public String title;
  public Content content;
  public String externalId;
  public List<Link> links = new ArrayList<Link>();
  public List<String> imageLinks;
  public String country = "US";
  public String lang = "en";
  public String brand;
  public String condition;
  public Price price;
  public Integer quantity;
  public List<String> colors;
  public List<Tax> taxes;
}

In order to allow the Google API Client Library to automatically translate this class into an XML representation, or parse the content of an XML representation into this class, we need to tell the Client Library which field corresponds to which XML element. We will do that using the @Key annotation.

(The @Key annotation is a normal Java annotation. If you haven't worked with Java annotations before, all you need to know for now is that you need to put something of the form @Key or @Key("some string") above a field in order to label it.)

You must import the declaration of the @Key using import com.google.api.client.util.Key; if you haven't done so already.

The @Key annotation takes one string argument. This string corresponds to the XML name of the element. If the argument is omitted, the name of the Java field itself is assumed to be the name of the XML element in the default Atom namespace.

A simple implementation of the Product class might therefore look as follows:

public class Product {
  @Key
  public String title;

  @Key
  public Content content;

  @Key("sc:id")
  public String externalId;

  @Key("link")
  public List<Link> links = new ArrayList<Link>();

  @Key("sc:image_link")
  public List<String> imageLinks;

  @Key("sc:target_country")
  public String country = "US";

  @Key("sc:content_language")
  public String lang = "en";

  @Key("scp:brand")
  public String brand;

  @Key("scp:condition")
  public String condition;

  @Key("scp:price")
  public Price price;

  @Key("scp:quantity")
  public Integer quantity;

  @Key("scp:color")
  public List<String> colors;

  @Key("scp:tax")
  public List<Tax> taxes;
}

For a complete list of all the fields please see the Product Item Requirements. For an implementation that contains all fields, please download the complete example.

Note that for the fields title and content we omitted the string parameter for @Key. This is equivalent to using @Key("title") and @Key("content"), respectively.

You probably have noted that some of the fields use custom data types, like Tax for the tax field, and Price for the price. These classes have to be modelled in the same way now. For Tax this is pretty straightforward with the methods we already have:

public class Tax {
  @Key("scp:tax_country")
  public String taxCountry;

  @Key("scp:tax_region")
  public String taxRegion;

  @Key("scp:tax_rate")
  public Float taxRate;

  @Key("scp:tax_ship")
  public Boolean taxShipping;
}

A slightly more interesting case is the implementation of Price. As you can see from the XML, price not only has a number, written between start and end tag, but also an attribute "unit". We model this as follows:

public class Price {
  @Key("@unit")
  public String unit;

  @Key("text()")
  public BigDecimal value;
}

If the string given to the @Key annotation begins with an @ symbol, the Client Library will consider it a value to be written within the open tag and named like the String given to the @Key annotation without the @. In the example above, the @Key("@unit") thus represents the unit="usd" part in <scp:price unit="usd">25</scp:price>.

If the @Key annotation gets the string "text()", the content of the field represents the content of the enclosing XML element, that is, the text between its open and close tag. This is named "text()" for strings as well as for other (plain) data types, like here for example for a BigDecimal. Note that a class representing an XML element can only have either a "text()" field or sub-elements (not beginning with an "@"), but never both.

Summary

Each field in a Java class that represents an XML element needs to be annotated in one of the following ways:

  • @Key("ns:element_name") for an element with the XML name ns:element_name.
  • @Key for an element with the XML name identical to the name of the field in the Java class.
  • @Key("@attribute_name") for an element attribute_name that appears within the open tag of the enclosing XML element.
  • @Key("text()") for the part between open and closing tag of the enclosing XML element. (This is named "text()" even for numbers and other data types.)
More information on the @Key annotation can be found in the corresponding Javadoc.

Each field in a Java class that represents an XML element needs to have the appropriate type, which can be:

  • A plain data type like String, Integer or BigDecimal.
  • Another Java class modelled as described above.
  • A collection of either of the two former options.
The field can be of any visibility (private, package private, protected, or public) and must not be static.

Advanced topics and troubleshooting

  • Providing a standard constructor

    Make sure to provide a constructor that takes no arguments. If you don't provide any constructors, this is implicitly done for you by Java. If you do provide at least one constructor however, Java will use only those constructors that are provided by you. In that case, make sure to provide a constructor that doesn't take any arguments, so that the Client Library can automatically create an instance of your class that it will parse the information from an XML into.

    For example, it might be useful to have a constructor for Price that takes two arguments, a string for the currency and a BigDecimal for the price. In that case we need to additionally provide an empty constructor without any arguments:

    public class Price {
      @Key("@unit")
      public String unit;
    
      @Key("text()")
      public BigDecimal value;
    
      public Price() {}
    
      public Price(String unit, BigDecimal value) {
        this.unit = unit;
        this.value = value;
      }
    }
        
  • Initializing data fields

    You will have noticed that some of the fields in the example are initialized with values or empty Collections. Depending on your use case you might or might not want to do this. When parsing XML into your class, the XML parser

    • will overwrite fields if they are given in the answer from the server,
    • will not change fields if they are missing in the answer from the server, and
    • will use initialized collections by adding elements rather than replacing them with new, empty collections.

  • Derived data representation classes

    You can use derived classes for representing XML elements. All @Key-annotated fields of the base class will also be available in the derived class. For example, product entries and housing entries both have a title and a content, so you might want to create a common base class Entry and derive your ProductEntry and HousingEntry classes from it.

    You can not define common fields in an Interface and derive from that, because by definition all fields in a Java Interface are static.

    Be very careful about using templates. The Client Library at the moment can not handle them well, though this might be added in future versions. For example, product feeds and housing feeds both contain entries, so you might be tempted to use one common base class Feed<T> containing a field List<T> entries, and derive a ProductFeed using ProductFeed extends Feed<ProductEntry>. The Client Library will in this case not be able to determine the correct type of entries and can not parse them.

  • Extending GenericXml

    So far, the data representation classes modelled with the methods above can only store data that is contained in the XML elements that it expects. There might be cases where a GET might return more data than that.

    If you want to store this additional data, you need to derive your data representation classes from com.google.api.client.xml.GenericXml. This will enable the class to store XML elements that are not associated to any field of the class, without changing the behaviour of the class otherwise. To extend Product from GenericXml, all you need to do is adding the derivation statement to the class declaration:

    public class Product extends GenericXml {
      ...
    }
        

    All XML elements that you provide @Key annotated fields for will still be parsed into those fields, and only additional data will be stored using the GenericXml infrastructure.

    Note: You need to do this for every class that might contain unexpected XML elements.

    A typical use case where this might be relevant is retrieving an item from the Content API for Shopping server, changing one of its values, and returning the updated item using an HTTP PUT command. Let's assume the item contains an XML element or XML attribute that your Product class is not prepared for. Without deriving Product from GenericXml, the Product class can not store this data, and the subsequent update request will accidentially delete the additional attribute while changing the value you actually wanted to change (because an update using HTTP PUT will always replace the entire product description with the new one). If you do derive from GenericXml however, the unexpected item will simply be stored as generic XML, and returned exactly the same way it was received.

    Although there are ways to access the data stored as GenericXml, for example using product.get("sc:id"), you should in general simply provide fields for all XML elements you want to access or modify, and just return the additional XML parts the way you received them.

Performing authentication

To insert items, you first need to be authenticated. You can authenticate using either ClientLogin or OAuth. For more information about authentication, see the Google Account Authentication documentation.

As a first step, we create a class for storing our user related constants:

public class UserInformation {
  public final String uid;
  public final ClientLoginCredentials clientLoginCredentials;

  public UserInformation(String uid, String username, String password) {
    this.uid = uid;
    this.clientLoginCredentials = new ClientLoginCredentials(username, password);
  }

  public static class ClientLoginCredentials {
    public final String username;
    public final String password;

    public ClientLoginCredentials(String username, String password) {
      this.username = username;
      this.password = password;
    }
  }
}
    

We can now create an instance that will hold our own values:

UserInformation userInformation = new UserInformation("1234567", "your.username@gmail.com", "yourPassword");
    

If you are planning to use OAuth, just leave the ClientLogin related constants blank.

ClientLogin

In our example, we authorize with ClientLogin using the following function:
  private static String authorizeUsingClientLogin(String username, String password)
      throws IOException {
    ClientLogin authenticator = new ClientLogin();
    authenticator.authTokenType = "structuredcontent";
    authenticator.username = username;
    authenticator.password = password;
    return authenticator.authenticate().auth;
  }

As you can see, first the ClientLogin authenticator is set up with username and password. The method authenticate() will internally create a temporary transport object, send the request, and parse the response. The only part of the response we are interested in is the authentication header, which we retrieve using the getAuthorizationHeaderValue() method.

Note that this example assumes you are writing an application that gets access to the username and password. If this is not the case, you will have to use OAuth instead.

OAuth (Coming soon)

The HttpRequestFactory object

Most of the functionality of the Client Library is encapsulated in an HttpRequestFactory object. This object will store your authentication header, the default headers to be used with every request, and the namespace dictionary. Our next step therefore is setting up an HttpRequestFactory instance which we will use for all future requests.

Either with ClientLogin or with OAuth we have now retrieved an authentication header. We need to add this authentication to the headers we will send with each request. In addition we need to set up a number of parsers and namespaces. In our example, we initialize the HttpRequestFactory in the following way:

  public static XmlNamespaceDictionary namespaceDictionary = new XmlNamespaceDictionary();
  protected static HttpRequestFactory httpRequestFactory;


  private static void setup(String username, String password)
      throws IOException {
    namespaceDictionary = namespaceDictionary
        .set("", "http://www.w3.org/2005/Atom")
        .set("app", "http://www.w3.org/2007/app")
        .set("gd", "http://schemas.google.com/g/2005")
        .set("sc", "http://schemas.google.com/structuredcontent/2009")
        .set("scp", "http://schemas.google.com/structuredcontent/2009/products")
        .set("xml", "http://www.w3.org/XML/1998/namespace");

    // authorize and create a HttpRequestFactory using the returned authentication
    // header
    String auth = authorizeUsingClientLogin(username, password);
    httpRequestFactory = createBareRequestFactory(auth, namespaceDictionary,
        errorNamespaceDictionary);
  }


  private static HttpRequestFactory createBareRequestFactory(String auth,
      XmlNamespaceDictionary namespaceDictionary,
      XmlNamespaceDictionary errorNamespaceDictionary) {
    HttpTransport transport = new NetHttpTransport();
    return transport.createRequestFactory(new HttpRequestInitializer() {
      @Override
      public void initialize(HttpRequest request) {
        GoogleHeaders headers = new GoogleHeaders();
        headers.setApplicationName("google-structuredcontentsample-1.0");
        headers.setGoogleLogin(auth);
        headers.gdataVersion = "1";
        request.setHeaders(headers);
        request.addParser(new AtomParser(namespaceDictionary));
      }
    });
  }

In this example, we simply store the created HttpRequestFactory as a static field. We also keep the namespace dictionaries as static class variables, because we will need them for sending requests. Now our HttpRequestFactory is initialized with all necessary values and we are ready to start managing our items with it. For a full example please download the sample code.

Note: The HttpRequestFactory is not thread-safe. If you want to use multi-threading, read the section on multi-threading below.

As a last setup step, we create a constant for the root URL:

  private static final String ROOT_URL = "https://content.googleapis.com/content/v1/";
    

Managing items

All actions on items -- getting, inserting, updating and deleting -- are performed by sending an HttpRequest to the Content API for Shopping server using the appropriate URL. (More information about these URLs is provided in the reference.)

First, you need to build the request using the HttpRequestFactory's methods buildPostRequest, buildPutRequest, buildDeleteRequest and buildGetRequest, which all take the destination URL and the data you want to send as arguments. Then you can send it using its execute() method, which will either return an HttpResponse or throw an HttpResponseException.

The HttpResponse object that is returned in case of a success provides the parseAs(Class clazz) method, which will try to parse the returned XML into a class of the type given as parameter. If there is no field available for an XML element, the information will be discarded. If an element appears more than once, but the class provides only a simple field instead of a collection, one of the returned instances will be parsed into it. Both the order in which elements are returned by the Content API for Shopping server and which of these elements will be written into the field by the Client Library are subject to change. You should therefore provide collections for all fields that may appear more than once.

Inserting items

The first step is to create the item you would like to insert, in this example a product item:

  private static Product createProduct(String id) {
    Product product = new Product();
    product.title = "Red wool sweater";
    product.content = new Content( "text",
        "Comfortable and soft, this sweater will keep you warm on those cold "
        + "winter nights. Red and blue stripes.");
    product.externalId = id;
    product.lang = "en";
    product.country = "US";
    product.condition = "new";
    product.price = new Price("usd", new BigDecimal(12.99f));

    // add a link
    Link link = new Link();
    link.rel = "alternate";
    link.href = "http://my.supercool.com/homepage/item1-info-page.html";
    link.type = "text/html";
    product.links.add(link);

    // set image links
    List imageLinks = new ArrayList();
    imageLinks.add("http://www.example.com/image1.jpg");
    imageLinks.add("http://www.example.com/image2.jpg");
    product.imageLinks = imageLinks;

    return product;
  }
    

This dummy implementation sets all fields to predefined constants except for the ID, which needs to be unique for each product item.

Now we just need to send this item to the server:

  private static Product insertProduct(Product product)
      throws IOException, HttpResponseException {
    String url = ROOT_URL + UID + "/items/products/schema";
    AtomContent atomContent = AtomContent.forEntry(namespaceDictionary, product);
    HttpRequest request = httpRequestFactory.buildPostRequest(new GoogleUrl(url), atomContent);
    return request.execute().parseAs(Product.class);
  }
    

If the inserting succeeded, the Content API for Shopping server returns the item you just inserted together with some additional attributes for your entry, such as the Atom id element and the creation date and time. The method above then parses this returned entry and returns it.

If the insertion failed, the method above throws an HttpResponseException. Handling of those exceptions is described below in the section on error handling.

Updating items

Updating an item is very similar to inserting it. By sending an HTML PUT request to the URL of the item, you will overwrite the old item. The request therefore needs to contain all of the attributes, even those you don't want to change. The only difference between inserting and updating an item therefore is the used URL, and the usage of a PUT instead of a POST request:

  private static Product updateProduct(Product product)
      throws IOException, HttpResponseException {
    String url = ROOT_URL + UID + "/items/products/schema/"
        + product.externalId;
    AtomContent atomContent = AtomContent.forEntry(namespaceDictionary, product);
    HttpRequest request = httpRequestFactory.buildPutRequest(new GoogleUrl(url), atomContent);
    return request.execute().parseAs(Product.class);
  }
    

Return type and error handling is the same as for inserting items. More information on error handling can be found in this section below.

Deleting items

For deleting an item, we simply send a delete request to its URL:

  private static void deleteProduct(String productId)
      throws IOException, HttpResponseException {
    String url = ROOT_URL + UID + "/items/products/schema/" + productId;
    HttpRequest request = httpRequestFactory.buildDeleteRequest(new GoogleUrl(url));
    request.execute();
  }
    

Also here an HttpResponseException will be thrown if and only if deletion failed. Handling of this exception is described in this section below.

Retrieving items

You can either get a single item or a list of them.

Retrieving a single item

For accessing a single item, you only need to know its ID. Using that ID you send an HTTP GET request to its url and parse the returned item:

  private static Product getProduct(String productId)
      throws IOException, HttpResponseException {
    String url = ROOT_URL + UID + "/items/products/schema/" + productId;
    HttpRequest request = httpRequestFactory.buildGetRequest(new GoogleUrl(url));
    return request.execute().parseAs(Product.class);
  }
    

Retrieving a list of items

For retrieving a list of your items, you send a GET request to the feed URL and get in return a feed containing several items:

  private static List<Product> getProducts()
      throws IOException, HttpResponseException {
    String url = ROOT_URL + UID + "/items/products/schema";
    HttpRequest request = httpRequestFactory.buildGetRequest(new GoogleUrl(url));
    ProductFeed feed = request.execute().parseAs(ProductFeed.class);
    return feed.getEntries();
  }
    

This feed does not necessarily contain all items. It will contain the fields openSearch:totalResults, openSearch:itemsPerPage and openSearch:startIndex indicating how many items you have stored, how many are returned per request, as well as the number of the first item in the returned list.

Additionally, if (and only if) there are more items, it will contain a link with the rel-value "next". You can send your next GET request to that URL in order to get the next batch of items. For doing so, we first need a class for storing the information returned in a product feed:

public class ProductFeed {
  @Key("link")
  public List<Link> links = new ArrayList<Link>();

  @Key
  public String id;

  @Key
  public String updated;

  @Key("entry")
  public List<Product> entries = new ArrayList<Product>();
}
    

Using this feed class, we now retrieve all product items using the given "next"-links.

  private static ProductFeed getProducts(String url)
      throws IOException, HttpResponseException {
    HttpRequest request = httpRequestFactory.buildGetRequest(new GoogleUrl(url));
    return request.execute().parseAs(ProductFeed.class);
  }


  private static List<Product> getAllProducts() throws IOException {
    // get first page
    ProductFeed feed = getProducts(ROOT_URL + UID + "/items/products/schema");
    List<Product> list = feed.getEntries();

    // If the last page that was retrieved had a "next" link, get items from
    // that link and add them to the list. Repeat if necessary.
    String nextUrl = null;
    while ((nextUrl = findNextLink(feed.links)) != null) {
      feed = getProducts(nextUrl);
      list.addAll(feed.getEntries());
    }
    return list;
  }


  private static String findNextLink(List<Link> links) {
    if (links != null) {
      for (Link link : links) {
        if ("next".equals(link.rel)) {
          return link.href;
        }
      }
    }
    return null;
  }
    

Note that in the downloadable example the findNextLink(...) function has been moved into the Link class for better readability.

Handling errors

The examples above do not contain any error-handling code to keep things simple. If anything goes wrong during a request, you either get an IOException, indicating that something went seriously wrong, or an HttpResponseException.

You can catch an HttpResponseException and parse the errors returned by the Content API for Shopping server into a ServiceErrors class. You need to provide this class yourself, modelling it as described above. A sample implementation is available as part of the downloadable example.

In order to handle errors this way, you first need to set up a new error namespace dictionary and parser. The relevant lines in the following code snippet are displayed as italic.

  public static XmlNamespaceDictionary namespaceDictionary = new XmlNamespaceDictionary();
  public static XmlNamespaceDictionary errorNamespaceDictionary = new XmlNamespaceDictionary();
  protected static HttpRequestFactory httpRequestFactory;


  private static void setup(String username, String password)
      throws IOException {
    namespaceDictionary = namespaceDictionary
        .set("", "http://www.w3.org/2005/Atom")
        .set("app", "http://www.w3.org/2007/app")
        .set("gd", "http://schemas.google.com/g/2005")
        .set("sc", "http://schemas.google.com/structuredcontent/2009")
        .set("scp", "http://schemas.google.com/structuredcontent/2009/products")
        .set("xml", "http://www.w3.org/XML/1998/namespace");

    errorNamespaceDictionary = errorNamespaceDictionary
        .set("", "http://schemas.google.com/g/2005");

    // authorize and create a transport using the returned authentication
    // header
    String auth = authorizeUsingClientLogin(username, password);
    httpRequestFactory = createBareRequestFactory(auth, namespaceDictionary,
        errorNamespaceDictionary);
  }


  private static HttpRequestFactory createBareRequestFactory(String auth,
      XmlNamespaceDictionary namespaceDictionary,
      XmlNamespaceDictionary errorNamespaceDictionary) {
    HttpTransport transport = new NetHttpTransport();
    return transport.createRequestFactory(new HttpRequestInitializer() {
      @Override
      public void initialize(HttpRequest request) {
        GoogleHeaders headers = new GoogleHeaders();
        headers.setApplicationName("google-structuredcontentsample-1.0");
        headers.setGoogleLogin(auth);
        headers.gdataVersion = "1";
        request.setHeaders(headers);
        request.addParser(new AtomParser(namespaceDictionary));
        request.addParser(XmlHttpParser.builder(createErrorNamespaceDictionary())
            .setContentType("application/vnd.google.gdata.error+xml")
            .build());
      }
    });
  }
    

A simple error handling section that just outputs the errors to the standard output might then look like this:

    try {
      ...
    } catch (HttpResponseException e) {
      ServiceErrors errors = e.getResponse().parseAs(ServiceErrors.class);
      System.out.println(errors.toString());
    }
    

Batching

When sending many requests, it is better to combine them and send as a single batch request instead of sending them individually. Batching reduces the time it takes to process your requests, and additionally reduces the load on your servers and ours.

For a general introduction to batching, read the GData documentation.

To perform batching with the Content API for Shopping server:

  1. Create a list of entries to be modified (inserted, updated, deleted) in one batch. Each entry of the list is a normal instance of the Product class, except that additionally the fields relevant to batching need to be set to appropriate values.
  2. Send the entire list created in the previous step to the server, and check the results for errors.

We will now discuss these two steps in detail.

Creating a list of entries to be modified

In order to tell the Content API for Shopping server what to do with the products in the list that you send as a batch, i.e. if it should insert, update or delete them, you need to add this information to every single product by setting its batch:status field. For update and delete requests you additionally need to set the id field to the edit link so that the server can identify the product.

You also may set the batch:id field to any value so that you can more easily identify the products in the answer returned by the server. In case of products, it usually makes sense to set the batch ID (batch:id) to the same value as the product ID (sc:id).

In order to set these fields, we first need to add them to the Product class. In the downloadable example on batching, we achieve this by creating a class Entry containing the basic XML elements, deriving a class BatchableEntry from Entry for adding the fields relevant to batching, and finally deriving Product from BatchableEntry for adding all fields we need for product entries. Our code therefore looks like this:

public class Entry extends GenericXml {
  @Key("link")
  public List<Link> links = new ArrayList<Link>();

  @Key
  public String title;

  @Key
  public Content content;
}


public class BatchableEntry extends Entry {
  @Key("batch:operation")
  public BatchOperation batchOperation = null;

  @Key("batch:id")
  public String batchID = null;

  @Key("batch:status")
  public BatchStatus batchStatus = null;

  @Key("batch:interrupted")
  public BatchInterrupted batchInterrupted;

  public static class BatchOperation {
    @Key("@type")
    public String type;
  }

  public static class BatchStatus {
    @Key("@code")
    public int code;

    @Key("@reason")
    public String reason;
  }

  public static class BatchInterrupted {
    @Key ("@error")
    public int error;

    @Key ("@parsed")
    public int parsed;

    @Key ("@reason")
    public String reason;

    @Key ("@success")
    public int success;

    @Key ("@unprocessed")
    public int unprocessed;
  }
}


public class Product extends BatchableEntry {
  @Key("sc:id")
  public String externalId;

  @Key("sc:target_country")
  public String country = "US";

  @Key("sc:content_language")
  public String lang = "en";

  // ...
}
    

Batch insert

For turning a product into a batch insert request, we just need to set its batch operation code to "insert". In addition we set the batch ID (batch:id) to the product ID (sc:id) so that we can identify the returned product more easily.

In the downloadable examples, we have the utility class BatchUtils that will take care of this for us:

  public static void configureForInsert(BatchableEntry entry,
      String batchId) {
    entry.batchOperation = new BatchableEntry.BatchOperation();
    entry.batchOperation.type = "insert";
    if (batchId != null) {
      entry.batchID = batchId;
    }
  }
    

In order to turn a normal product entry myProduct into a batch insert entry, we therefore can simply use this:

  BatchUtils.configureEntryForInsert(myProduct, myProduct.externalId);
    

Batch delete

For deleting a product, we need to set the batch operation to "delete" in the same way, and in addition we must set its Atom ID, i.e. the id field, to the edit link of the product. This edit link can be retrieved from the links that are returned by the server when querying a product. Alternatively, you can determine it yourself, since it is of the form https://content.googleapis.com/content/v1/123456/items/products/schema/99988877, where 123456 is your account ID and 99988877 is the ID of the product.

Adding the batch delete operation is again done by a function in BatchUtils:

  public static void configureForDelete(BatchableEntry entry,
      String batchId) {
    entry.batchOperation = new BatchableEntry.BatchOperation();
    entry.batchOperation.type = "delete";
    if (batchId != null) {
      entry.batchID = batchId;
    }
  }
    

Since we additionally need to set the Atom ID (id) to the edit link, our complete code looks like this:

  BatchUtils.configureEntryForDelete(myProduct, myProduct.externalId);
  myProduct.atomId = Link.find(myProduct.links, "edit");
    

Batch update

Batch updating works very similar to batch deletion in that you have to provide the exact edit link as id, and set the batch operation to "update". We again have a function in BatchUtils for this:

  public static void configureForUpdate(BatchableEntry entry,
      String batchId) {
    entry.batchOperation = new BatchableEntry.BatchOperation();
    entry.batchOperation.type = "update";
    if (batchId != null) {
      entry.batchID = batchId;
    }
  }
    

The entire code for modifying a product looks like this:

  BatchUtils.configureEntryForUpdate(myProduct, myProduct.externalId);
  myProduct.atomId = Link.find(myProduct.links, "edit");
    

Sending the list of entries to be modified

Once we have a list of items that all have their batch:operation and id fields set to the appropriate values, we can send it to the Content API for Shopping server. In order to do so, we first need a class to represent an Atom feed, which in our case basically is a list of product entries:

public class ProductFeed {
  @Key("link")
  public List<Link> links = new ArrayList<Link>();

  @Key("id")
  public String id;

  @Key("updated")
  public String updated;

  @Key("entry")
  public List<Product> entries = new ArrayList<Product>();
}
    

Now we write a function that takes a list of products (assuming they all have their batch:operation and id fields set properly), and sends it to the batch URL of the server, in our case https://content.googleapis.com/content/v1/123456/items/products/schema/batch, where 123456 is the user ID. The corresponding code looks like this:

  private void executeProductBatch(List<Product> productList)
      throws IOException {
    ProductFeed batchedProducts = new ProductFeed();
    batchedProducts.entries = productList;
    String url = ROOT_URL + UID + "/items/products/schema/batch";
    AtomContent atomFeedContent = AtomContent.forFeed(namespaceDictionary, batchedProducts);
    HttpRequest request = httpRequestFactory.buildPostRequest(new GoogleUrl(url), atomFeedContent);
    HttpResponse response = request.execute();

    // interpret the results, handle errors
    // ...
  }
    

Error handling

For simplicity, the code above contains no error handling yet. When batching, the batch request should return an HTTP success code, even if there were errors for individual products. For retrieving such errors, we therefore need to parse the returned batch:status fields that are contained in every single returned product entry. They contain HTTP status codes for each product, as well as the reason given for the error or status. If it's a success status code, the reason given might simply be "Product successfully created". For an error, the reason field will contain the error message that you would have received if you had sent the product as an individual request.

In rare cases a batch interruption can occur. In this case, you will get an entry returned that doesn't contain any product or batch:id, but instead contains the field batch:interrupted. This means that all products that haven't been reported as processed yet have been dropped.

The downloadable batching example contains code to parse and report all of these errors.

Comments

  • A batch sent to the server can contain any combination of insert, update and delete entries. It may however not contain two or more requests that affect the same product. If it does, there will not necessarily be an error message, but the outcome is undefined and subject to change.
  • If you are inserting a new product that has the same sc:id as another one of your products that is already stored on the server, then the new product will overwrite the one already present. This is another way of updating products.

Multi-threading

As mentioned earlier, the HttpRequestFactory is not thread-safe. If you wish to use multiple threads, you must make sure that they don't access the HttpRequestFactory at the same time. This also applies to the HttpRequests generated by the HttpRequestFactory, as they internally keep a pointer to the original HttpTransport!

The easiest way to use multi-threading is therefore putting a synchronized block around the entire get, insert, update or delete block that accesses the HttpRequestFactory, locking it on the HttpRequestFactory instance itself. If you can make sure that all methods accessing the HttpRequestFactory do this, you are on the safe side.

A thread safe implementation of the insert method could for example look like this:

  private static Product insertProduct(Product product)
      throws IOException, HttpResponseException {
    String url = ROOT_URL + UID + "/items/products/schema";
    AtomContent atomContent = AtomContent.forEntry(namespaceDictionary, product);

    synchronized (httpRequestFactory) {
      HttpRequest request = httpRequestFactory.buildPostRequest(new GoogleUrl(url), atomContent);
      return request.execute().parseAs(Product.class);
    }
  }
    

The generic projection

Instead of the schema projection, which is used in all of these examples, the Google API Client Library for Java can also handle the generic projection. The Product class is shorter in this case because it loses all scp: elements:

public class Product {
  @Key
  public String title;

  @Key
  public Content content;

  @Key("sc:id")
  public String externalId;

  @Key("link")
  public List<Link> links = new ArrayList<Link>();

  @Key("sc:image_link")
  public List<String> imageLinks;

  @Key("sc:target_country")
  public String country = "US";

  @Key("sc:content_language")
  public String lang = "en";

  @Key("sc:attribute")
  public List<Attribute> attributes;

  @Key("sc:group")
  public List<AttributeGroup> attributeGroups;
}
    

Instead of the scp: attributes, we get a list of generic attributes (sc:attribute) and generic attribute groups (sc:group). For representing them, we need the following two classes:

public class Attribute {
  @Key("@name")
  public String name;

  @Key("@type")
  public String type;

  @Key("@unit")
  public String unit;

  @Key("text()")
  public String value;
}


public class AttributeGroup {
  @Key("@name")
  public String name;

  @Key("sc:attribute")
  public List<Attribute> attributes;
}
    

Beyond that, everything stays the same. The only difference is that we now need to add all attributes as instances of the Attribute class instead of setting them directly. The price of the product for example now needs to be set like this:

  Attribute price = new Attribute();
  price.name = "price";
  price.type = "float";
  price.unit = "USD";
  price.value = "12.99";

  myProduct.attributes.add(price);
    

Similarly, the price can't be read any more using myProduct.price, but instead the entire list of attributes returned by the server must be searched for one attribute with the attribute name set to "price".

The advantage of this method is that it can not only be used for products, but also for other categories of items that might be available in future. The downside is that locating and using returned attributes becomes more complicated, potentially slower, and error-prone because of the unsafe type conversion logic.

Downloading the complete example

The example programs together with some descriptions can be found here.

Note: Some of the code snippets in this documentation are slightly simplified versions of the actual code contained in the examples.

Authentication required

You need to be signed in with Google+ to do that.

Signing you in...

Google Developers needs your permission to do that.