Synchronize resources efficiently

This guide describes how to implement incremental synchronization of calendar data. Using this method, you can keep data for all calendar collections in sync while saving bandwidth.

Contents

Overview

Incremental synchronization consists of two stages:

  1. Initial full sync: Performed once at the beginning to fully synchronize the client state with the server state. The client obtains a sync token that it must persist.

  2. Incremental sync: Performed repeatedly to update the client with all changes made since the previous sync. Each time, the client provides the previous sync token obtained from the server and stores the new sync token from the response.

Initial full sync

The initial full sync is the original request for all the resources of the collection you want to synchronize. You can optionally restrict the list request using request parameters if you only want to synchronize a specific subset of resources.

In the response to the list operation, the response contains a field named nextSyncToken representing a sync token. You must store the value of nextSyncToken. If the result set is too large and the response is paginated, then the nextSyncToken field is present only on the last page.

Incremental sync

Incremental sync lets you retrieve all the resources that have been modified since the last sync request. To do this, perform a list request with your most recent sync token specified in the syncToken field. Keep in mind that the result always contains deleted entries, so clients can remove them from storage.

In cases where a large number of resources have changed since the last incremental sync request, you might find a pageToken instead of a syncToken in the list result. In these cases, perform the same list query used to retrieve the first page in the incremental sync (with the same syncToken), append the pageToken to it, and paginate through subsequent requests until you find another syncToken on the last page. Store this syncToken for the next sync request.

The following examples show an incremental paginated sync:

Original query

GET /calendars/primary/events?maxResults=10&singleEvents=true&syncToken=CPDAlvWDx70CEPDAlvWDx

Result contains the following:

{
  "nextPageToken": "CiAKGjBpNDd2Nmp2Zml2cXRwYjBpOXA"
}

Retrieving next page

GET /calendars/primary/events?maxResults=10&singleEvents=true&syncToken=CPDAlvWDx70CEPDAlvWDx&pageToken=CiAKGjBpNDd2Nmp2Zml2cXRwYjBpOXA

Full sync required by server

The server sometimes invalidates sync tokens due to token expiration or changes in related ACLs. In such cases, the server responds to an incremental request with HTTP status code 410. When this occurs, clear the client storage and perform a new full sync.

Sample code

The following sample demonstrates how to use sync tokens with the Java client library. The first time the run() method is called, it performs a full sync and stores the sync token. On each subsequent execution, it loads the saved sync token and performs an incremental sync.

  private static void run() throws IOException {
    // Construct the {@link Calendar.Events.List} request, but don't execute it yet.
    Calendar.Events.List request = client.events().list("primary");

    // Load the sync token stored from the last execution, if any.
    String syncToken = syncSettingsDataStore.get(SYNC_TOKEN_KEY);
    if (syncToken == null) {
      System.out.println("Performing full sync.");

      // Set the filters you want to use during the full sync. Sync tokens aren't compatible with
      // most filters, but you may want to limit your full sync to only a certain date range.
      // In this example we are only syncing events up to a year old.
      Date oneYearAgo = Utils.getRelativeDate(java.util.Calendar.YEAR, -1);
      request.setTimeMin(new DateTime(oneYearAgo, TimeZone.getTimeZone("UTC")));
    } else {
      System.out.println("Performing incremental sync.");
      request.setSyncToken(syncToken);
    }

    // Retrieve the events, one page at a time.
    String pageToken = null;
    Events events = null;
    do {
      request.setPageToken(pageToken);

      try {
        events = request.execute();
      } catch (GoogleJsonResponseException e) {
        if (e.getStatusCode() == 410) {
          // A 410 status code, "Gone", indicates that the sync token is invalid.
          System.out.println("Invalid sync token, clearing event store and re-syncing.");
          syncSettingsDataStore.delete(SYNC_TOKEN_KEY);
          eventDataStore.clear();
          run();
        } else {
          throw e;
        }
      }

      List<Event> items = events.getItems();
      if (items.size() == 0) {
        System.out.println("No new events to sync.");
      } else {
        for (Event event : items) {
          syncEvent(event);
        }
      }

      pageToken = events.getNextPageToken();
    } while (pageToken != null);

    // Store the sync token from the last request to be used during the next execution.
    syncSettingsDataStore.set(SYNC_TOKEN_KEY, events.getNextSyncToken());

    System.out.println("Sync complete.");
  }

Legacy synchronization

For event collections, you can perform legacy synchronization by saving the value of the updated field from an events list request and then using the updatedMin field to retrieve updated events. This approach is no longer recommended because it is more error-prone (for example, it doesn't enforce query restrictions) and is available only for events.