Get Started with Cloud Storage

Google Display & Video 360 automatically stores Entity Read Files in Google Cloud Storage. These files are only accessible to accounts that have been granted access.

Create a project

Create and configure a project as follows:

  1. Go to the Google API Console Enabled APIs page.

    If you are not already signed into Google, you are prompted to sign in.

  2. From the project drop-down, select an existing project or create a new one.
    • If you have multiple existing projects, make sure to select the project for which you want to activate Google Cloud Storage, then select Continue.
    • Or, if you do not have a project, select Create a new project, enter a Project name, then select Create.

      Note: You should be aware that some resource identifiers (such as project IDs) might be retained beyond the life of your project. For this reason, avoid storing sensitive information in resource identifiers.

  3. In the list of Enabled APIs, make sure the Google Cloud Storage component is listed. If it's not listed, click the Google APIs tab, search for and select the Google Cloud Storage component, and click Enable API.

    Note that for most projects, the Google Cloud Storage component is already enabled. If you plan to access Google Cloud Storage using the JSON API, then you should also verify that the Google Cloud Storage JSON API is enabled.

  4. Enable billing for the project.

    Before you can use Google Cloud Storage, you need to enable billing for your project if you haven't already done so. Enabling billing does not necessarily mean you will be charged. For more information, see Pricing.

    1. Still within the cloud storage project, click the gallery menu in the upper left corner, then select Billing.
    2. If you don't yet have a billing account, follow the instructions to create a new billing account.
    3. If you have an existing billing account, select it, then select Set account.

In the Google API Console, stay within the same project and go to the next section.

Set up access to Cloud Storage

  1. Still in the Google API Console within the required project, click the gallery menu in the upper left corner, select APIs & Services, and then click Credentials. On the Credentials page, look for Service account keys.

    Find the Service account key email address:

    1. Select the Manage Service Accounts link (to the right of the Service account keys section).
    2. The email address is viewable on the resulting Permissions page.

    If there are no Service account keys, create a new Service account:

    1. Select the Create credentials drop-down, then choose Service account key.
    2. In the Service account drop-down, choose New service Account.
    3. Enter a Name for the service account. The Service account ID is automatically generated from the name and project name.
    4. Make a note of the service account ID gserviceaccount.com email address. It is not displayed on the Credentials page after you select Create.
    5. In the Key type section, choose JSON.
    6. Click Create. The JSON file with the account's public/private key pair is saved to your Downloads folder. This key is used to authenticate your application to the API and sign all of the requests it sends to the API. Store the generated JSON file in a safe place, somewhere where your application will be able to access it.
  2. Contact the Display & Video 360 Support Team to request enablement of Entity Read files and/or the Google DoubleClick Bid Manager API.

    After the Support Team confirms access, create a dedicated Google Group (or groups) and update the following fields in Display & Video 360 partner details (note that only users with partner-level access will be able to make these updates):

    • Log Read Google Group
      The read-only group only has permissions to read files and list the content of the storage buckets.
    • Log Management Google Group
      The management group has all of the permissions of the read-only group, but is also able to change the ACLs of files within the buckets. Note that even the management group does not have permission to change bucket ACLs, nor do they have permission to create, modify, or delete files.

    After the Google Group is added, you can add users to and remove users from your Google Group as required without any further involvement from the Display & Video 360 support team; only those users who are in your group will have access to your data.

Access your data

Access data using gsutil

The gsutil tool is a command-line application that lets you access your Google Cloud Storage data easily without any programming experience. You could, for example, use gsutil as part of a script or batch file instead of creating custom applications.

To get started with gsutil, read the gsutil documentation. You'll still need to authenticate using OAuth 2 as before, but gsutil itself will prompt you for the credentials the first time you use it and then store them for use later on.

Access data programmatically

Google Cloud Storage has APIs for many programming languages that allow you to access your data in a programmatic way. The sample below shows you how to perform a simple download of one of the Entity Read Files from Google Cloud Storage. Remember that only the public Entity Read Files are stored in the gdbm-public bucket; your private Entity Read Files will be stored in Partner- and Advertiser-specific buckets.

Java

The following example uses the Google APIs Client Library for Java, which makes accessing the data held in Google Cloud Storage easy. In addition to the main general Google APIs Client Library, you'll also need to download the separate library for the Cloud Storage API.

You will first need to set up the credentials to be used when communicating with the API to get authorization to access your data. This is done by creating a Credential object that uses the Service Account's developer.gserviceaccount.com email address as the ServiceAccountId and also loads the private key file that was downloaded from the Google API Console when creating the Service Account. This will ensure that all future requests to the API are signed with the Service Account's private key:

NetHttpTransport HTTP_TRANSPORT = new NetHttpTransport();
JacksonFactory JSON_FACTORY = new JacksonFactory();

...

Credential credential = new GoogleCredential.Builder().setTransport(HTTP_TRANSPORT)
   .setJsonFactory(JSON_FACTORY)
   .setServiceAccountId("...@developer.gserviceaccount.com")
   .setServiceAccountScopes(StorageScopes.DEVSTORAGE_READ_ONLY)
   .setServiceAccountPrivateKeyFromP12File(new File("...-privatekey.p12"))
   .build();

After you have created the Credential object, you can then use it to instantiate a new Storage object that you can use for all future requests to the Google Cloud Storage API:

Storage storage = new Storage(HTTP_TRANSPORT, JSON_FACTORY, credential);

Now that you have a new Storage object, you are ready to begin making requests to the API. Here you are making a request to retrieve a specific item bucketObjectName from the bucket bucket. The library handles the HTTP download of the file from Google Cloud Storage, and you then simply print the contents out to the console:

Get bucketObject = storage.objects().get(bucketName, bucketObjectName);

ByteArrayOutputStream output = new ByteArrayOutputStream();
bucketObject.getMediaHttpDownloader().setDirectDownloadEnabled(true);
bucketObject.executeAndDownloadTo(output);

System.out.println(output.toString());

Your values for bucket and bucketObjectName will vary depending on what part of the API you are accessing; refer to the appropriate section on naming conventions.

C#

The following example uses the Google APIs Client Library for .NET, which makes accessing the data held in Google Cloud Storage simple and easy.

You will first need to set up the credentials to be used when communicating with the API to get authorization to access your data. This is done by creating a AssertionFlowClient object that uses the Service Account's developer.gserviceaccount.com email address as the ServiceAccountId and also loads the private key file that was downloaded from the Google API Console when creating the Service Account.

string keyPath = @"...-privatekey.p12";
X509Certificate2 certificate = new X509Certificate2(keyPath, "notasecret", X509KeyStorageFlags.Exportable);
var provider = new AssertionFlowClient(GoogleAuthenticationServer.Description, certificate)
{
    ServiceAccountId = "...@developer.gserviceaccount.com",
    Scope = StorageService.Scopes.DevstorageRead_only.GetStringValue()
};

After you have created the AssertionFlowClient object, you can then use it to instantiate a new OAuth2Authenticator object that you can use to instantiate a new Storage instance for all future requests to the Google Cloud Storage API:

var auth = new OAuth2Authenticator<AssertionFlowClient>(provider, AssertionFlowClient.GetState);
StorageService service = new StorageService(auth);

Now that you have a new Storage object, you are ready to begin making requests to the API. Here you are making a request to retrieve a specific item, bucketObjectName, from the bucket bucket. We use standard .NET client library classes to handle the HTTP download from Google Cloud Storage, but you have to manually handle the case where there is a redirect to ensure that there is always the correct Authorization header present in the request:

var results = service.Objects.Get(bucketName, bucketObjectName).Fetch();

HttpWebRequest request = createRequest(results.Media.Link, auth);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();

// Handle redirects manually to ensure that the Authorization header is present if
// your request is redirected.
if (response.StatusCode == HttpStatusCode.TemporaryRedirect)
{
    request = createRequest(response.Headers["Location"], auth);
    response = (HttpWebResponse)request.GetResponse();
}

The contents of the response can then be read from the HttpWebResponse object via the GetResponseStream as usual. Your values for bucket and bucketObjectName will vary depending on what part of the API you are accessing; refer to the appropriate section on naming conventions.

Appendix: Complete Samples

Java

/*
 * Copyright (c) 2013 Google Inc.
 *
 * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except
 * in compliance with the License. You may obtain a copy of the License at
 * http://www.apache.org/licenses/LICENSE-2.0.
 *
 * Unless required by applicable law or agreed to in writing, software distributed under the License
 * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
 * or implied. See the License for the specific language governing permissions and limitations under
 * the License.
 */
package com.google.bidmanager.api.samples;

import java.io.BufferedReader;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.InputStreamReader;

import com.google.api.client.auth.oauth2.Credential;
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.http.javanet.NetHttpTransport;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.services.storage.Storage;
import com.google.api.services.storage.Storage.Objects.Get;
import com.google.api.services.storage.StorageScopes;

/**
 * A simple class that demonstrates how to download a specific object from a bucket using a
 * service account
 */
public class ServiceDownload {

  /**
   * This is the HTTP Transport object used for automatically refreshing access tokens.
   */
  static final NetHttpTransport HTTP_TRANSPORT = new NetHttpTransport();

  /**
   * This is the JSON factory used for parsing refresh token responses.Your first requirement
   */
  static final JacksonFactory JSON_FACTORY = new JacksonFactory();

  /**
   * The main method will attempt to download a specific named object from the gdbm-public bucket
   * using a service account.
   *
   * @param args Not used.
   */
  public static void main(String[] args) {

    try {
      // Prompt the user for the details of the object to download.
      System.out.print("Name of object to download, e.g. entity/20130430.0.Browser.json:");
      BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
      String bucketObjectName = in.readLine();
      in.close();
      String bucketName = "gdbm-public";

      Credential credential = new GoogleCredential.Builder().setTransport(HTTP_TRANSPORT)
           .setJsonFactory(JSON_FACTORY)
           .setServiceAccountId("...@developer.gserviceaccount.com")
           .setServiceAccountScopes(StorageScopes.DEVSTORAGE_READ_ONLY)
           .setServiceAccountPrivateKeyFromP12File(
               new File("...-privatekey.p12"))
           .build();

      Storage storage = new Storage(HTTP_TRANSPORT, JSON_FACTORY, credential);

      Get bucketObject = storage.objects().get(bucketName, bucketObjectName);

      ByteArrayOutputStream output = new ByteArrayOutputStream();
      bucketObject.getMediaHttpDownloader().setDirectDownloadEnabled(true);
      bucketObject.executeAndDownloadTo(output);
      System.out.println(output.toString());

    } catch (Exception e) {
      System.err.println("Unexpected exception caught: " + e.getMessage());
      e.printStackTrace();
    }

  }


}

C#

/*
 * Copyright (c) 2013 Google Inc.
 *
 * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except
 * in compliance with the License. You may obtain a copy of the License at
 * http://www.apache.org/licenses/LICENSE-2.0.
 *
 * Unless required by applicable law or agreed to in writing, software distributed under the License
 * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
 * or implied. See the License for the specific language governing permissions and limitations under
 * the License.
 */
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Net;
using System.Security.Cryptography.X509Certificates;

using DotNetOpenAuth.OAuth2;

using Google.Apis;
using Google.Apis.Requests;
using Google.Apis.Authentication.OAuth2;
using Google.Apis.Authentication.OAuth2.DotNetOpenAuth;
using Google.Apis.Storage.v1beta1;
using Google.Apis.Storage.v1beta1.Data;
using Google.Apis.Util;

namespace ApiSamples
{
    /// <summary>
    /// A simple class that demonstrates how to download a specific object from a bucket using a
    /// service account
    /// </summary>
    class ServiceDownload
    {
        /// <summary>
        /// The main method will attempt to download a specific named object from the
        /// gdbm-public bucket using a service account.
        /// </summary>
        /// <param name="args">Not used.</param>
        public static void Main(string[] args)
        {

            // Prompt the user for the details of the object to download
            Console.WriteLine("Name of object to download, e.g. entity/20130430.0.Browser.json:");
            string bucketObjectName = Console.ReadLine();
            string bucketName = "gdbm-public";

            try
            {
                string keyPath = @"...-privatekey.p12";
                X509Certificate2 certificate = new X509Certificate2(keyPath, "notasecret", X509KeyStorageFlags.Exportable);
                var provider = new AssertionFlowClient(GoogleAuthenticationServer.Description, certificate)
                {
                    ServiceAccountId = "...@developer.gserviceaccount.com",
                    Scope = StorageService.Scopes.DevstorageRead_only.GetStringValue()
                };
                var auth = new OAuth2Authenticator<AssertionFlowClient>(provider, AssertionFlowClient.GetState);

                StorageService service = new StorageService(auth);
                var results = service.Objects.Get(bucketName, bucketObjectName).Fetch();

                HttpWebRequest request = createRequest(results.Media.Link, auth);
                HttpWebResponse response = (HttpWebResponse)request.GetResponse();

                // Handle redirects manully to ensure that the Authorization header is present if
                // our request is redirected.
                if (response.StatusCode == HttpStatusCode.TemporaryRedirect)
                {
                    request = createRequest(response.Headers["Location"], auth);
                    response = (HttpWebResponse)request.GetResponse();
                }

                Stream stream = response.GetResponseStream();
                StreamReader reader = new StreamReader(stream);
                String data = reader.ReadToEnd();
                Console.Write(data);
            }
            catch (Exception e)
            {
                Console.WriteLine("Unexpected exception caught: " + e.Message);
                Console.Write(e.StackTrace);
            }
            Console.ReadKey();
        }

        /// <summary>
        /// Generate a HttpWebRequest for the given URL with the appropriate OAuth2 authorization
        /// header applied.  The HttpWebRequest object returned has its AllowAutoRedirect option
        /// disabled to allow us to manually handle redirects.
        /// </summary>
        /// <param name="url">URL that is to be requested with this object</param>
        /// <param name="auth">The OAuth2Authenticator instance that contains the appropriate keys.</param>
        /// <returns>HttpWebRequest object ready to be used to make requests to the API</returns>
        private static HttpWebRequest createRequest(string url, OAuth2Authenticator<AssertionFlowClient> auth)
        {
            HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
            request.Headers.Add("Authorization", "Bearer " + auth.State.AccessToken);
            request.AllowAutoRedirect = false;
            return request;
        }

    }
}