Video Creative Ingest API

The Video Creative Ingest API allows third-party ad systems to proactively ingest creatives into YouTube and DAI products. This process ensures that when an ad is decisioned from a non-Google ad server, the creatives that are returned are ready and available to be served into a YouTube or DAI video stream.

The process of proactively ingesting creatives from third-party ad systems involves two main parts. The first part allows the third-party ad system to provide creatives to Google and monitor the progress of creatives as they are ingested into the Google video infrastructure. Once the creatives have been successfully ingested, identifiers are passed back to the ad system.

The second step involves the ad system taking those identifiers and including them in the VAST documents that reference the submitted creative, using either the Universal Ad ID (VAST 4.0+) or a Creative Extension of type "UniversalAdId". When a VAST document from the ad system is encountered at decision time in DAI or YouTube, the creative system extracts those identifiers from the VAST and fetches the appropriate creative from Google video infrastructure to be inserted for playback.

Media requirements

Creative media submitted to the API should meet the following criteria to ensure it can be processed successfully by the system.

Media types

The following media types are accepted by Google. Any other media types could be rejected upon submission.

  • video/mp4
  • video/webm
  • video/quicktime
  • video/avi
  • video/mpeg
  • video/x-flv
  • flv-application/octet-stream
  • application/octet-stream
  • video/3gpp
  • video/ogg
  • audio/mp4
  • audio/mpeg

Media properties

The following criteria must be met by the submitted media:

  • Media must have at least one audio or video track.
  • Media must have a reasonable duration. Nothing exceeding several minutes.
  • Media must have audio or video that is longer than 1 second in duration.
  • If media has audio, it must have a standard audio channel configuration such as mono, stereo, or surround sound.

Submit creatives for ingestion

Flow diagram of ingestion process

  1. A third-party ad system sends an HTTP Request to the Video Creative Ingest API with a list of media files associated with the creative.
  2. The Video Creative Ingest API selects the highest quality creative from the provided list and downloads it from the CDN.
  3. The Video Creative Ingest API ingests the downloaded creative into the Google Video Infrastructure, making it available to both the YouTube and DAI products.
  4. A Pub/Sub notification is sent back to the third-party ad system with the status of the creative. The notification will indicate if it was successfully loaded for both YouTube and DAI. If loading the creative resulted in failure, the notification will indicate the cause of the failure.

Follow these guidelines when submitting creatives.

Authentication

You must access the API using a Google Cloud service account. At time of integration, account information must be provided to Google to gain the appropriate authorization.

Once account information has been provided to Google through creativeingestion@google.com, access will be granted to the API endpoints and the Cloud PubSub topics used to publish notifications.

To start the authentication process, follow these steps:

  1. Create a Google Cloud account.
  2. Set up a project with the account for sending requests to the DAI API.
  3. Within the project, go to the IAM & Admin tab on the left, and create a new service account; requests are sent using the service account.

  4. Enable the PubSub API and create a single unique Partner Notification topic to which the API publishes all ingest success and failure notifications.

    If you have a domain restriction organization policy in place, you need to add the Google customer ID to the allowed customer list. Contact creativeingestion@google.com for the ID. Once this is done, you can add principals (our robot account) from allowed customer lists (the API's customer ID) to IAM policies (PubSub Publisher).

    Add video-creative-library-discovery@system.gserviceaccount.com as a PubSub Publisher to the topic.

  5. Provide creativeingestion@google.com with the service account email address and the PubSub topic to register this information with the API.

  6. Enable the DAI API after the account is added as a service consumer.

  7. To set your credentials for accessing the API:

    • Download credentials from the service account: Click the service account in the Service accounts section of the IAM & Admin tab, click Create Key, and store the credentials.

    • Authenticate your API caller. Set GOOGLE_APPLICATION_CREDENTIALS to the path of the JSON file. See code examples for authenticating gRPC clients in common languages.

Creative ingestion

Each time a new creative is available in a partner's ad system, the creative should be submitted by the partner's system to the Creative Ingest API.

When new creatives are submitted to Google to load into the video infrastructure, the same creative data that is included in the VAST document generated by the ad system should be provided. This ensures that the behavior of proactively ingesting creatives into Google has the same behavior as loading creatives when they are first discovered in an ad response.

Each submitted creative creates a long-running ingest operation on the server whose status can be queried. Once the operation successfully completes, it results in a creative resource within the system. The response to submitting a creative contains the ID of the operation that resulted from the request. The resulting operation ID is included in the Cloud PubSub notifications that are sent when the operation completes to map back to the original creative that was submitted. The operation ID can also be used to check the progress of the operation at any given time.

Creative requests

Data is sent to the API as Protocol Buffers or as JSON objects. Protocol Buffers, sent using gRPC, are recommended as they are necessary for consuming the PubSub notifications, and unmarshaling the JSON has added complications due to google.protobuf.Any fields in the response.

Creative responses

On success, the server returns a Protocol Buffer response or JSON response, depending on the request type. The response contains the newly created ingest operation name and the details of the operation.

Upon initial submit, some of the metadata associated with the operation may be empty and the result may not be present. Once the operation completes, the result is populated with the Creative resource (or an error) and the metadata contains information about the ingest status.

Note that the metadata must be unmarshaled from an Any proto into the CreateCreativeOperationMetadata proto, and the response (if present) must be unmarshaled into the Creative proto.

Client library to access API

Follow these steps to integrate with the API:

  1. Download protocol buffer dependencies and move the entire google folder to your src directory. This can be skipped if using Google Cloud App Engine.
  2. Use the protoc plugin to generate gRPC code from the proto file and put the generated code in the relevant directory.
  3. Create a new proto file for the proto returned in the PubSub notification; repeat the previous step to generate code from the proto.

  4. Use gRPC to make requests to the API:

    1. Get Default Credentials with the following scopes:

    https://www.googleapis.com/auth/video-ads,

    https://www.googleapis.com/auth/pubsub

    1. Send the request to Creative Ingest.
    • Connect to dai.googleapis.com:443 using the default credentials.
    • Make a client stub to the Creative Ingest service using the gRPC proto-generated code.
    • Call CreateCreative using the client stub.

    Optionally, create a stub to the Operations service using long-running operations gRPC code—included in the googleapis directory downloaded in Step 2—which can be used to get the status of a given operation (GetOperation).

  5. Check PubSub for notifications:

    1. Use the PubSub client library or gRPC to connect to PubSub service and poll for PubSub notifications published to topic provided.

    2. Unmarshal the PubSub message into the proto from Step 4.

    3. Whenever a long-running Operation is returned from calls to CreateCreative, GetOperation, or in a PubSub notification, unmarshal the metadata into CreateCreativeOperationMetadata in discovery.proto from the google.protobuf.Any proto and any response into Creative in discovery.proto.

Example Go client code

// Copyright 2024 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package main

import (
  dai "google.golang.org/genproto/googleapis/ads/dai/v1"
  psnotif "protos/dai"
  "google.golang.org/genproto/googleapis/longrunning"
  "cloud.google.com/go/pubsub"
  "golang.org/x/net/context"
  "google.golang.org/grpc"
  "google.golang.org/grpc/credentials"
  "google.golang.org/grpc/credentials/oauth"
  "github.com/golang/protobuf/proto"
  anypb "github.com/golang/protobuf/ptypes/any"
)

var scopes = []string{"https://www.googleapis.com/auth/video-ads", "https://www.googleapis.com/auth/pubsub"}

const (
  subID = "sub-id" // The ID for the subscription to the topic you provided us (projects/<project ID>/subscriptions/<subscription ID>)
  project = "your-project-id" // Your Google Cloud Project ID
)

func main() {
  ctx := context.Background()
  crpc, err := oauth.NewApplicationDefault(ctx, scopes...)
  if err != nil {
    // TODO: Handle error
  }

  conn, err := grpc.Dial("dai.googleapis.com:443",
    grpc.WithTransportCredentials(credentials.NewClientTLSFromCert(nil, "")),
    grpc.WithPerRPCCredentials(crpc))
  if err != nil {
    // TODO: Handle error 
  }

  apiclient := dai.NewPartnerDiscoveryServiceClient(conn)
  opclient := longrunning.NewOperationsClient(conn)

  op, err := apiclient.CreateCreative(ctx, &dai.CreateCreativeRequest{
    // TODO: Add Creative with metadata and media files.
  })
  if err != nil {
    // TODO: Handle error
  }

  pubsubClient, err := pubsub.NewClient(ctx, project)
  if err != nil {
    // TODO: Handle error
  }
  sub := pubsubClient.Subscription(subID)

  // TODO: Pull from Pub/Sub subscription to receive messages, and acknowledge them
  var msg *pubsub.Message // Assume msg = a received Pub/Sub message

  // Unmarshal the Pub/Sub message into the proto created in step 9a of guide
  var n psnotif.CreateCreativeStatus
  if err := proto.Unmarshal(msg.Data, &n); err != nil {
    // TODO: Handle error
  }
  op = n.GetDetails()
  md, err = anyToMetadata(op.GetMetadata())
  if err != nil {
    // TODO: Handle error
  }
  // TODO: Check if ingest was successful and get Google Video ID from md.GetGvRegistryId()
  // TODO: Enable serving of creative using Google Video ID in VAST document

  // Optionally check the status of an ingest request using the status endpoint
  op, err = opclient.GetOperation(ctx, &longrunning.GetOperationRequest{Name:   op.Name})
  if err != nil {
    // TODO: Handle error
  }
}

// anyToMetadata converts the metadata Any field in an Operation message to
// CreateCreativeOperationMetadata.
func anyToMetadata(msg *anypb.Any) (*dai.CreateCreativeOperationMetadata, error) {
  var md dai.CreateCreativeOperationMetadata
  return &md, proto.Unmarshal(msg.Value, &md)
}

// anyToCreative converts the response Any field in an Operation message to
// Creative.
func anyToCreative(msg *anypb.Any) (*dai.Creative, error) {
  var cr dai.Creative
  return &cr, proto.Unmarshal(msg.Value, &cr)
}

Creative lookup during playback

Flow diagram of creative lookup process

  1. User requests to view content in either YouTube or DAI.
  2. YouTube / DAI obtains a VAST response from a third-party ad system for ads to insert into the content.
  3. The returned ads are mapped to previously ingested creatives. Those creatives are ready for playback - they're fetched from the Google Video Infrastructure and played for the user.

Partner modified VAST documents

When a VAST response is returned from the ad system after the creative has been successfully loaded into the Google video infrastructure, the document should contain a reference to the processed creative. The VAST document should continue to include the media files for a given creative but there is no guarantee made that these media files will be used by the system.

There are two ways to reference a processed creative in a VAST document: adding a new UniversalAdId element to the VAST 4+ document or adding a custom Extension.

Universal Ad ID

The gvRegistryID returned by the Video Creative Ingest API is a universal public-facing ID used to uniquely identify a video creative across multiple ad systems. This ID should be added to the ad system's VAST 4+ documents so when a VAST document is received by YouTube or by DAI, the ID can be used to map back to the originally submitted creative.

Ad systems should add a new UniversalAdId element to their VAST 4+ documents under the Creative node. The idRegistry attribute should be set to "googlevideo" and the value should be the value provided in the status of the successfully processed creative notification.

The following is a sample VAST 4 document with the new Universal Ad ID added:

<VAST xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="vast.xsd" version="4.0">
  <Ad id="318230556">
    <InLine>
      <AdSystem>DCM</AdSystem>
      <AdTitle>In-Stream Video</AdTitle>
      <Creatives>
        <Creative id="79534759" AdID="" sequence="1">
          <UniversalAdId idRegistry="googlevideo">dQw4w9WgXcQ</UniversalAdId>
          <Linear>
            <Duration>00:00:15</Duration>
            <MediaFiles>
              <Mezzanine>
                <![CDATA[https://cdn.com/media.mp4]]>
              </Mezzanine>
              <MediaFile delivery="progressive" width="640" height="360" type="video/mp4" bitrate="313">
                <![CDATA[https://cdn.com/low-res-media.mp4]]>
              </MediaFile>
            </MediaFiles>
          </Linear>
        </Creative>
      </Creatives>
    </InLine>
  </Ad>
</VAST>

CreativeExtension

Since the Universal Ad ID element was only introduced in VAST 4.0, the same data needs a custom extension in order to be exposed in VAST documents with version lower than 4.0. This is done using a CreativeExtension element under the Creative element of the VAST spec.

The new extension introduced is a generic extension enabled to allow any Universal AD ID value be populated. In this particular case the Universal Ad ID would belong to the googlevideo registry and the ID would be the google video ID.

<VAST version="3.0">
  <Ad id="318230556">
    <InLine>
      <AdSystem>My Ad Server</AdSystem>
      <AdTitle>Car Company</AdTitle>
      <Creatives>
        <Creative id="79534759" AdID="" sequence="1">
          <Linear>...</Linear>
          <CreativeExtensions>
            <CreativeExtension type="UniversalAdId">
              <UniversalAdId idRegistry="googlevideo">dQw4w9WgXcQ</UniversalAdId>
            </CreativeExtension>
          </CreativeExtensions>
        </Creative>
      </Creative>
    </InLine>
  </Ad>
</VAST>

The CreativeExtension approach can also be used in VAST 4+ in the case the UniversalAdId is already populated with an ID for a different UniversalAdId registry:

<VAST xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="vast.xsd" version="4.0">
  <Ad id="318230556">
    <InLine>
      <AdSystem>DCM</AdSystem>
      <AdTitle>In-Stream Video</AdTitle>
      <Creatives>
        <Creative id="79534759" AdID="" sequence="1">
          <UniversalAdId idRegistry="other-registry">other-id</UniversalAdId>
          <Linear>
            <Duration>00:00:15</Duration>
            <MediaFiles>
              <Mezzanine>
                <![CDATA[https://cdn.com/media.mp4]]>
              </Mezzanine>
              <MediaFile delivery="progressive" width="640" height="360" type="video/mp4" bitrate="313">
                <![CDATA[https://cdn.com/low-res-media.mp4]]>
              </MediaFile>
            </MediaFiles>
          </Linear>
          <CreativeExtensions>
            <CreativeExtension type="UniversalAdId">
              <UniversalAdId idRegistry="googlevideo">dQw4w9WgXcQ</UniversalAdId>
            </CreativeExtension>
          </CreativeExtensions>
        </Creative>
      </Creatives>
    </InLine>
  </Ad>
</VAST>