Migrate from Entity Read Files

Entity Read Files (ERFs) are JSON representations of a partner’s campaign objects that, on request, are generated daily and made available through Google Cloud Storage. As of June 2021, ERFs have been deprecated and are no longer actively supported. All existing Entity Read File users are encouraged to migrate to the Display & Video 360 API in order to continue retrieving Display & Video 360 resources.

This guide discusses how to migrate from Entity Read Files to the Display & Video 360 API by:

  • Giving an overview of the differences between the two interfaces
  • Comparing ERF tables to API services
  • Providing guidance on entity retrieval through the API
  • Acknowledging existing data gaps
  • Presenting a mapping of all ERF fields to comparable API resource fields

Overview

When migrating from ERFs to the Display & Video 360 API, there are a number of key differences to consider, including:

  • Data freshness. ERFs are generated daily and in bulk while the API retrieves the most up-to-date version of a resource.
  • Resource structure. The API uses different JSON structures than ERF to represent the same resource types. Some resources, such as public targeting settings, may use a different ID space.
  • Retrieval method. The Display & Video 360 API only enables the retrieval of resources individually or in paginated lists, in contrast to the large, exhaustive JSON files provided by ERF.
  • Scope. As opposed to ERFs, which are scoped by partner ID, most API resources are scoped by advertiser ID. Resources included in responses are limited to resources within that scope.
  • Available data. Certain resource subtypes (such as YouTube & partners resources) are not available through the API. Similarly, Floodlight activity objects are not retrievable through the API.

ERF data representation in the API

Entity Read Files are separated into “Public” and “Private” tables. Public tables provide information that is available and applicable to all users, such as targeting values. Private tables provide data that is specific to a partner, such as creative or line item resources.

The Display & Video 360 API does not use this dichotomy, instead making all of this information retrievable through various services and using different JSON structures. This section compares the information provided through public and private ERF tables to that made available through Display & Video 360 API resources and services.

Public information

ERF public tables provide reference materials for users to use when interpreting the targeting settings of their retrieved private resources and assigning targeting through Structured Data Files (SDFs) uploaded through the UI. These reference materials are the same for all users and consist of a numeric ID, used for mapping, and more descriptive details, such as a display name.

When using the Display & Video 360 API, targeting reference information can be retrieved through the targetingTypes.targetingOptions service. Similar to the public tables, this service provides the IDs and details of targeting options for a specific targeting type. Refer to our existing Set Targeting page for a code example demonstrating targeting option ID retrieval.

Public tables and SDFs

Entity Read Files and Structured Data Files use the same ID space for targeting settings. If you are an SDF user currently using ERF public tables for interpreting or assigning targeting settings via SDF, you can instead download this reference material in CSV format through the Display & Video 360 UI.

Private resources

ERF private tables provide a daily snapshot of the current settings of private resources owned by a partner. Due to the sheer volume of resources that can be created under a single partner, these files can grow to be very large and difficult to download and process.

In the API, each private table has a corresponding service that provides endpoints for retrieval and management of that resource type. Resources can be retrieved in bulk using each service’s respective list method. The JSON structure for each resource is different in the API compared to ERF, utilizing different field names and shared resources.

Certain information available in the ERF representation of a resource, such as a resource’s assigned targeting settings or a channel’s sites, are represented in the API as children of the original resource and must be retrieved through additional API requests.

Entity retrieval in the API

Each resource type is retrievable through a different API service. Resources can be retrieved individually or in bulk using the appropriate service’s get or list method, respectively. Important properties of Display & Video 360 API list methods include:

  • Required scope. Unlike ERFs, which are scoped by partner, most resources in the API are scoped by advertiser. Retrieving all of a resource type, such as line items, under a partner may require an individual list request for each child advertiser of that partner. Exceptions include direct children of a partner, such as advertisers and partner-owned channels.
  • Pagination. API list methods employ pagination to guarantee that responses are within a reasonable size, limiting most individual request responses, or pages, to 100 resources. If the number of relevant resources is larger than the page size, consecutive list calls are required to retrieve subsequent pages of the full list response. A code example paging a list response is provided in a section of our Targeting guide page regarding retrieving available targeting options .
  • Additional requests required for targeting retrieval. A resource’s targeting settings are not included in its API JSON object, but are instead child resources known as assigned targeting options. These child resources must be retrieved through a separate request. For example, for each line item retrieved through an advertisers.lineItems.list request, a separate advertisers.lineItems.bulkListLineItemAssignedTargetingOptions request must be made to retrieve all targeting information.

Optimizing resource retrieval

The Display & Video 360 API may require multiple requests to retrieve the same amount of information that is available in a single Entity Read File. Optimizing how you retrieve resources can help retrieve the data you need more efficiently:

  • Make concurrent requests to the API. The API’s default rate limit of 600 queries per minute per project lets you implement a multithreaded solution that will reduce the total time it takes to retrieve all necessary resources. Although pagination requires that all resources of a type within a certain scope are retrieved through consecutive calls, retrieval of resources within another scope or of another type can be done concurrently.
  • Utilize filters and order by parameters in your list calls to retrieve only relevant resources. For example, if you are only interested in line items that have been updated in the last day, you can use the filter parameter of the advertisers.lineItems.list method to only return line items with an updateTime greater than a given timestamp. This can significantly reduce the number of requests that need to be made.
  • Cache regularly used IDs to avoid unnecessary API requests. Certain reference information, such as targeting options IDs and Google Audience IDs, are relatively stable and can be safely stored to avoid the need for retrieval upon every use. However, cached values should be checked on a weekly basis to account for infrequent changes or deprecations.

Known API data gaps

There are notable data gaps you might encounter when migrating from ERF to Display & Video 360 API, such as:

  • Floodlight Activity resources. Floodlight activities are not retrievable through the Display & Video 360 API. Use the UI to retrieve this information.
  • Certain resources subtypes. All YouTube & partners resources as well as story insertion orders are not retrievable through the API. YouTube & partners resources can be retrieved with SDF download. Story insertion orders must be retrieved through the Display & Video 360 UI.
  • A subset of resource fields. A small number of resource fields present in ERF objects are not available in the corresponding resources retrieved through the Display & Video 360 API.

Appendix: Mapping ERF Fields to API

Public table mapping

The tables below map the fields of ERF public tables to existing targeting types and targeting option fields in the Display & Video 360 API. Although the value of one field might map to another, that does not guarantee that they utilize the same data type, enum values, or ID space.

App Collection

Retrievable under targeting type TARGETING_TYPE_APP_CATEGORY.

ERF Field NameDV360 Availability
id TargetingOption.targetingOptionId field.
name TargetingOption.appCategoryDetails.displayName field.

Browser

Retrievable under targeting type TARGETING_TYPE_BROWSER.

ERF Field NameDV360 Availability
id TargetingOption.targetingOptionId field.
is_mobile Not available.
name TargetingOption.browserDetails.displayName field.

DataPartner

There is no equivalent resource or fields available in Display & Video 360 API.

DeviceCriteria

Retrievable under targeting types TARGETING_TYPE_OPERATING_SYSTEM, TARGETING_TYPE_DEVICE_MAKE_MODEL, and TARGETING_TYPE_DEVICE_TYPE.

ERF Field NameDV360 Availability
id TargetingOption.targetingOptionId field.
is_mobile Not available.
name TargetingOption.operatingSystemDetails.displayName, TargetingOption.deviceMakeModelDetails.displayName, or TargetingOption.deviceTypeDetails.deviceType fields, depending on targeting type.
criteria_type TargetingOption.targetingType field.
operating_system_id Not available.
mobile_brand_name Not available.
mobile_model_name Not available.
mobile_make_model_id Not available.
device_type TargetingOption.deviceTypeDetails.deviceType field.

GeoLocation

Retrievable under targeting type TARGETING_TYPE_GEO_REGION.

ERF Field NameDV360 Availability
id TargetingOption.targetingOptionId field.
canonical_name TargetingOption.geoRegionDetails.displayName field.
geo_name Not available.
country_code Not available.
region_code Not available.
city_name Not available.
postal_name Not available.
dma_code Not available.

Isp

Retrievable under targeting type TARGETING_TYPE_CARRIER_AND_ISP.

ERF Field NameDV360 Availability
id TargetingOption.targetingOptionId field.
is_mobile Not available.
name TargetingOption.carrierAndIspDetails.displayName field.
secondary_criteria_id TargetingOption.targetingOptionId field.

Language

Retrievable under targeting type TARGETING_TYPE_LANGUAGE.

ERF Field NameDV360 Availability
id TargetingOption.targetingOptionId field.
name Not available. Full display name for a language is available at TargetingOption.languageDetails.displayName field.

SiteToPlacementId

There is no equivalent resource or fields available in Display & Video 360 API.

SupportedExchange

Retrievable under targeting type TARGETING_TYPE_EXCHANGE.

ERF Field NameDV360 Availability
id TargetingOption.targetingOptionId field.
name TargetingOption.exchangeDetails.exchange field.

UniversalSite

There is no equivalent resource or fields available in Display & Video 360 API. Individual sites can be targeted directly under targeting type TARGETING_TYPE_URL.

Private table field mapping

The tables below map the fields of ERF private tables to existing fields or services in the Display & Video 360 API. Although the value of one field may map to another, that does not guarantee that they utilize the same data type, enum values or ID space.

Advertiser

ERF Field NameDV360 Availability
common_data.id Advertiser.advertiserId field.
common_data.name Advertiser.displayName field.
common_data.active Advertiser.entityStatus field.
common_data.integration_code Advertiser.integrationDetails.integrationCode field.
partner_id Advertiser.partnerId field.
currency_code Advertiser.generalConfig.currencyCode field.
timezone_code Advertiser.generalConfig.timeZone field.
landing_page_url Advertiser.generalConfig.domainUrl field.
available_channel_ids Retrievable through advertisers.channels.list method.
blacklist_channel_id Retrievable through advertisers.targetingTypes.assignedtargetingOptions.list method under targeting type TARGETING_TYPE_CHANNEL. If AssignedTargetingOption.channelDetails.negative is true, it is a blacklisted channel.
dcm_configuration Not available.
dcm_network_id Advertiser.adServerConfig.cmHybridConfig.cmAccountId field.
dcm_advertiser_id Not available.
dcm_floodlight_group_id Advertiser.adServerConfig.cmHybridConfig.cmFloodlightConfigId field.
dcm_syncable_site_ids Advertiser.adServerConfig.cmHybridConfig.cmSyncableSiteIds field.
enable_oba_tags Not available.

Campaign

ERF Field NameDV360 Availability
common_data.id Campaign.campaignId field.
common_data.name Campaign.displayName field.
common_data.active Campaign.entityStatus field.
common_data.integration_code Not available.
advertiser_id Campaign.advertiserId field.
budget Campaign.campaignFlight and Campaign.campaignBudgets fields.
frequency_cap Campaign.frequencyCap field.
default_target_list Retrievable through advertisers.campaigns.bulkListCampaignAssignedTargetingOptions method.
uses_video_creatives Not available.
uses_display_creatives Not available.
uses_audio_creatives Not available.
objective Campaign.campaignGoal.campaignGoalType field.
metric Campaign.campaignGoal.performanceGoal.performanceGoalType field.
objective_description Campaign.campaignGoal.performanceGoal.performanceGoalString field.
metric_amount_micros Campaign.campaignGoal.performanceGoal.performanceGoalAmountMicros field.

Creative

ERF Field NameDV360 Availability
common_data.id Creative.creativeId field.
common_data.name Creative.displayName field.
common_data.active Creative.entityStatus field.
common_data.integration_code Creative.integrationCode field.
advertiser_id Creative.advertiserId field.
dcm_placement_id Creative.cmPlacementId field.
width_pixels Creative.dimensions.widthPixels field.
height_pixels Creative.dimensions.heightPixels field.
approval_status Creative.reviewStatus field.
expanding_direction Creative.expandingDirection field.
creative_type Creative.creativeType field.

CustomAffinity

ERF Field NameDV360 Availability
id CustomList.customListId field.
name CustomList.displayName field.
description Not available.
advertiser_id Not available.

FloodlightActivity

There is no equivalent resource or fields available in Display & Video 360 API.

InsertionOrder

ERF Field NameDV360 Availability
common_data.id InsertionOrder.insertionOrderId field.
common_data.name InsertionOrder.displayName field.
common_data.active InsertionOrder.entityStatus field.
common_data.integration_code InsertionOrder.integrationDetails.integrationCode field.
advertiser_id InsertionOrder.advertiserId field.
campaign_id InsertionOrder.campaignId field.
overall_budget Not available. Can be calculated using contents of the InsertionOrder.budget.budgetSegments field.
scheduled_segments InsertionOrder.budget.budgetSegments field.
frequency_cap InsertionOrder.frequencyCap field.
default_partner_costs InsertionOrder.partnerCosts field.
default_target_list Retrievable through advertisers.insertionOrders.bulkListInsertionOrderAssignedTargetingOptions method.

InventorySource

ERF Field NameDV360 Availability
id InventorySource.inventorySourceId field.
unclassified Not available.
inventory_name InventorySource.displayName field.
exchange_id InventorySource.exchange field.
accessing_advertisers Not available.
external_id InventorySource.dealId field.
min_cpm_micros InventorySource.rateDetails.rate.nanos field, depending on the value of the InventorySource.rateDetails.inventorySourceRateType field.
min_cpm_currency_code InventorySource.rateDetails.rate.currencyCode field.

LineItem

ERF Field NameDV360 Availability
common_data.id LineItem.lineItemId field.
common_data.name LineItem.displayName field.
common_data.active LineItem.entityStatus field.
common_data.integration_code LineItem.integrationDetails.integrationCode field.
line_item_type LineItem.lineItemType field.
insertion_order_id LineItem.insertionOrderId field.
creative_ids LineItem.creativeIds field.
max_cpm_advertiser_micros LineItem.bidStrategy.maximizeSpendAutoBid.maxAverageCpmBidAmountMicros or LineItem.bidStrategy.performanceGoalAutoBid.maxAverageCpmBidAmountMicros fields, depending on the strategy scheme used.
performance_goal LineItem.bidStrategy.maximizeSpendAutoBid.performanceGoalType or LineItem.bidStrategy.performanceGoalAutoBid.performanceGoalType fields, depending on the strategy scheme used.
goal_advertiser_micros LineItem.bidStrategy.performanceGoalAutoBid.performanceGoalAmountMicros field.
partner_revenue_model LineItem.partnerRevenueModel field.
cost_tracking_pixels LineItem.conversionCounting.floodlightActivityConfigs field.
budget.start_time_usec LineItem.flight.dateRange.startDate field.
budget.end_time_usec LineItem.flight.dateRange.endDate field.
budget.max_impressions LineItem.budget.maxAmount field if LineItem.budget.budgetUnit is BUDGET_UNIT_IMPRESSIONS.
budget.max_spend_advertiser_micros LineItem.budget.maxAmount field if LineItem.budget.budgetUnit is BUDGET_UNIT_CURRENCY.
budget.pacing_type LineItem.pacing.pacingPeriod field.
budget.pacing_max_impressions LineItem.pacing.dailyMaxImpressions field.
budget.pacing_max_spend_advertiser_micros LineItem.pacing.dailyMaxMicros field.
budget.pacing_distribution LineItem.pacing.pacingType field.
frequency_cap LineItem.frequencyCap field.
partner_costs LineItem.partnerCosts field.
target_list Retrievable through advertisers.lineItems.bulkListLineItemAssignedTargetingOptions method.

NegativeKeywordList

ERF Field NameDV360 Availability
id NegativeKeywordList.negativeKeywordListId field.
name NegativeKeywordList.displayName field.
advertiser_id NegativeKeywordList.advertiserId field.

Partner

ERF Field NameDV360 Availability
common_data.id Partner.partnerId field.
common_data.name Partner.displayName field.
common_data.active Partner.entityStatus field.
common_data.integration_code Not available.
currency_code Partner.generalConfig.currencyCode field.
exchange_settings Partner.exchangeConfig.enabledExchanges field.
default_partner_costs Not available.
default_partner_revenue Not available.
default_target_list Not available.

Pixel

There is no equivalent resource or fields available in Display & Video 360 API.

UniversalChannel

ERF Field NameDV360 Availability
id Channel.channelId field.
name Channel.displayName field.
site_ids Retrievable through advertisers.channels.sites.list and partners.channels.sites.list methods, depending on the type of owner.
accessing_advertisers Not available.
is_deleted Not available.
is_brand_safe_channel Not available.

UserList

ERF Field NameDV360 Availability
id FirstAndThirdPartyAudience.firstAndThirdPartyAudienceId field.
name FirstAndThirdPartyAudience.displayName field.
data_partner_id Not available.
accessing_advertisers Not available.
partner_pricing Not available.
advertiser_pricings Not available.