Because requests for large reports can take awhile to generate, the Search Ads 360 API provides an asynchronous technique for requesting and downloading reports. With this technique, you send an initial request that specifies the data you want in the report, and then send additional polling requests until Search Ads 360 finishes generating the report. Depending on the size of the report, Search Ads 360 splits the data into multiple files. Once the report has been generated, you send requests to download each report file. If you're requesting smaller amounts of data, you can send just a single synchronous request.
To make an asynchronous request
- Call
Reports.request()to specify the type of data you want in the report. See Report Types for the types of data you can request.Search Ads 360 validates the request and returns a report ID—a unique identifier for this request.
- Call
Reports.get()with the report id.The response from Search Ads 360 indicates:
- Whether the report is ready for download.
- If the report is ready, one or more URLs for downloading the report.
- Call
Reports.getFile()to the download the report files encoded, or just directly download from the URLs.Search Ads 360 returns the report in a UTF-8 encoded file.
How frequently should I poll reports to see if they are ready?
The amount of time Search Ads 360 needs to generate a report depends mostly on the amount of data in the report. Try polling for the report status once per minute, and then adjust the frequency if your average report request takes significantly more or less time to finish.
Splitting asynchronous reports into multiple files
In response to an asynchronous request, Search Ads 360 automatically splits large reports into
multiple files. Use the Reports.request.maxRowsPerFile
property to specify a maximum file size. Each resulting report file is guaranteed to have
at most maxRowsPerFile report rows (not counting headers). A different URL
is generated for each file and returned in response to Reports.get(). For information
about downloading report files, see Download
the report.
For CSV reports, the header is repeated in each file.
Asynchronous Example
Below are sample requests and responses using the asynchronous technique.
JSON
POST https://www.googleapis.com/doubleclicksearch/v2/reports
Authorization: Bearer your OAuth 2.0 access token
Content-type: application/json
{
"reportScope": {
"agencyId": "12300000000000456", // Replace with your ID
"advertiserId": "21700000000011523", // Replace with your ID
},
"reportType": "keyword", // This report covers all keywords in the
// advertiser specified in reportScope.
"columns": [
{ "columnName": "campaignId" }, // Here are some attribute columns available for keyword
{ "columnName": "keywordText" }, // reports.
{ "columnName": "keywordLandingPage" },
{ "columnName": "date" }, // The date column segments the report by individual days.
{ "columnName": "dfaRevenue" }, // Here are some metric columns available for keyword
{ // reports
"columnName": "visits",
"startDate": "2013-01-01", // Each metric column can optionally specify its own start
"endDate": "2013-01-31", // and end date; by default the report timeRange is used.
"headerText": "visits last month" // Every column can optionally specify a headerText, which
// changes the name of the column in the report.
}
],
"timeRange" : {
"startDate" : "2012-05-01", // Dates are inclusive and specified in YYYY-MM-DD format.
"endDate" : "2012-05-02"
// Alternatively, try the "changedMetricsSinceTimestamp" or "changedAttributesSinceTimestamp"
// options. See Incremental reports.
},
"filters": [
{
"column" : { "columnName": "keywordLandingPage" },
"operator" : "startsWith",
"values" : [ // With this filter, only keywords with landing pages
"http://www.foo.com", // rooted at www.foo.com or www.bar.com are returned.
"http://www.bar.com" // See Filtered reports.
]
}
],
"downloadFormat": "csv",
"maxRowsPerFile": 6000000, // Required. See Splitting reports into multiple files.
"statisticsCurrency": "agency", // Required. See Currency for statistics.
"verifySingleTimeZone": false, // Optional. Defaults to false. See Time zone.
"includeRemovedEntities": false // Optional. Defaults to false.
}
Java
/**
* Creates a campaign report request, submits the report, and returns the report ID.
*/
private static String createReport(Doubleclicksearch service) throws IOException {
try {
return service.reports().request(createSampleRequest()).execute().getId();
} catch (GoogleJsonResponseException e) {
System.err.println("Report request was rejected.");
for (ErrorInfo error : e.getDetails().getErrors()) {
System.err.println(error.getMessage());
}
System.exit(e.getStatusCode());
return null; // Unreachable code.
}
}
/**
* Creates a simple static request that lists the ID and name of all
* campaigns under agency 12300000000000456 and advertiser 21700000000011523.
* Substitute your own agency ID and advertiser IDs for the IDs in this sample.
*/
private static ReportRequest createSampleRequest() {
return new ReportRequest()
.setReportScope(new ReportScope()
.setAgencyId(12300000000000456L) // Replace with your ID
.setAdvertiserId(21700000000011523L)) // Replace with your ID
.setReportType("campaign")
.setColumns(Arrays.asList(
new ReportApiColumnSpec[] {
new ReportApiColumnSpec().setColumnName("campaignId"),
new ReportApiColumnSpec().setColumnName("campaign")
}))
.setTimeRange(new TimeRange()
.setStartDate("2012-05-01")
.setEndDate("2012-05-01"))
.setDownloadFormat("csv")
.setStatisticsCurrency("usd")
.setMaxRowsPerFile(5000000);
}
.NET
This function creates a report that lists campaigns under an advertiser and assigns the returned token toreportId.
using api = Google.Apis.Doubleclicksearch.v2;
/// <summary>
/// Creates a report with a sample request and returns the report ID.
/// </summary>
/// <param name="service">Search Ads 360 API service.</param>
private static string CreateReport(api.DoubleclicksearchService service)
{
var req = service.Reports.Request(CreateSampleRequest());
var report = req.Execute();
Console.WriteLine("Created report: ID={0}", report.Id);
return report.Id;
}
/// <summary>
/// Returns a simple static request that lists the ID and name of all
/// campaigns under an advertiser.
/// Substitute your own agency ID and advertiser IDs for the IDs in this sample.
/// </summary>
private static api.Data.ReportRequest CreateSampleRequest()
{
return new api.Data.ReportRequest
{
ReportScope = new api.Data.ReportRequest.ReportScopeData
{
AgencyId = 12300000000000456, // Replace with your ID
AdvertiserId = 21700000000011523 // Replace with your ID
},
ReportType = ReportType.CAMPAIGN,
Columns = new List<api.Data.ReportApiColumnSpec>
{
new api.Data.ReportApiColumnSpec
{
ColumnName = "campaignId",
},
new api.Data.ReportApiColumnSpec
{
ColumnName = "campaign",
},
},
TimeRange = new api.Data.ReportRequest.TimeRangeData
{
StartDate = "2015-01-01",
EndDate = "2015-01-07",
},
DownloadFormat = "csv",
StatisticsCurrency = "usd",
MaxRowsPerFile = 5000000,
};
}
Python
def request_report(service): """Request sample report and print the report ID that DS returns. See Set Up Your Application. Args: service: An authorized Doubleclicksearch service. Returns: The report id. """ request = service.reports().request( body= { "reportScope": { "agencyId": "12300000000000456", // Replace with your ID "advertiserId": "21700000000011523", // Replace with your ID "engineAccountId": "700000000073991" // Replace with your ID }, "reportType": "keyword", "columns": [ { "columnName": "campaignId" }, { "columnName": "keywordText" }, { "columnName": "keywordLandingPage" }, { "columnName": "date" }, { "columnName": "dfaRevenue" }, { "columnName": "visits", "startDate": "2013-01-01", "endDate": "2013-01-31", "headerText": "visits last month" } ], "timeRange" : { "startDate" : "2012-05-01", "endDate" : "2012-05-02" }, "filters": [ { "column" : { "columnName": "keywordLandingPage" }, "operator" : "startsWith", "values" : [ "http://www.foo.com", "http://www.bar.com" ] } ], "downloadFormat": "csv", "maxRowsPerFile": 6000000, "statisticsCurrency": "agency", "verifySingleTimeZone": "false", "includeRemovedEntities": "false" } ) json_data = request.execute() return json_data['id']
If validation succeeds
If the report passes validation, Search Ads 360 returns a report id. Search Ads 360 also returns metadatavregarding currency code and time zone.
{
"kind": "adsdartsearch#report",
"id": "MTMyNDM1NDYK", // This is the report id.
"isReportReady": false, // The report is not finished generating.
"request": { // The request that created this report.
...
},
"statisticsCurrencyCode": "CAD", // The currency used for statistics. E.g., if
// advertiser currency was requested, this would be
// currency code of the advertiser in scope.
"statisticsTimeZone": "America/New_York" // If all statistics in the report were sourced from
// a single time zone, this would be it. If
// verifySingleTimeZone was set to true in the request,
// then this field will always be populated (or the
// request will fail).
}
If validation fails
If the report does not pass validation, Search Ads 360 returns an HTTP 400 response
with an error object. For example, the example request above did not specify a real
agency:
{
"error": {
"code": 400,
"message": "statisticsCurrency: the agency in scope does not have a valid currency. Please make sure the agency is properly initialized in Search Ads 360."
}
}
Poll for report status
Call Report.Get() with the report id.
JSON
GET https://www.googleapis.com/doubleclicksearch/v2/reports/MTMyNDM1NDYK
Java
/**
* Polls the reporting API with the reportId until the report is ready.
* Returns the report.
*/
private static Report pollUntilReportIsFinished(Doubleclicksearch service, String reportId)
throws IOException, InterruptedException {
long delay = 1;
while (true) {
Report report = null;
try {
report = service.reports().get(reportId).execute();
} catch (GoogleJsonResponseException e) {
System.err.println("Report generation has failed.");
System.exit(e.getStatusCode());
}
if (report.getIsReportReady()) {
return report;
}
System.out.format("Report %s is not ready - waiting %s seconds.%n", reportId, delay);
Thread.sleep(TimeUnit.SECONDS.toMillis(delay));
delay = delay + delay; // Double the delay for the next iteration.
}
}
.NET
using api = Google.Apis.Doubleclicksearch.v2;
/// <summary>
/// Polls until the report with the given ID completes.
/// </summary>
/// <param name="service">Search Ads 360 API service.</param>
/// <param name="reportId">Report ID to poll.</param>
/// <exception cref="ApplicationException">
/// Thrown when the report completes, but has failed.
/// </exception>
private static api.Data.Report PollUntilReportIsFinished(
api.DoubleclicksearchService service,
string reportId)
{
TimeSpan delay = TimeSpan.FromSeconds(1);
while (true)
{
api.Data.Report report;
try
{
report = service.Reports.Get(reportId).Execute();
}
catch (Google.GoogleApiException ex)
{
throw new ApplicationException("Report generation failed", ex);
}
if (report.IsReportReady.GetValueOrDefault(false))
{
return report;
}
Console.WriteLine("Report is not ready - waiting {0}", delay);
Thread.Sleep(delay);
delay = delay.Add(delay); // Double the delay for the next iteration.
}
}
Python
import pprint
import simplejson
from googleapiclient.errors import HttpError
def poll_report(service, report_id):
"""Poll the API with the reportId until the report is ready, up to ten times.
Args:
service: An authorized Doubleclicksearch service.
report_id: The ID DS has assigned to a report.
"""
for _ in xrange(10):
try:
request = service.reports().get(reportId=report_id)
json_data = request.execute()
if json_data['isReportReady']:
pprint.pprint('The report is ready.')
# For large reports, DS automatically fragments the report into multiple
# files. The 'files' property in the JSON object that DS returns contains
# the list of URLs for file fragment. To download a report, DS needs to
# know the report ID and the index of a file fragment.
for i in range(len(json_data['files'])):
pprint.pprint('Downloading fragment ' + str(i) + ' for report ' + report_id)
download_files(service, report_id, str(i)) # See Download the report.
return
else:
pprint.pprint('Report is not ready. I will try again.')
time.sleep(10)
except HttpError as e:
error = simplejson.loads(e.content)['error']['errors'][0]
# See Response Codes
pprint.pprint('HTTP code %d, reason %s' % (e.resp.status, error['reason']))
break
If the report is not ready
If the report is not ready, Search Ads 360 returns an HTTP 202 response, and the isReportReady
field is false.
{
"kind": "doubleclicksearch#report",
"id": "MTMyNDM1NDYK",
"isReportReady": false,
"request": {
...
},
...
}
When the report is ready
When the report is completed and ready for download, the isReportReady
field is true. The response also has an additional field, files,
that contains URLs for downloading the report files.
{
"kind": "doubleclicksearch#report",
"id": "MTMyNDM1NDYK",
"isReportReady": true,
"request": {
...
},
...
"rowCount": 1329, // Total rows in the report, not counting headers.
"files": [
{
"url": "https://www.googleapis.com/doubleclicksearch/v2/reports/MTMyNDM1NDYK/files/0"
"byteCount": "10242323"
},
{
"url": "https://www.googleapis.com/doubleclicksearch/v2/reports/MTMyNDM1NDYK/files/1"
"byteCount": "10242323"
}
],
}
If report generation fails
If Search Ads 360 is unable to generate a report, it returns one of several HTTP error codes, along with a description.
{
"error" : {
"code" : 410, // Or other error codes.
"message" : "Processing was halted on the backend and will not continue."
}
}
See Response Codes for the list of errors that Search Ads 360 can return.
Download the report
You can download the report by directly hitting each report file URL, or by calling Reports.getFile() with
the report ID and the file number (0 indexed). Search Ads 360 returns the report in a UTF-8 encoded file.
Here are examples of Reports.getFile() requests:
JSON
GET https://www.googleapis.com/doubleclicksearch/v2/reports/MTMyNDM1NDYK/files/0?alt=mediaand
GET https://www.googleapis.com/doubleclicksearch/v2/reports/MTMyNDM1NDYK/files/1?alt=media
Java
/**
* Downloads the shards of a completed report to the given local directory.
* Files are named CampaignReport0.csv, CampaignReport1.csv, and so on.
*/
private static void downloadFiles(
Doubleclicksearch service,
Report report,
String localPath) throws IOException {
for (int i = 0; i < report.getFiles().size(); i++) {
FileOutputStream outputStream =
new FileOutputStream(new File(localPath, "CampaignReport" + i));
service.reports().getFile(report.getId(), i).executeAndDownloadTo(outputStream);
outputStream.close();
}
}
.NET
using api = Google.Apis.Doubleclicksearch.v2;
/// <summary>
/// Downloads the shards of a completed report to the given local directory.
/// Files are named CampaignReport0.csv, CampaignReport1.csv, and so on.
/// </summary>
/// <param name="service">Search Ads 360 API service.</param>
/// <param name="report">Report ID to download.</param>
/// <param name="localPath">Path of the directory to place downloaded files.</param>
private static void DownloadFiles(
api.DoubleclicksearchService service,
api.Data.Report report,
string localPath)
{
Directory.CreateDirectory(localPath);
for (int i = 0; i < report.Files.Count; ++i)
{
string fileName = Path.Combine(
localPath,
string.Format("CampaignReport{0}.csv", i));
Console.WriteLine("Downloading shard {0} to {1}", i, fileName);
using (Stream dst = File.OpenWrite(fileName))
{
service.Reports.GetFile(report.Id, i).Download(dst);
}
}
}
Python
def download_files(service, report_id, report_fragment):
"""Generate and print sample report.
Args:
service: An authorized Doubleclicksearch service.
report_id: The ID DS has assigned to a report.
report_fragment: The 0-based index of the file fragment from the files array.
"""
f = file('report-' + report_fragment + '.csv', 'w')
request = service.reports().getFile_media(reportId=report_id, reportFragment=report_fragment)
f.write(request.execute().decode('utf-8'))
f.close()
Example report
Here's an example of a CSV report. Each report file fragment has its own header row. For details about this format, see RFC 4180.
keywordText,campaignId,landingPageUrl,day,revenue,visits last month,my revenue
google,71700000002104742,http://www.google.com,2012-05-01,10.2,5,20
google,71700000002104742,http://www.google.com,2012-05-02,11,10,60.23
Download Id mapping file
You can download the file that contains ID mappings between the previous Search Ads 360 and the new Search Ads 360. For the requested advertiser, the file includes all children entities (e.g. engine accounts, campaigns, ad groups, etc.) that exist in both the previous Search Ads 360 and the new Search Ads 360.
Python
def download_mapping_file(service, file_name, agency_id, advertiser_id):
"""Generate and save mapping file to a csv.
Args:
service: An authorized Doubleclicksearch service.
file_name: Filename to write the ID mapping file.
agency_id: DS ID of the agency.
advertiser_id: DS ID of the advertiser.
"""
request = service.reports().getIdMappingFile_media(agencyId=agency_id,
advertiserId=advertiser_id)
response = request.execute()
response = response.decode('utf-8')
f = open(file_name + '.csv', 'w')
f.write(response)
f.close()