Google's user agent for API push notifications
APIs-Google (as opposed to
If you administer a website that allows developers to access Google's APIs, you
might see requests in your access logs that specify a user-agent of
User-Agent: APIs-Google (+https://developers.google.com/webmasters/APIs-Google.html)
APIs-Google is the user agent used by Google APIs to deliver push notification messages. Application developers can request these notifications to avoid the need for continually polling Google's servers to find out if the resources they are interested in have changed. To make sure nobody abuses this service, Google requires developers to prove that they own the domain before allowing them to register a URL with a domain as the location where they want to receive messages.
For webmasters: APIs-Google and your site
How APIs-Google accesses your site
APIs-Google sends each push notification using an HTTPS POST request. If the request fails due to an error condition that might be temporary, APIs-Google will try sending the notification again. If the request still doesn't succeed, it will continue to retry—based on an exponential backoff schedule—up to a maximum of several days.
For example, if the application fails to acknowledge each push notification request promptly with one of the success HTTP status codes (such as 200), APIs-Google might consider this a timeout error and start retrying the notification. There are also a number of retryable error HTTP status codes; for example 503 is used when the application is temporarily down or overloaded.
The rate at which APIs-Google accesses your site varies by how many push notification requests were created for servers on your site, by how fast the monitored resources are getting updated, and by the number of retries occurring. As a result, the APIs-Google traffic patterns can be consistent in some scenarios, but in other scenarios the traffic can be sporadic or spiky.
Blocking APIs-Google from calling into your site
If you administer a domain that has subdomains or URL subspaces that are owned
or administered separately, one of those other owners may have set up
applications that use push notifications. If you want to block APIs-Google, we
suggest that you first attempt to contact anyone who might have set up an
application like this. Alternatively, you can use the standard robots.txt
directives to block APIs-Google from
accessing your site. The user-agent to specify in the robots.txt file is
APIs-Google. Since APIs-Google does not follow
Googlebot directives, it can be controlled separately from other
If you decide to create a robots.txt file, there may be a small delay before APIs-Google discovers your changes. If APIs-Google continues to send messages to your site several days after you've blocked it in robots.txt, check that the robots.txt is in the correct location. It must be in the top directory of the server, for example, www.example.com/robots.txt; placing the file in a subdirectory won't have any effect.
Making sure your site is APIs-Google friendly
Since APIs-Google uses HTTPS to deliver push notifications, it requires your site to have a valid SSL certificate. Invalid certificates include:
- Self-signed certificates.
- Certificates signed by an untrusted source.
- Certificates that have been revoked.
Also, to avoid unnecessary retry requests, the applications must be well-designed and respond promptly (within seconds) to notification messages.
Handling potentially fake requests
The IP addresses used by APIs-Google change from time to time. In addition,
anyone can set the user-agent to whatever they like. The best way to confirm that such accesses are from Google is to use a reverse DNS lookup that's similar to the kind you would use to verify that a bot accessing your server really is Googlebot. In this case, search in your logs for any IP addresses identified with the
APIs-Google user agent. The reverse DNS lookup should identify a "googlebot.com" domain.