Google App Engine

Go Runtime Environment

Python |Java |PHP |Go

Welcome to Google App Engine for Go. With App Engine, you can build web applications using the Go Programming Language. Your Go application runs on Google's scalable infrastructure and uses large-scale persistent storage and services.

  1. Introduction
  2. Selecting the Go runtime
  3. Services and packages
  4. Requests and domains
  5. Requests and HTTP
  6. Request headers
  7. Responses
  8. The request timer
  9. The sandbox
  10. Logging
  11. The environment
  12. Quotas and limits
  13. SPDY
  14. Go tools

Introduction

App Engine builds and executes Go application code using a safe "sandboxed" environment. Your app receives web requests, performs work, and sends responses by interacting with this environment.

The Go SDK provides an interface similar to the standard Go http package; writing Go App Engine apps is akin to writing stand-alone Go web servers.

The Go runtime environment uses Go version 1.2. The SDK includes the Go compiler and standard library, so it has no additional dependencies. As with the other runtimes, not all the standard library's functionality is available inside the sandbox. For example, attempts to open a socket or write to a file will return an os.ErrPermission error.

The SDK includes an automated build service to compile your app, so you'll never need to invoke the compiler yourself. And—as with the Python SDK—your app will be automatically re-built whenever you change the source.

The Go runtime environment for App Engine provides full support for goroutines, but not for parallel execution: goroutines are scheduled onto a single operating system thread. This single-thread restriction may be lifted in future versions. Multiple requests may be handled concurrently by a given instance; that means that if one request is, say, waiting for a datastore API call, another request may be processed by the same instance.

Go apps run inside a secure "sandbox" environment with a reduced set of libraries. For instance, an app cannot write data to the local file system or make arbitrary network connections. Instead, apps use scalable services provided by App Engine to store data and communicate over the Internet.

If you haven't already, see the Go Getting Started Guide for an introduction to developing web applications with Go and Google App Engine.

Selecting the Go runtime

App Engine knows to use the Go runtime environment for your application code when you use the tool named appcfg.py from the Go SDK with a configuration file named app.yaml. You select the Go runtime environment using the following configuration elements:

runtime: go
api_version: go1

The first element, runtime, selects the Go runtime environment.

The second element, api_version, selects which version of the Go runtime environment to use. As of this writing, the most recent version of the Go environment is go1. The version identifier will be incremented when the App Engine team releases changes to the environment that may not be compatible with existing code. This means you can continue to use the older APIs until you are able to update your app and change the api_version setting.

Services and packages

Go apps on App Engine can use packages under the appengine package hierarchy to interact with App Engine services and discover App Engine-specific information about their execution environment. See the appengine package reference for more details.

Apps can use the App Engine Datastore for reliable, scalable persistent storage of data. The Go Datastore API provides an idiomatic means of storing and retrieving Go data structures.

The Blobstore API allows your app to store and serve data objects, called blobs, that are much larger than the size allowed for objects in the Datastore service.

The Channel API creates a persistent connection between your application and JavaScript clients, allowing it to send and receive messages in real time without the use of polling.

The App Engine Memcache provides fast, transient distributed storage for caching the results of datastore queries and calculations.

The Task Queue API allows your app to perform background processing by inserting tasks (modeled as web hooks) into a queue.

Apps use the URL Fetch service to access resources over the web, and to communicate with other hosts using the HTTP and HTTPS protocols.

An application can use Google Accounts for user authentication. The user account creation and sign-in is handled by Google Accounts, and a user that already has a Google account (such as a GMail account) can use that account with your app. An app can detect when the current user is signed in, and can access the user's email address.

Requests and domains

App Engine determines that an incoming request is intended for your application using the domain name of the request. A request whose domain name is http://your_app_id.appspot.com is routed to the application whose ID is your_app_id. Every application gets an appspot.com domain name for free.

appspot.com domains also support subdomains of the form subdomain-dot-your_app_id.appspot.com, where subdomain can be any string allowed in one part of a domain name (not .). Requests sent to any subdomain in this way are routed to your application.

You can set up a custom top-level domain using Google Apps. With Google Apps, you assign subdomains of your business's domain to various applications, such as Google Mail or Sites. You can also associate an App Engine application with a subdomain. For convenience, you can set up a Google Apps domain when you register your application ID, or later from the Administrator Console. See Deploying your Application on your Google Apps URL for more information.

Requests for these URLs all go to the version of your application that you have selected as the default version in the Administration Console. Each version of your application also has its own URL, so you can deploy and test a new version before making it the default version. The version-specific URL uses the version identifier from your app's configuration file in addition to the appspot.com domain name, in this pattern: http://version_id-dot-latest-dot-your_app_id.appspot.com You can also use subdomains with the version-specific URL: http://subdomain-dot-version_id-dot-latest-dot-your_app_id.appspot.com

The domain name used for the request is included in the request data passed to the application. If you want your app to respond differently depending on the domain name used to access it (such as to restrict access to certain domains, or redirect to an official domain), you can check the request data (such as the Host request header) for the domain from within the application code and respond accordingly.

If your app uses backends, you can address requests to a specific backend and a specific instance with that backend. For more information about backend addressability, please see Properties of Backends.

Please note that in April of 2013, Google stopped issuing SSL certificates for double-wildcard domains hosted at appspot.com (i.e. *.*.appspot.com). If you rely on such URLs for HTTPS access to your application, please change any application logic to use "-dot-" instead of ".". For example, to access version "1" of application "myapp" use "https://1-dot-myapp.appspot.com" instead of "https://1.myapp.appspot.com." If you continue to use "https://1.myapp.appspot.com" the certificate will not match, which will result in an error for any User-Agent that expects the URL and certificate to match exactly.

Requests and HTTP

The App Engine Go API uses the standard http package as an interface between your Go program and the App Engine servers. When App Engine receives a web request for your application, it invokes the http.Handler associated with the request URL. (The URL must also be specified as a Go handler in the application's app.yaml configuration file.)

App Engine uses multiple web servers to run your application, and automatically adjusts the number of servers it is using to handle requests reliably. A given request may be routed to any server, and it may not be the same server that handled a previous request from the same user. Multiple requests may be handled concurrently by a given server.

The following example is a complete Go app that outputs a hard-coded HTML string to the user.

package hello

import (
    "fmt"
    "net/http"
)

func init() {
    http.HandleFunc("/", hello)
}

func hello(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "<h1>Hello, world</h1>")
}

Request headers

An incoming HTTP request includes the HTTP headers sent by the client. For security purposes, some headers are sanitized or amended by intermediate proxies before they reach the application.

The following headers are removed from the request:

  • Accept-Encoding
  • Connection
  • Keep-Alive
  • Proxy-Authorization
  • TE
  • Trailer
  • Transfer-Encoding

In addition, the header Strict-Transport-Security is removed from requests served to any domains other than appspot.com or *.appspot.com.

These headers relate to the transfer of the HTTP data between the client and server, and are transparent to the application. For example, the server may automatically send a gzipped response, depending on the value of the Accept-Encoding request header. The application itself does not need to know which content encodings the client can accept.

As a service to the app, App Engine adds some headers:

X-AppEngine-Country
Country from which the request originated, as an ISO 3166-1 alpha-2 country code. App Engine determines this code from the client's IP address.
X-AppEngine-Region
Name of region from which the request originated. This value only makes sense in the context of the country in X-AppEngine-Country. For example, if the country is "US" and the region is "ca", that "ca" means "California", not Canada.
X-AppEngine-City
Name of the city from which the request originated. For example, a request from the city of Mountain View might have the header value mountain view.
X-AppEngine-CityLatLong
Latitude and longitude of the city from which the request originated. This string might look like "37.386051,-122.083851" for a request from Mountain View.

Responses

App Engine calls the handler with a Request and a ResponseWriter, then waits for the handler to write to the ResponseWriter and return. When the handler returns, the data in the ResponseWriter's internal buffer is sent to the user.

This is practically the same as when writing normal Go programs that use the http package. The one notable difference is that App Engine does not support streaming data in response to a single request.

Dynamic responses are limited to 32MB. If a script handler generates a response larger than this limit, the server sends back an empty response with a 500 Internal Server Error status code. This limitation does not apply to responses that serve data from the Blobstore or Google Cloud Storage.

If the client sends HTTP headers with the request indicating that the client can accept compressed (gzipped) content, App Engine compresses the response data automatically and attaches the appropriate response headers. It uses both the Accept-Encoding and User-Agent request headers to determine if the client can reliably receive compressed responses. Custom clients can indicate that they are able to receive compressed responses by specifying both Accept-Encoding and User-Agent headers with a value of gzip. The Content-Type of the response is also used to determine whether compression is appropriate; in general, text-based content types are compressed, whereas binary content types are not.

The following headers are ignored and removed from the response:

  • Connection
  • Content-Encoding
  • Content-Length
  • Date
  • Keep-Alive
  • Proxy-Authenticate
  • Server
  • Trailer
  • Transfer-Encoding
  • Upgrade

In addition, the header Strict-Transport-Security is removed from responses served from any domains other than *.appspot.com.

Headers with non-ASCII characters in either the name or value are also removed. In addition, the following headers are added or replaced in the response:

Cache-Control, Expires and Vary

These headers specify caching policy to intermediate web proxies (such as Internet Service Providers) and browsers. If your script sets these headers, they will usually be unmodified, unless the response has a Set-Cookie header, or is generated for a user who is signed in using an administrator account. Static handlers will set these headers as directed by the configuration file. If you do not specify a Cache-Control, the server may set it to private, and add a Vary: Accept-Encoding header.

If you have a Set-Cookie response header, the Cache-Control header will be set to private (if it is not already more restrictive) and the Expires header will be set to the current date (if it is not already in the past). Generally, this will allow browsers to cache the response, but not intermediate proxy servers. This is for security reasons, since if the response was cached publicly, another user could subsequently request the same resource, and retrieve the first user's cookie.

Content-Encoding
Depending upon the request headers and response Content-Type, the server may automatically compress the response body, as described above. In this case, it adds a Content-Encoding: gzip header to indicate that the body is compressed.
Content-Length or Transfer-Encoding
The server always ignores the Content-Length header returned by the application. It will either set Content-Length to the length of the body (after compression, if compression is applied), or delete Content-Length, and use chunked transfer encoding (adding a Transfer-Encoding: chunked header).
Content-Type

If you do not explicitly set this header, the http.ResponseWriter class detects the content type from the start of the response body, and sets the Content-Type header accordingly.

Date
Set to the current date and time.
Server
Set to Google Frontend. The development server sets this to Development/x, where x is the version number.

If you access your site while signed in using an administrator account, App Engine includes per-request statistics in the response headers:

X-AppEngine-Estimated-CPM-US-Dollars
An estimate of what 1,000 requests similar to this request would cost in US dollars.
X-AppEngine-Resource-Usage
The resources used by the request, including server-side time as a number of milliseconds.

Responses with resource usage statistics will be made uncacheable.

If the X-AppEngine-BlobKey header is in the application's response, it and the optional X-AppEngine-BlobRange header will be used to replace the body with all or part of a blobstore blob's content. If Content-Type is not specified by the application, it will be set to the blob's MIME type. If a range is requested, the response status will be changed to 206 Partial Content, and a Content-Range header will be added. The X-AppEngine-BlobKey and X-AppEngine-BlobRange headers will be removed from the response. You do not normally need to set these headers yourself, as the blobstore_handlers.BlobstoreDownloadHandler class sets them. See Serving a Blob for details.

The request timer

A request handler has a limited amount of time to generate and return a response to a request, typically around 60 seconds. Once the deadline has been reached, the request handler is interrupted.

When a Go request handler exceeds the deadline, its process is terminated and the runtime environment returns an HTTP 500 Internal Server Error to the client.

While a request can take as long as 60 seconds to respond, App Engine is optimized for applications with short-lived requests, typically those that take a few hundred milliseconds. An efficient app responds quickly for the majority of requests. An app that doesn't will not scale well with App Engine's infrastructure.

Refer to Dealing with DeadlineExceededErrors for common DeadlineExceededError causes and suggested workarounds.

The sandbox

To allow App Engine to distribute requests for applications across multiple web servers, and to prevent one application from interfering with another, the application runs in a restricted "sandbox" environment. In this environment, the application can execute code, store and query data in the App Engine datastore, use the App Engine mail, URL fetch and users services, and examine the user's web request and prepare the response.

An App Engine application cannot:

  • write to the filesystem. Applications must use the App Engine datastore for storing persistent data. Reading from the filesystem is allowed, and all application files uploaded with the application are available.

  • respond slowly. A web request to an application must be handled within a few seconds. Processes that take a very long time to respond are terminated to avoid overloading the web server.

  • make other kinds of system calls.

The Go runtime restricts the use of standard library packages that would violate the sandbox policies. Such functions have been replaced by stubs that return an os.EPERM error or removed entirely.

Logging

The Go API's Context object has Debugf, Infof, Warningf, Errorf, and Criticalf methods that send log messages to the App Engine web server.

Log data for your application can be viewed and analyzed using the Administration Console, or downloaded using appcfg.py request_logs.

The following example demonstrates an HTTP handler that constructs an appengine.Context object from the *http.Request and logs the requested URL.

package hello

import (
    "appengine"
    "net/http"
)

func init() {
    http.HandleFunc("/", Logger)
}

func Logger(w http.ResponseWriter, r *http.Request) {
    c := appengine.NewContext(r)
    c.Infof("Requested URL: %v", r.URL)
}

The environment

The Go runtime provides access to environment information through the appengine.Context interface. See the appengine package reference for details.

Quotas and limits

Google App Engine automatically allocates resources to your application as traffic increases. However, this is bound by the following restrictions:

  • App Engine reserves automatic scaling capacity for applications with low latency, where the application responds to requests in less than one second. Applications with very high latency (over one second per request for many requests) and high throughput require Silver, Gold, or Platinum support. Customers with this level of support can request higher throughput limits by contacting their support representative.
  • Applications that are heavily CPU-bound may also incur some additional latency in order to efficiently share resources with other applications on the same servers. Requests for static files are exempt from these latency limits.

Each incoming request to the application counts toward the Requests limit. Data sent in response to a request counts toward the Outgoing Bandwidth (billable) limit.

Both HTTP and HTTPS (secure) requests count toward the Requests, Incoming Bandwidth (billable), and Outgoing Bandwidth (billable) limits. The Quota Details page of the Admin Console also reports Secure Requests, Secure Incoming Bandwidth, and Secure Outgoing Bandwidth as separate values for informational purposes. Only HTTPS requests count toward these values. See the Quotas page, and the "Quota Details" section of the Admin Console for more information.

In addition to system-wide safety limits, the following limits apply specifically to the use of request handlers:

Limit Amount
request size 32 megabytes
response size 32 megabytes
request duration 60 seconds
maximum total number of files (app files and static files) 10,000 total
1,000 per directory
maximum size of an application file 32 megabytes
maximum size of a static file 32 megabytes
maximum total size of all application and static files first 1 gigabyte is free
$ 0.026 per gigabyte per month after first 1 gigabyte

SPDY

App Engine applications will automatically use the SPDY protocol when accessed over SSL by a browser that supports SPDY. This is a replacement for HTTP designed by Google and intended to reduce the latency of web page downloads. The use of SPDY should be entirely transparent to both applications and users (applications can be written as if normal HTTP was being used). For more information, see the SPDY project page.

Go tools

The App Engine Go SDK uses the tools from the Python SDK for testing your application and uploading application files.

The development server runs your application on your local computer for testing your application. The server simulates the App Engine Datastore, services and sandbox restrictions. The development server can also generate configuration for Datastore indices based on the queries the app performs during testing.

A multipurpose tool called appcfg.py handles all command-line interaction with your application running on App Engine. The appcfg.py tool can upload your application to App Engine, or just update the datastore index configuration so you can build new indexes before updating the code. It can also download the app's log data, so you can analyze your app's performance using your own tools.

The SDK also includes some tools from the Go tool chain:

Godoc extracts and generates documentation for Go programs. Godoc uses the source code of the Go standard library and Go App Engine SDK that is included in the SDK zip file.

Gofmt formats Go programs.

Authentication required

You need to be signed in with Google+ to do that.

Signing you in...

Google Developers needs your permission to do that.