Google Compute Engine

Load Balancing

Google Compute Engine offers server-side load balancing so you can distribute incoming network traffic across multiple virtual machine instances. Load balancing is useful because it helps you support heavy traffic and provides redundancy to avoid failures.

Google Compute Engine load balancing allows you to create forwarding rule objects that match and direct certain types of traffic to a load balancer. A target pool object controls the load balancer and contains the set of instances to send traffic to. Target pools also contain a health check object to determine the health of instances in the corresponding target pool. For example, a forwarding rule may match TCP traffic to port 80 on a public IP address of 1.2.3.4. Any traffic destined to that IP, protocol, and port is matched by this forwarding rule and directed to the virtual machine instances in a target pool associated with the forwarding rule.

A target pool contains a set of virtual machine instances that handles incoming traffic from a forwarding rule. Google Compute Engine will route any traffic destined for an IP and port that is being served by a forwarding rule to instances in the specified target pools. Traffic is then spread across all instances in the target pool. A target pool can only contain virtual machines in the same region, and that region must be the same as any forwarding rules that reference it. You can associate target pools in different zones with a single forwarding rule to provide protection from planned or unplanned zone outages.

The following diagram is an example load balancing setup, where there are two forwarding rule objects that direct traffic to a single target pool.

Click to enlarge

Load distribution algorithm

By default, to distribute traffic to instances, Google Compute Engine picks an instance based on a hash of the source IP and port and the destination IP and port. Incoming TCP connections are spread across instances and each new connection may go to a different instance. All packets for a connection are directed to the same instance until the connection is closed.

It is also possible to change the default hash method that determines how traffic is distributed. For more information, see sessionAffinity.

Contents

Prerequisites

In order to use load balancing, you must:

  1. Use an image that is dated 2013-07-23 or newer for your instances.

    If your instances are using an older image, you must recreate your instances using a newer image, or, if you're using a custom image and do not want to use a new image, you will need to explicitly add the IP address from your forwarding rules to your operating system's virtual interface so it can accept connections destined to that load balanced IP. You will need to do this for every instance that should receive load balanced traffic on that IP address. For more details, see using an older image section.

  2. Use gcutil version 1.8.3 or newer.

    Download the newest version if you have not already.

Quickstart

Note: This quickstart assumes you are familiar with bash.

This quickstart guide provides step-by-step instructions on how to set up the Google Compute Engine load balancing service to dispatch traffic to a few virtual machines. In addition to this quickstart, you can also start using the service with the Google Developers Console.

The rest of this quickstart discusses how to:

  1. Set up your instances to run Apache.
  2. Configure the load balancing service.
  3. Send some traffic to your instances.

This quickstart demonstrates each of these steps by setting up an Apache server on a virtual machine instance that is used by a target instance to handle traffic from a forwarding rule.

Set Up Your instances to run Apache

To begin this quickstart, we're going to create some instances with Apache installed.

  1. Create some startup scripts for your new instances.

    Depending on your operating system, your startup script contents might differ:

    • If you are planning to using Debian on your instances, run the following command:
      me@local:~$ echo "apt-get update && apt-get install -y apache2 && hostname > /var/www/index.html" > \
      $HOME/lb_startup.sh
    • If you are planning on use CentOS for your instances, run the following command:
      me@local:~$ echo "yum -y install httpd && service httpd restart && hostname > /var/www/html/index.html" > \
      $HOME/lb_startup.sh
  2. Create a tag for your future virtual machines, so you can apply a firewall to them later:
    me@local~$: TAG="www-tag"
  3. Choose a zone and a region for your virtual machines:
    me@local:~$ ZONE="us-central1-b"
    me@local:~$ REGION="us-central1"
  4. Create three new virtual machines:
    me@local:~$ gcutil --project=<project-id> addinstance www1 www2 www3 --zone=$ZONE \
                       --tags=$TAG --metadata_from_file=startup-script:$HOME/lb_startup.sh
  5. Create a firewall rule to allow external traffic to this virtual machine instances:
    me@local:~$ gcutil --project=<project-id> addfirewall www-firewall --target_tags=$TAG --allowed=tcp:80

Now that your virtual machine instances are prepared, you can start setting up your load balancing configuration. You can verify that your instances are running by sending a curl request to each of the instance's external IP address:

me@local:~$ curl <ip-address>

To get the IP addresses of your instances, use gcutil listinstances.

Configure the load balancing service

Next, you will set up the load balancing service.

  1. Add a HTTP health check object.

    For this example, you are going to use the default settings for the health check mechanism, but you can customize this on your own.

    me@local:~$ gcutil --project=<project-id> addhttphealthcheck basic-check
  2. Add a target pool in the same region as your virtual machine instances.

    The region of your virtual machine instances is us-central1. You are also going to use your newly-created health check object for this target pool.

    me@local:~$ gcutil --project=<project-id> addtargetpool www-pool --region=$REGION --health_checks=basic-check \
                       --instances=$ZONE/instances/www1,$ZONE/instances/www2,$ZONE/instances/www3

    You can also add or remove instances after you've created the target pool. Instances within a target pool must belong to the same region but can be spread out across different zones in the same region. For example, you can have instances in zone us-central1-a and instances in zone us-central1-b in one target pool because they are in the same region, us-central1.

  3. Add a forwarding rule serving on behalf of an external IP and port range, that points to your target pool.

    You can choose to use a reserved static IP address or an ephemeral IP address assigned by Google Compute Engine for your forwarding rules. For this example, you are going to use an ephemeral IP address by leaving out the --ip flag in the command:

    me@local:~$ gcutil --project=<project-id> addforwardingrule www-rule --region=$REGION --port_range=80 --target=www-pool

Now that you have configured your load balancing service, you can start sending traffic to the forwarding rule and watch the traffic be dispersed to different instances.

Send traffic to your instances

To start sending traffic to your forwarding rule, you need to grab the forwarding rule's external IP address:

me@local:~$ gcutil --project=<project-id> listforwardingrules
+----------+-------------+-------------+---------+----------+------------+----------------------------------+
|   name   | description |   region    |   ip    | protocol | port-range |              target              |
+----------+-------------+-------------+---------+----------+------------+----------------------------------+
| www-rule |             | us-central1 | 1.2.3.4 | TCP      | 80-80      | us-central1/targetPools/www-pool |
+----------+-------------+-------------+---------+----------+------------+----------------------------------+

Assign the external IP address for the www-rule forwarding rule to the IP alias:

me@local:~$ IP="1.2.3.4"

Next, use curl to access the IP address. The response will alternate randomly among the three instances. If your response is initially unsuccessful, wait 30 seconds or so for the configuration to be fully loaded and for your instances to be marked healthy before trying again:

me@local:~$ while true; do curl -m1 $IP; done

Forwarding Rules

Forwarding rules work in conjunction with target pools and target instances to support load balancing and protocol forwarding features. To use load balancing and protocol forwarding, you must create a forwarding rule that directs traffic to specific target pools (for load balancing) or target instances (for protocol forwarding). It is not possible to use either of these features without a forwarding rule.

Forwarding Rule resources live in the Forwarding Rules collection. Each forwarding rule matches a particular IP address, protocol, and optionally, port range to a single target pool or target instance. When traffic is sent to an external IP address that is served by a forwarding rule, the forwarding rule directs that traffic to the corresponding target pool or target instances. You can create up to 50 forwarding rule objects per project.

A forwarding rule object contains the following properties:

  • name - [Required] The name of the forwarding rule. The name must be unique in this project, from 1-63 characters long and match the regular expression: [a-z]([-a-z0-9]*[a-z0-9])? which means the first character must be a lowercase letter, and all following characters must be a dash, lowercase letter, or digit, except the last character, which cannot be a dash.
  • region - [Required] The region where this forwarding rule resides. For example:
    "region" : "https://www.googleapis.com/compute/v1/projects/<project-id>/regions/<region-name>"
  • IPAddress - [Optional] A single IP address this forwarding rule matches to. All traffic directed to this IP address will be handled by this forwarding rule. The IP address must be a static reserved IP address or, if left empty, an ephemeral IP address is assigned to the forwarding rule upon creation. For example:
    "IPAddress" : "1.2.3.4"
  • target [Required] - The Target Pool or Target Instance resource that this forwarding rule directs traffic to. Must be a fully-qualified URL such as:
    http://www.googleapis.com/compute/v1/projects/<project-id>/regions/<region>/targetPools/<target-pool-name>

    For target instances, the URL will look like:

    http://www.googleapis.com/compute/v1/projects/<project-id>/zones/<zone>/targetInstances/<target-pool-name>

    The target pool or target instance must exist before you create your forwarding rule and must reside in the same region as the forwarding rule.

  • IPProtocol - [Optional] The type of protocol that this forwarding rule matches. Valid values are:

    If left empty, this field will default to TCP. Also note that certain protocols can only be used with target pools or target instances:

    • If you use ESP, AH, or SCTP, you must specify a target instance. It is not possible to specify a target pool when using these protocols.
    • If you use TCP or UDP, you can specify either a target pool or a target instance.
  • portRange - [Optional] A single port or single contiguous port range, ranging from low to high for which this forwarding rule matches. Packets of the specified protocol sent to these ports will be forwarded on to the appropriate target pool or target instance. If this field is left empty, then the forwarding matches traffic for all ports for the specified protocol. For example:
    "portRange" : ["200-65536"]

    You can only specify this field for TCP, UDP, and SCTP protocols.

Adding a Forwarding Rule

To add a new forwarding rule, you can use the gcutil addforwardingrule command or create a POST request to the ForwardingRules collection. To create a forwarding rule using gcutil:

gcutil --project=<project-id> addforwardingrule <forwarding-rule-name> \
       [--description=<description-text>] --ip=<external-ip-address> \
       [--target_pool=<target-pool> --target_instance=<target-instance> \
       --protocol=<protocol>] [--port_range=<port-range>]  \
       --region=<region>

Important Flags and Parameters:

--project=<project-id>
[Required] The project ID for this forwarding rule.
<forwarding-rule-name>
[Required] The name for this forwarding rule.
--description=<description-text>
[Optional] The description for this forwarding rule.
--ip=<external-ip-address>
[Optional] An external static IP that this forwarding rule serves on behalf of. This can be a reserved static IP, or if left blank or unspecified, the default is to assign an ephemeral IP address. Multiple forwarding rules can use the same IP address as long as their port range and protocol do not overlap. For example, --ip="1.2.3.106".
--target_pool=<target-pool>
[Optional] You must specify only one of --target_pool or --target_instance. Specifies a target pool that handles traffic from this forwarding rule. The target pool must already exist before you can use it for a forwarding rule and it must reside in the same region as the forwarding rule. This is specifically for load balancing. For example: 'mytargetpool'.
--target_instance=<target-instance>
[Optional] You must specify only one of --target_pool or --target_instance. Specifies target instance that handles traffic from this forwarding rule. This is specifically for protocol forwarding. You must specify only one of either --target_pool or --target_instance..
--protocol=<protocol>
[Optional] The protocol that this forwarding rule is handling. If left empty, this field will default to TCP. Also note that certain protocols can only be used with target pools or target instances:
  • If you use ESP, AH, or SCTP, you must specify a target instance. It is not possible to specify a target pool when using these protocols.
  • If you use TCP or UDP, you can specify either a target pool or a target instance.
--port_range=<port-range>
[Optional] The list of ports for which this forwarding rule is responsible for. Packets of the specified protocol sent to these ports will be forwarded on to the appropriate target pool. If this field is left empty, then the forwarding rule sends traffic for all ports for the specified protocol. Can be a single port, or a range of ports. You can only set this field for TCP, UDP, and SCTP protocols.
--region=<region>
[Required] The region where this forwarding rule should reside. For example, us-central1. This must be the same region as the target pool.

To add a forwarding rule using the API, perform a POST request to the following URI:

https://www.googleapis.com/compute/v1/projects/<project-id>/regions/<region>/forwardingRules

Your request body should contain the following fields:

 bodyContent = {
   "name": <name>,
   "IPAddress": <external-ip>,
   "IPProtocol": <tcp-or-udp>,
   "portRange": <port-range>,
   "target": <uri-to-target-resource>
 }

Listing Forwarding Rules

To get a list of forwarding rules, use gcutil listforwardingrules.

gcutil --project=<project-id> listforwardingrules [--region=<region>]

Important Flags and Parameters:

--project=<project-id>
[Required] The project for which you want to list your forwarding rules.
--region=<region>
[Optional] The region for which you want to list forwarding rules. If not specified, all forwarding rules across all regions are listed.

In the API, make an empty GET request to the following URI:

https://www.googleapis.com/compute/v1/project/<project-id>/regions/<region>/forwardingRules

Getting Forwarding Rules

To get information about a single forwarding rules, use gcutil getforwardingrule.

gcutil --project=<project-id> getforwardingrule <forwarding-rule-name> --region=<region>

Important Flags and Parameters:

--project=<project-id>
[Required] The project for which you want to get your forwarding rule.
--region=<region>
[Required] The region where the forwarding rule resides.
<forwarding-rule-name>
[Required] The forwarding rule name.

In the API, make an empty GET request to the following URI:

https://www.googleapis.com/compute/v1/project/<project-id>/regions/<region>/forwardingRules/<forwarding-rule-name>

Updating the Forwarding Rule Target

If you have already created a forwarding rule but want to change the target pool that the forwarding rule is using, you can do so using the gcutil setforwardingruletarget command:

gcutil --project=<project-id> setforwardingruletarget <forwarding-rule-name> \
       --region=<region> [--target_pool=<target-pool-name> --target_instance=<target-instance-name>]

Important Flags and Parameters:

--project=<project-id>
[Required] The project for this request.
<forwarding-rule-name>
[Required] The forwarding rule name.
--region=<region>
[Required] The region where the forwarding rule resides.
--target_pool=<target-pool>
[Optional] You must specify only one of --target_pool or --target_instance. Specifies a target pool to add or update. The target pool must already exist before you can use it for a forwarding rule and it must reside in the same region as the forwarding rule. This is specifically for load balancing. For example: 'mytargetpool'.
--target_instance=<target-instance>
[Optional] You must specify only one of --target_pool or --target_instance. Specifies target instance to add or update for this forwarding rule. This is specifically for protocol forwarding. You must specify only one of either --target_pool or --target_instance..

In the API, make a POST request to the following URI:

https://www.googleapis.com/compute/v1/projects/<project-id>/regions/<region>/forwardingRules/<forwardingRule>/setTarget

Your request body should contain the URL to the target instance or target pool resource you want to set. For instance, for target pools, the URI format should be:

body = {
  "target": "https://www.googleapis.com/compute/v1/projects/<project-id>/regions/<region>/targetPools/<target-pool-name>"
}

Deleting Forwarding Rules

To delete a forwarding rule, use the gcutil deleteforwardingrule command:

gcutil --project=<project-id> deleteforwardingrule [-f] <forwarding-rule-name> [--region=<region>]

Important Flags and Parameters:

--project=<project-id>
[Required] The project ID where this forwarding rule lives.
-f, --force
[Optional] Bypass the confirmation prompt to delete this forwarding rule.
<forwarding-rule-name>
[Required] The forwarding rule to delete.
--region=<region>
[Optional] The region of this forwarding rule. If you do not specify this flag, gcutil performs an extra API request to determine the region for your forwarding rule.

To delete a forwarding rule from the API, make a DELETE request to the following URI, with an empty request body:

https://www.googleapis.com/compute/v1/project/<project-id>/regions/<region>/forwardingRules/<forwarding-rule-name>

Target pools

Note: If you intend for your target pool to contain a single virtual machine instance, you should consider using the Protocol Forwarding feature instead.

A Target Pool resource defines a group of instances that should receive incoming traffic from forwarding rules. When a forwarding rule directs traffic to a target pool, Google Compute Engine picks an instance from these target pools based on a hash of the source IP and port and the destination IP and port. See the Load Distribution Algorithm for more information about how traffic is distributed to instances.

Target pools can only be used with forwarding rules that handle TCP and UDP traffic. For all other protocols, you must create a target instance. You must create a target pool before you can use it with a forwarding rule. Each project can have up to 50 target pools. A target pool is made up of the following properties:

name
[Required] The name of this target pool. The name must be unique in this project, from 1-63 characters long and match the regular expression: [a-z]([-a-z0-9]*[a-z0-9])?, which means the first character must be a lowercase letter, and all following characters must be a dash, lowercase letter, or digit, except the last character, which cannot be a dash.
description
[Optional] A user-defined description of this target pool.
region
[Required] The fully-qualified URL to the region where this target pool should live. This should be the same region where your desired instances will live. For example:
"region" : "https://www.googleapis.com/compute/v1/projects/<project-id>/regions/<region-name>"
healthChecks[ ]
[Optional] An optional list of health checks for this target pool. See Health Checking for more information.
instances[ ]
[Required] A list of instance URLs that should handle traffic for this target pool. All instances must reside in the same region as the target pool, but instances can belong to different zones within a single region. For example:
"instances" : [
  "https://www.googleapis.com/compute/v1/projects/<project-id>/zones/us-central1-a/instances/<instance-name>",
  "https://www.googleapis.com/compute/v1/projects/<project-id>/zones/us-central1-b/instances/<instance-name>"
]
sessionAffinity
[Optional] Describes the method used to select a backend virtual machine instance. You can only set this value during the creation of the target pool. Once set, you cannot modify this value. By default, a 5-tuple method is used and the default value for this field is NONE. The 5-tuple hash method selects a backend based on:
  • Layer 4 Protocol (e.g. TCP, UDP)
  • Source / Destination IP
  • Source / Destination Port

5-tuple hashing provides a good distribution of traffic across many virtual machines but if you wanted to use a single backend with a specific client ('stick' a client to a single virtual machine instance), you can also specify the following options:

CLIENT_IP_PROTO
3-tuple hashing, which includes the source / destination IP and network protocol
CLIENT_IP
2-tuple hashing, which includes the source / destination IP

In general, if you select a 3-tuple or 2-tuple method, it will provide for better session affinity than the default 5-tuple method, at the cost of possibly unequal distribution of traffic.

Caution: If a large portion of your clients are behind a proxy server, you should not use the sessionAffinity feature because it would force all clients behind the proxy to be pinned to a specific backend.

backupPool
[Optional] A fully-qualified URL to another target pool resource. You must also define failoverRatio to use this feature. If the ratio of healthy virtual machines in your primary target pool falls below the failoverRatio, Google Compute Engine sends traffic to your backup pool. You can only provide one backup pool per primary target pool. The backup pool must be in the same region as the primary target pool. If the ratio of healthy instances in your primary target pool falls below your configured failover ratio, Google Compute Engine uses the following rules to route your traffic:
  1. If a primary target pool is declared unhealthy (falls below the failover ratio), traffic will be sent to healthy instances in the backup pool.
  2. If the primary target pool is declared unhealthy, but there are no remaining healthy instances in the backup pool, traffic is sent to the remaining healthy instances in the primary pool.
  3. If the primary pool is unhealthy and there are no remaining healthy instances in either pools, traffic will be sent to all instances in the primary pool so as to not drop traffic.
  4. If the primary pool doesn't contain any instances, and none of the instances in the backup pool are healthy, traffic will be sent to all instances in the backup pool so as to not drop any traffic.

At most, only one level of failover is supported. For example, if target pool A has backup pool B and back pool B has a backup pool C, then traffic intended for target pool A can only reach up to backup pool B and not C.

Note: If you intend to use backup target pools, you should set up health checks because backup target pools will not work correctly without health checks enabled.

failoverRatio - [Optional]
A float between 0.0 and 1.0, which determines when this target pool is declared unhealthy. For example, if this value is set to .1, then this target pool is declared unhealthy if the number of healthy instances is below .1 (10%). You must define this if you define the backupPool field.

The diagram below helps visualize how the failover ratio and backup pool works together:

Adding a target pool

To add a target pool, use the gcutil addtargetpool command or make a request to the target pool URI. To use gcutil:

gcutil --project=<project-id> addtargetpool <target-pool-name> \
       --region=<region> [--health_checks=<health-checks> \
       --backup_pool=<backup-pool> --failover_ratio=<failover-ratio>
       --session_affinity=<session-affinity>] --instances=<instance-names>

Important flags and parameters:

--project=<project-id>
[Required] The project ID for this target pool.
--region=<region>
[Required] The region to create this target pool in. For example, --region=us-central1.
<target-pool-name>
[Required] The name for this target pool. The name must be unique in this project, from 1-63 characters long and match the regular expression: [a-z]([-a-z0-9]*[a-z0-9])?, which means the first character must be a lowercase letter, and all following characters must be a dash, lowercase letter, or digit, except the last character, which cannot be a dash.
--health_checks=<health-checks>
[Optional] The HTTP health check resources to use for instances this target pool. The load balancing service supports, at most, one health check attached to a target pool. If this is left empty, which is the default value, traffic is sent to all instances within the pool as if the instances were all healthy. The health status for this pool will appear as unhealthy as a warning that the target pool is not protected from an instance failure or application failure.
--backup_pool=<backup-pool>
[Optional] Specifies the backup target pool to use if the ratio of healthy instances in this target pool falls under the --failover_ratio. The backup pool must be in the same region as the target pool. If you set this flag, you must also set the --failover_ratio flag. See backupPools for more information.

Note: If you intend to use backup target pools, you should set up health checks because backup target pools will not work correctly without health checks enabled.

--failover_ratio=<failover-ratio>
[Optional] Specifies the ratio of the healthy instances in your target pool before it is considered unhealthy. You must declare a float between 0.0 and 1.0. For example, if you set this to .1 (10%), and the percentage of healthy instances in this target pool falls below 10%, Google Compute Engine will start using the backup pool. You must specify this flag if you set the --backup_pool flag. See failoverRatio for more information.
--session_affinity=<session-affinity>
[Optional] Specifies the session affinity option for the load balancing service. Session affinity describes how the load balancing service should connect external clients to backend machines. By default, this is NONE, which means that Google Compute Engine uses a 5-tuple hashing method to connect traffic from external clients to backend instances. Other available values are the CLIENT_IP value which means that the service should use a 2-tuple hashing method to connect external clients to a single backend or a CLIENT_IP_PROTO value, which means that the service should use a 3-tuple hashing method to connect external clients to a backend. For more information, see sessionAffinity.
--instances=<instance-names>
[Optional] A comma-separated list of instances to use for this target pool. For example, --instances=us-central1-a/instances/myinst,us-central1-b/instances/testinstance,us-central1-a/instances/testinstance2. All instances in one target pool must belong to the same region as the target pool. Instances do not need to exist at the time the target pool is created and can be created afterwards.

If you try to specify duplicate instance names, such as us-central1-a/instances/foo,us-central1-b/instances/bar,us-central1-a/instances/foo, Google Compute Engine ignores the duplicate entries and accepts us-central1-a/instances/foo, us-central1-b/instances/bar.

To create a target pool in the API, make a HTTP POST request to the following URI:

https://www.googleapis.com/v1/compute/projects/<project-id>/regions/<region>/targetPools

Your request body must contain the required properties described above:

bodyContent = {
    "name": name,
    "instances": [
       "http://www.googleapis.com/v1/compute/project/<project-id>/zone/<zone>/instances/<instance-name>",
       "http://www.googleapis.com/v1/compute/project/<project-id>/zone/<zone>/instances/<instance-name>",
    ]
  }

Listing target pools

To list target pools, run gcutil listtargetpools.

gcutil --project=<project-id> listtargetpools

Important flags and parameters:

--project=<project-id>
[Required] The project ID for which you want to list target pools.

In the API, make an empty HTTP GET request to the following URI:

https://www.googleapis.com/compute/v1/projects/<project-id>/regions/<region>/targetPools

Getting a target pool

To get information about a single target pool, use the gcutil gettargetpool command:

gcutil --project=<project-id> gettargetpool --region=<region> <target-pool-name>

Important flags and parameters:

--project=<project-id>
[Required] The project ID for this target pool.
--region=<region>
[Required] The region this target pool is from. For example, --region=us-central1.
<target-pool-name>
[Required] The target pool to get.

In the API, send an empty HTTP GET request to the following URI:

https://www.googleapis.com/compute/v1/projects/<project-id>/regions/<region>/targetPools/<target-pool-name>

Getting the health status of instances

To check the current health status of an instance in your target pool or of all instances in the target pool, you can use the gcutil getargetpoolhealth command. The command returns the health status as determined by the configured health check, either healthy or unhealthy.

To use gcutil:

gcutil --project=<project-id> gettargetpoolhealth <target-pool-name> \
       [--instances=<zone-1>/instances/<instance-1>,<zone-2>/instances/<instance-2>,<zone-n>/instances/<instance-n>] \
       --region=<region>

Important flags and parameters:

--project=<project-id>
[Required] The project ID for this request.
<target-pool>
[Required] The target pool for which you want to query the instance.
--instances=<zone-1>/instances/<instance-1>,<zone-2>/instances/<instance-2>,<zone-n>/instances/<instance-n>
[Optional] Required if your want to get the health status of specific instances within a target pool. The instance or instances to query for must be in the format <zone>/instances/<instance>. For example, us-central1-a/instances/instance1,us-central1-b/instances/instance2.

If you do not define any instances, Google Compute Engine will check the health status of all instances in the target pool.

--region=<region>
[Required] The region for the target pool.

In the API, make a HTTP POST request to the following URI with the instance specified in the request body:

https://www.googleapis.com/compute/v1/projects/<project-id>/regions/<region>/targetPools/<targetPool>/getHealth

The request body should contain the following:

body = {
  "instance": "http://www.googleapis.com/compute/v1/projects/<project-id>/zones/<zone>/instances/<instance-name>"
}

Deleting a target pool

To delete a target pool, you must first make sure that the target pool is not being referenced by any forwarding rules. If a forwarding rule is currently referencing a target pool, you must delete the forwarding rule to remove the reference.

After you remove a target pool from being referenced by any forwarding rules, delete it using the gcutil deletetargetpool command:

gcutil --project=<project-id> deletetargetpool --zone=<zone> [--f] <target-pool-name>

Important flags and parameters:

--project=<project-id>
[Required] The project ID for this target pool.
--region=<region>
[Required] The region to delete this target pool from. For example, --region=us-central1.
--f, --force
[Optional] Bypass the confirmation prompt to delete this target pool.
<target-pool-name>
[Required] The target pool to delete.

In the API, make an empty HTTP DELETE request to the following URI:

https://www.googleapis.com/compute/v1/projects/<project-id>/regions/<region>/targetPools/<target-pool-name>

Updating target pools

If you created you target pool and want to update the target pool specifications, such as adding or removing instances from the instance list, or adding or removing a health check object, you can use the following custom verbs to do so.

Adding and removing an instance from a target pool

You can add or remove instances from an existing target pool using the gcutil addtargetpoolinstance and gcutil removetargetpoolinstance commands.

In the API, you can make a HTTP POST request to the following URIs:

https://www.googleapis.com/compute/v1/projects/<project-id>/regions/<region>/targetPools/<targetPool>/removeInstance
https://www.googleapis.com/compute/v1/projects/<project-d>/regions/<region>/targetPools/<targetPool>/addInstance

The body of your request should include the fully-qualified URIs to the instances that you want to add or remove:

body = {
...
 "instances": [
    {
      "instance": "http://www.googleapis.com/compute/v1/projects/<project-id>/zones/<zone>/instances/<instance-name>"
    },
    {
      "instance": "http://www.googleapis.com/compute/v1/projects/<project-id>/zones/<zone>/instances/<instance-name>"
    },
    ...
  ]
...
}

To use the gcutil addtargetpoolinstance command:

gcutil --project=<project-id> addtargetpoolinstance <target-pool> \
       --instances=<zone-1>/instances/<instance-1>,<zone-2>/instances/<instance-2>,<zone-n>/instances/<instance-n>

Important flags and parameters:

--project=<project-id>
[Required] The project ID for this request.
<target-pool>
[Required] The target pool for which you want to add an instance.
--instance=<zone-1>/instances/<instance-1>,<zone-2>/instances/<instance-2>,<zone-n>/instances/<instance-n>
[Required] The instance to add to this target pool, in the format <zone>/instances/<instance>. For example, us-central1-a/instances/instance1. The instance must be in the same region as the target pool.
--region=<region>
[Required] The region for the target pool.

Note: If you add an instance that already exists in the target pool, nothing will happen, although the operation will return as successful.

To use the gcutil removetargetpoolinstance command:

gcutil --project=<project-id> removetargetpoolinstance <target-pool> \
       --instances=<zone-1>/instances/<instance-1>,<zone-2>/instances/<instance-2>,<zone-n>/instances/<instance-n> \
       --region=<region>

Important flags and parameters:

--project=<project-id>
[Required] The project ID for this request.
<target-pool>
[Required] The target pool for which you want to remove an instance.
--instance=<zone-1>/instances/<instance-1>,<zone-2>/instances/<instance-2>,<zone-n>/instances/<instance-n>
[Required] The instance or instances to remove from this target pool, in the format <zone>/instances/<instance>. For example,us-central1-a/instances/instance1,us-central1-b/instances/instance2.
--region=<region>
[Required] The region for the target pool.

For more information, see the API reference documentation for targetPools:addInstance and targetPools:removeInstance.

Adding and removing a health check from a target pool

Health check objects are standalone, global resources that can be associated or disassociad from any target pool. If you'd like to disassociate or associate a health check to an existing target pool, you can do so using the gcutil addtargetpoolhealthcheck and gcutil removetargetpoolhealthcheck commands. You can also use the API directly.

If you disassociate all health checks from a target pool, Google Compute Engine will treat all instances as healthy and send traffic to all instances in the target pool. However, if you query for the health status of a target pool without a health check, the status will return as unhealthy to indicate that the target pool does not have a health check. We recommend that your target pools should have associated health checks to help you manage your instances.

To associate or disassociate a health check using the API, make a HTTP POST request to the appropriate URIs:

https://www.googleapis.com/compute/v1/projects/<project-id>/regions/<region>/targetPools/<targetPool>/removeHealthCheck
https://www.googleapis.com/compute/v1/projects/<project-d>/regions/<region>/targetPools/<targetPool>/addHealthCheck

The body of your request should contain the health check to associate or disassociate:

body = {
  "healthCheck": "http://www.googleapis.com/compute/v1/projects/<project-id>/global/httpHealthChecks/<httpHealthCheck>"
  }

For more information, see the API reference documentation for targetPools:addHealthCheck and targetPools:removeHealthCheck.

To use the gcutil addtargetpoolhealthcheck command:

gcutil --project=<project-id> addtargetpoolhealthcheck <target-pool> \
       --health_check=<health-check-name> --region=<region>

Important flags and parameters:

--project=<project-id>
[Required] The project ID for this request.
<target-pool>
[Required] The target pool for which you want to add a health check.
--health_check=<health-check-name>
[Required] The health check to add to this target pool.
--region=<region>
[Required] The region for the target pool.

To use the gcutil removetargetpoolhealthcheck command:

gcutil --project=<project-id> removetargetpoolhealthcheck <target-pool> \
       --health_check=<health-check-name> --region=<region>

Important flags and parameters:

--project=<project-id>
[Required] The project ID for this request.
<target-pool>
[Required] The target pool for which you want to remove a health check.
--health_check=<health-check-name>
[Required] The health check to remove from this target pool.
--region=<region>
[Required] The region for the target pool.

Adding and removing a backup target pool

When you first create a target pool, you can choose to apply a backup target pool that receives traffic if your primary target pool becomes unhealthy. If you didn't specify a backup target pool when you created your primary target pool and would like to add one, or if you wanted to remove a backup target pool from an existing target pool, you can do so using gcutil settargetpoolbackup or using the setBackup() method in the API. If you have never set up a backup target pool before, note that you should also set up health checks for the feature to work correctly.

Caution: If your target pool currently has the sessionAffinity field set, resizing the target pool could cause requests that come from the same IP to go to a different instance initially. Eventually, all connections from the IP will go to the same virtual machine, as the old connections close.

To update the a backup pool resource using gcutil:

gcutil --project=<project-id> settargetpoolbackup <primary-target-pool> \
            --backup_pool=<backup-pool> --failover_ratio=<failover-ratio>

Important flags and parameters

--project=<project-id>
[Required] The project ID for this request.
<primary-target-pool>
[Required] The primary target pool for which you want to add or remove a backup pool.
--backup_pool=<backup-pool>
[Required] Sets the backup pool. If you do not set this flag, Google Compute Engine removes the backup pool from your primary target pool. If you set this flag, you must also set the --failover_ratio flag; additionally, your backup target pool must also reside in the same region as your primary target pool.
--failover_ratio=<failover-ratio>
[Optional] Specifies the ratio of the healthy instances in your target pool before it is considered unhealthy. You must declare a float between 0.0 and 1.0. For example, if you set this to .1 (10%), and the percentage of healthy instances in this target pool falls below 10%, Google Compute Engine will start using the backup pool. You must specify this flag if you set the --backup_pool flag. If you do not set this value, Google Compute Engine removes any existing failover ratio for this target pool, disabling the backup target pool.

To make a request to update or remove a backup pool through the API, make a HTTP POST request to the following URI:

https://www.googleapis.com/compute/v1/projects/project/regions/<region>/targetPools/<targetPool>/setBackup?failoverRatio=<failover>

Your resource body must contain the URL to your backup pool:

body = {
  "target": "https://www.googleapis.com/compute/v1/projects/<project-id>/regions/<region>/targetPools/<backup-pool-name>"
}

If you define an empty target, or do not define a failover ratio, the backup pool behavior will be disabled for this target pool.

For more information, see the API reference documentation for targetPools:setBackup.

Health checking

When adding a target pool, you can also specify a health check object to use to determine the health of your instances. Health check objects are global resources that can be associated with any target pools.

To perform health checks, Compute Engine sends health check requests from the IP 169.254.169.254 to each instance within a target pool and the result of the health check is used to determine if the instance receives new connections. Compute Engine will forward new connections only to instances that are marked healthy. If an instance becomes unhealthy, it continues to receive packets for its existing connections until they are terminated or the instance terminates. Unhealthy instances will not receive new connections. This allows instances to shutdown gracefully without abruptly breaking TCP connections. To take advantage of this, your application should fail its health check for a period of time before shutting down or before you remove it from the target pool.

Although health checks are optional, Google Compute Engine recommends health checks for your target pools to protect from failures.

Steps to set up health checks

To set up health checking, you must:

  1. Create a health check object.

    A health check object must first exist before you can use it. For the full details for creating a health check object, see adding a health check. You can quickly add a health check with the default settings by running:

    $ gcutil --project=<project-id> addhttphealthcheck my-new-healthcheck
  2. Add a health check object to a target pool.

    You can add a health check to an existing target pool, or to a new target pool. This example adds the health check object to an existing target pool:

    $ gcutil --project=<project-id> --region=<region> addtargetpoolhealthcheck my-pool-name --health_check=my-new-healthcheck
  3. Allow connections from the Compute Engine health check URL.

    Compute Engine sends health check requests to each instance from the IP 169.254.169.254.You will need to ensure that the correct firewall rules are in place to allow 169.254.169.254 to connect to your instances.

Adding a health check

Before you can associate a health check with a target pool, you must first create an HTTP Health Check resource. Keep in mind that creating a health check resource does not automatically apply the health check resource to a target pool. You must manually associate a health check resource with a target pool before it can perform health checking for you. For more information, see the Adding a Target Pool section, which describes how to associate a health check during the creation of new target pool, or review the Adding or removing a health check from a target pool section to add a health check to an existing target pool.

A health check resource is made up of the following properties:

name
[Required] The name for this health check.
host
[Optional] The value of the host header used in this HTTP health check request. For example, if you are serving webpages from an instance using the domain www.mydomain.com, you may want to set the hostname as www.mydomain.com so that a health check for that host is performed. By default, this is empty and Google Compute Engine automatically sets the host header in health requests to the same external IP address as the forwarding rule associated with this target pool. For example, if a forwarding rule has an external IP address of 1.2.3.4 and is directing traffic to a target pool named tp1 that has a health check object with a default host setting, the host header would be set to 1.2.3.4.
requestPath
[Optional] The request path for this health check. For example, /healthcheck. The default value is /.
port
[Optional] The TCP port number for this health check request. The default value is 80.
checkIntervalSec
How often, in seconds, to perform a health check for an instance. The default value is 5 seconds.
timeoutSec
If Google Compute Engine doesn't receive a HTTP 200 response from the instance by the timeoutSec, the health check request is considered a failure. The default is 5 seconds.
unhealthyThreshold
The number of consecutive health check failures before a healthy instance is marked as unhealthy. The default is 2.
healthyThreshold
The number of consecutive successful health check attempts before an unhealthy instance is marked as healthy. The default is 2.

Note: It is not possible to define different health check parameters for each instance. You can only define health check parameters that apply to all instances in that target pool.

To add a health check object, use the gcutil addhttphealthcheck command or make a HTTP POST request to the health check URI. To use the gcutil addhttphealthcheck command:

gcutil --project=<project-id> addhttphealthcheck <health-check-name> \
            [--check_interval_sec=<interval-in-secs> --check_timeout_sec=<timeout-secs> \
            --description=<description> --healthy_threshold=<healthy-threshold> \
            --unhealthy_threshold=<unhealthy-threshold> --host=<host> \
            --request_path=<path> --port=<port>]

Important flags and parameters:

--project=<project-id>
[Required] The project ID for this request.
<health-check-name>
[Required] Name of the health check.
--check_interval_sec=<interval-in-secs>
[Optional] How often, in seconds, to perform a health check. The default value is 5 seconds.
--check_timeout_sec=<timeout-secs>
[Optional] If Google Compute Engine does not receive a HTTP 200 response from the instance by timeoutSec, the health check request is considered a failure. The default is 5 seconds.
--description=<description>
[Optional] A description for this health check.
--healthy_threshold=<healthy-threshold>
[Optional] The number of consecutive of successful health check attempts before an unhealthy instance is considered healthy. The default is 2.
--unhealthy_threshold=<unhealthy-threshold>
[Optional] The number of consecutive health check failures before a healthy instance is considered unhealthy. The default is 2.
--host=<host>
[Optional] The value of the host header used in this HTTP health check request. For example, if you are serving webpages from an instance using the domain www.mydomain.com, you may want to set the hostname as www.mydomain.com so that a health check for that host is performed. By default, this is empty and Google Compute Engine automatically sets the host header in health requests to the same external IP address as the forwarding rule associated with this target pool. For example, if a forwarding rule has an external IP address of 1.2.3.4 and is directing traffic to a target pool named tp1 that has a health check object with a default host setting, the host header would be set to 1.2.3.4.
--request_path=<path>
[Optional] The request path to use for this health check. By default, this is /.
--port=<port>
[Optional] The port to use for this health check. This can differ from the ports configured on your forwarding rule. By default, this is port 80.

To make a request to the API, make a HTTP POST request to the following URI:

http://www.googleapis.com/compute/v1/projects/<project-id>/global/httpHealthChecks

The body of your request should contain, at a minimum, the following fields:

body = {
  "name" : <health-check-name>,
}

Refer to the httpHealthChecks reference documentation for more information on constructing a http health check API request.

Updating health checks

If you need to update the properties of an existing health check, you can use the gcutil updatehttphealthcheck command or make a HTTP PUT or HTTP PATCH request to the appropriate URI. To use gcutil updatehttphealthcheck, provide all the same flags as if you're adding a health check. The values of your request will replace the existing values of the health check:

gcutil --project=<project-id> updatehttphealthcheck <health-check-name> \
            --check_interval_sec=<interval-in-secs> --check_timeout_sec=<timeout-secs> \
            --description=<description> --healthy_threshold=<healthy-threshold> \
            --unhealthy_threshold=<unhealthy-threshold> --host=<host> \
            --request_path=<path> --port=<port>

See Adding a health check for a detailed description of the above flags.

In the API, you can choose to update your health check using the standard HTTP PUT request or use HTTP PATCH to partially update your health check. Make either a HTTP PUT, or HTTP PATCH request to the following URI:

https://www.googleapis.com/compute/v1/projects/<project>/global/httpHealthChecks/<health-check-name>

Your request body should contain your desired fields and values to update for this health check. See the httpHealthCheck resource representation to see all available fields for this resource.

Listing health checks

To list health checks, perform a gcutil listhttphealthchecks command:

gcutil --project=<project-id> listhttphealthchecks

Important flags and parameters:

--project=<project-id>
[Required] The project ID for this request.

In the API, make a HTTP GET request to the following URI:

http://www.googleapis.com/compute/v1/projects/<project-id>/global/httpHealthChecks

Getting a health check

To get information about a health check, use gcutil gethttphealthcheck:

gcutil --project=<project-id> gethttphealthcheck <health-check-name>

Important parameters and flags:

--project=<project-id>
[Required] The project ID for this request.
<health-check-name>
[Required] The name of the health check to get.

In the API, make a HTTP GET request to the following URI:

http://www.googleapis.com/compute/v1/projects/<project-id>/global/httpHealthChecks/<health-check-name>

Deleting a health check

In order to delete a health check object, you must remove references to the health check from any existing target pools, or delete the target pools that are using the health check object altogether. If you just want to update the health check resource, you can do using the gcutil updatehttphealthchecks command.

To delete a health check object, use gcutil deletehttphealthcheck or make a HTTP DELETE request to the appropriate URI. To use gcutil deletehealtcheck:

gcutil --project=<project-id> deletehttphealthcheck [-f] <health-check-1> <health-check-2> <health-check-n>

Important parameters and flags:

--project=<project-id>
[Required] The project ID for this request.
--f, --force
[Optional] Bypass the confirmation prompt to delete these health checks.
<health-check-n>
[Required] The name of the health check or health checks to delete.

In the API, make an empty HTTP DELETE request to the following URI:

http://www.googleapis.com/compute/v1/projects/<project-id>/global/httpHealthChecks/<health-check-name>

Using an older image

In order to use load balancing, your instances must use an image dated later than 2013-07-18. If you would prefer to use an older image or a custom image, you need to explicitly add the IP address from your forwarding rules to your operating system's virtual interface so it can accept connections that are destined to that load balanced IP. You will need to do this for all instances that should receive load balanced traffic on that IP address. Without this, your operating system's network stack will reject packets destined to that IP since the load balancing service does not translate load-balanced addresses to your internal instances' IP addresses.

For example, if a forwarding rule is load balancing on behalf of the external IP address 1.2.3.4, you would need to add that IP address to all the virtual machine instances that the forwarding rule is routing traffic to. To do that, run the following command on each of your instances:

sudo ip route add to local 1.2.3.4/32 dev eth0 proto 66

You can replace eth0:1 with your desired virtual interface. If you would like to programmatically get the list of external IPs pointing at your virtual machine instance, you can query the metadata server at the following URI:

http://metadata/computeMetadata/v1beta1/instance/network-interfaces/0/forwarded-ips/?recursive=true

Authentication required

You need to be signed in with Google+ to do that.

Signing you in...

Google Developers needs your permission to do that.