Google Compute Engine

Instances

An instance is a virtual machine hosted on Google's infrastructure. Instances can run the Linux images provided by Google, or run any customized versions of these images. You can also build and run images of other operating systems.

Google Compute Engine also lets you specify the machine properties of your instance, such as the number of CPUs and the amount of RAM, based on the machine type you use.

Instances are a per-zone resource.

Contents

Overview

At the core of Google Compute Engine is the Instance resource. Every instance is a virtual machine that is customizable and manageable by you; there are few restrictions on how you use your instance.

You can perform basic instance configuration and management using either gcutil tool, the Google Developers Console, or the REST API, but to perform any advanced configuration, you must ssh into the instance. By default, all instances support ssh capability for the instance creator, and optionally for other users.

As an instance creator you have full root privileges on any instances you have started. An instance administrator can also add system users using standard Linux commands.

To start an instance using gcutil, call the gcutil addinstance command. This starts the process of reserving the instance, starting it, and then running any startup scripts that you specify. You can check the status of an instance by running gcutil --project=<project-id> getinstance <instance-name> and looking for a status of RUNNING. Use gcutil to add instances to a project and start them, specifying a set of properties for this instance such as desired hardware, image type, zone, and optionally any startup scripts that you want to run. Currently adding and removing an instance is the same as starting and stopping an instance; you cannot add an instance to a project without starting it, or remove it without stopping it.

A project holds one or more instances but an instance can be a member of one and only one project. When you start an instance, you must specify which project and zone it should belong to. When you stop an instance, it is removed from the project. Project information can be viewed using the gcutil tool, but you must use the Google Developers Console to create and manage projects.

Instances can communicate with other instances in the same network and with the rest of the world through the Internet. A Network object is restricted to a single project, and cannot communicate with other Network objects. See Networks and Communication for more information about network communication to and from an instance.

Useful gcutil commands:

  • gcutil listinstances
  • gcutil getinstance
  • gcutil addinstance
  • gcutil deleteinstance
$ gcutil --project=my-project getinstance myinstance
+------------------------+--------------------------------------------------------------------------------------------+
|        property        |                                       value                                                |
+------------------------+--------------------------------------------------------------------------------------------+
| name                   | myinstance                                                                                 |
| description            |                                                                                            |
| creation-time          | 2013-01-18T11:15:54.054-08:00                                                              |
| machine                | n1-standard-1                                                                              |
| image                  | projects/debian-cloud/global/images/debian-7-wheezy-vYYYYMMDD                              |
| zone                   | <zone>                                                                                     |
| tags-fingerprint       | 42WmSpB8rSM=                                                                               |
| metadata-fingerprint   | 42WmSpB8rSM=                                                                               |
| status                 | RUNNING                                                                                    |
| status-message         |                                                                                            |
|                        |                                                                                            |
| disk                   | 0                                                                                          |
|   type                 | PERSISTENT                                                                                 |
|   mode                 | READ_WRITE                                                                                 |
|   deviceName           | pd1                                                                                        |
|   source               | http://www.googleapis.com/compute/v1/projects/<project-id>/zones/<zone>/disks/<disk> |
|                        |                                                                                            |
| network-interface      |                                                                                            |
|   network              | default                                                                                    |
|   ip                   | 00.000.000.000                                                                             |
|   access-configuration | External NAT                                                                               |
|     type               | ONE_TO_ONE_NAT                                                                             |
|     external-ip        | 000.000.00.000                                                                             |
|                        |                                                                                            |
| metadata               |                                                                                            |
| fingerprint            | 42WmSpB8rSM=                                                                               |
|                        |                                                                                            |
| tags                   |                                                                                            |
| fingerprint            | 42WmSpB8rSM=                                                                               |
+------------------------+--------------------------------------------------------------------------------------------+

Creating and Starting an Instance

An instance is created and started in a single step. Google Compute Engine does not currently allow you add an instance to a project without starting it. An instance takes a few moments to start up. You must check the instance status to learn when it is actually running. See Checking Instance Status for more information.

Each instance must have a root persistent disk that stores the instances root filesystem. It is possible to create this root persistent disk when you create your instance, or create it separately and attach it to the instance.

Start an Instance Using gcutil

To start an instance using gcutil, run the following command.

gcutil --project=<project-id> addinstance <instance-name> <other-flags>

Important Flags and Parameters:

--project=<project-id>
[Required] The ID of the project in which to add the instance. This flag is required for every gcutil command except help, unless you have previously specified the --cache_flag_values flag to store your project ID information.
instance-name
[Required] The name to assign to the instance. You will use this name to ssh into the instance, or attach resources to it. The name must be 1-63 characters long and match the regular expression [a-z]([-a-z0-9]*[a-z0-9])? which means the first character must be a lowercase letter, and all following characters must be a dash, lowercase letter, or digit, except the last character, which cannot be a dash.
--external_ip_address=<external-ip>
[Optional] Specifies which externally visible IP address, if any, to assign to this instance. See details on supported flag values. If you omit this flag, an external ephemeral IP address will be assigned to the instance.
--machine_type=<machine-type>
[Optional] What machine type to host the instance. Call gcutil listmachinetypes for a list and description of available host hardware, or gcutil getmachinetype for details about a specific hardware configuration. If you do not specify a machine type, gcutil prompts you to select one from a list.
--image=<fully-qualified-image-name>
[Optional] The name of the image to install, from the project's images collection. Call gcutil listimages for a list of available images, or gcutil getimage for a complete description of a particular image. If you do not specify an image, gcutil prompts you to select one from a list.
--metadata=startup-script-url and --metadata_from_file=startup-script
[Optional] These flags are used to specify any startup scripts that you want associated with this instance. If the server is restarted due to a failure, these scripts will be rerun. See Specifying a Startup Script for more information.
--disk=<disk-name>[,deviceName=<alias-name>,mode=<mode>,boot]
[Optional] A persistent disk to associate with this instance. If you do not provide this flag, a root persistent disk will be created for your instance, with the same name as the instance (e.g. if an instance is named newinstance, the associated root persistent disk created in this manner will also be named newinstance. Non-root persistent disks must be created and attached to an instance separately. Alternatively, you can also attach a persistent disk to a running instance. You can choose to attach a disk in read-only or read-write mode; by default, disks are attached in read-write mode. For details about creating a persistent disk resource for an Instance, see Creating and Using Disks.
--[no]auto_delete_boot_disk
[Optional] Determines if the root persistent disk for this instance should be deleted automatically when the instance is deleted. The default is false.
--service_account_scope=<scopes>
[Optional] Set up this instance to use a service account to access other Google services.
Other flags
To see all the flags that you can set with addinstance, run the gcutil help addinstance command.

Tip: To save your flag values for quick reuse, use the --cache_flag_values=True and optional --cached_flags_file=<somefile> flags in your startup call. This lets you reuse frequently used flags.

The following example shows starting a new Instance named vm1.

$ gcutil --project=my-project addinstance vm1 --auto_delete_boot_disk
... select a zone, machine type, and image...
INFO: Waiting for insert of myinstance. Sleeping for 3s.
INFO: Waiting for insert of myinstance. Sleeping for 3s.

Table of resources:

+--------------+---------------+---------------------------------------------------------------+---------+----------------+--------------+----------------+-------------+------------------+----------------------+---------+----------------+
|     name     |  machine-type |                            image                              | network |   network-ip   | external-ip  |     disks      |    zone     | tags-fingerprint | metadata-fingerprint | status  | status-message |
+--------------+---------------+---------------------------------------------------------------+---------+----------------+--------------+----------------+-------------+------------------+----------------------+---------+----------------+
|     vm1      | n1-standard-1 | projects/debian-cloud/global/images/debian-7-wheezy-vYYYYMMDD | default | 00.000.000.000 | 000.000.0.00 |                |    <zone>   | 42WmSpB8rSM=     | 42WmSpB8rSM=         | RUNNING |                |
+--------------+---------------+---------------------------------------------------------------+---------+----------------+--------------+----------------+-------------+------------------+----------------------+---------+----------------+

Table of operations:

+------------------------------------------------+--------+----------------+----------------------------------+----------------+-------+---------+
|                      name                      | status | status-message |              target              | operation-type | error | warning |
+------------------------------------------------+--------+----------------+----------------------------------+----------------+-------+---------+
| operation-1358529117225-4d3933570d191-0fc3d1da | DONE   |                |     <zone>/instances/vm1         |     insert     |       |         |
+------------------------------------------------+--------+----------------+----------------------------------+----------------+-------+---------+

Starting an Instance in the API

To start an instance in the API, construct a request with a source image:

  body = {
    'name': NEW_INSTANCE_NAME,
    'machineType': <fully-qualified-machine-type-url>,
    'networkInterfaces': [{
      'accessConfigs': [{
        'type': 'ONE_TO_ONE_NAT',
        'name': 'External NAT'
       }],
      'network': <fully-qualified-network-url>
    }],
    'disk': [{
       'autoDelete': 'true',
       'boot': 'true',
       'type': 'PERSISTENT',
       'initializeParams': {
          'diskName': 'my-root-disk',
          'sourceImage': '<fully-qualified-image-url>',
       }
     }]
  }

By providing the following in your request:

'disk': [{
  'autoDelete': 'true',
  'boot': 'true',
  'type': 'PERSISTENT',
  'initializeParams': {
    'diskName': 'my-root-disk',
    'sourceImage': '<fully-qualified-image-url>',
  }
}]

Compute Engine also creates a root persistent disk with the source image you specify. You can only provide initalizeParams for a root persistent disk, and can only provide it once per instance creation request. Note that the autoDelete flag also indicates to Compute Engine that the root persistent disk should be automatically deleted when the instance is deleted.

When you are using the API to specify a root persistent disk:

  • You can only specify the boot field on one disk. You may attach multiple persistent disks but only one can be the root persistent disk.
  • You must attach the root persistent disk as the first disk for that instance.
  • When the source field is specified, you cannot specify the initializeParams field, as this conflicts with each other. Providing a source indicates that the root persistent disk exists already, whereas specifying initializeParams indicates that Compute Engine should create the root persistent disk.

If you're using the API client library, you can start a new instance using the instances().insert function. Here is a snippet from the Python client library:

def addInstance(auth_http, gce_service):
  # Construct the request body
   body = {
    'name': NEW_INSTANCE_NAME,
    'machineType': <fully-qualified-machine_type_url>,
    'networkInterfaces': [{
      'accessConfigs': [{
        'type': 'ONE_TO_ONE_NAT',
        'name': 'External NAT'
       }],
      'network': <fully-qualified-network_url>
    }],
    'disk': [{
       'autoDelete': 'true',
       'boot': 'true',
       'type': 'PERSISTENT',
       'initializeParams' :{
          'diskName': 'my-root-disk',
          'sourceImage': '<fully-qualified-image-url>',
       }
    }]
  }

  # Create the instance
  request = gce_service.instances().insert(
       project=PROJECT_ID, body=body, zone=DEFAULT_ZONE)
  response = request.execute(auth_http)
  response = _blocking_call(gce_service, auth_http, response)

  print response

You could also make a request to the API directly by sending a POST request to the instances URI with the same request body:

def addInstance(http, listOfHeaders):
  url = 'https://www.googleapis.com/compute/v1/project/<project-id>/zones/<zone>/instances'

  body = {
    'name': NEW_INSTANCE_NAME,
    'machineType': <fully-qualified-machine_type_url>,
    'networkInterfaces': [{
      'accessConfigs': [{
        'type': 'ONE_TO_ONE_NAT',
        'name': 'External NAT'
       }],
      'network': <fully-qualified-network_url>
    }],
    'disk': [{
       'autoDelete': 'true,
       'boot': 'true',
       'type': 'PERSISTENT',
       'initializeParams': {
          'diskName': 'my-root-disk',
          'sourceImage': '<fully-qualified-image-url>',
       }
     }]

  bodyContentURLEncoded = urllib.urlencode(bodyContent)
  resp,content = http.request(uri=url, method="POST", body=dumps(bodyContent), headers=listOfHeaders)
  print resp
  print content

Checking Instance Status

When you first create an instance, you should check the instance status to see if it is running before you can expect it to respond to requests. It can take a couple seconds before your instance is fully up and running after the initial addinstance request. You can also check the status of an instance at anytime after instance creation.

To check the status of an instance, call gcutil listinstances or gcutil getinstance <instance-name>.

The following states are returned:

  • PROVISIONING - Resources are being reserved for the instance. The instance isn't running yet.
  • STAGING - Resources have been acquired and the instance is being prepared for launch.
  • RUNNING - The instance is booting up or running. You should be able to ssh into the instance soon, though not immediately, after it enters this state.
  • STOPPING - The instance is being stopped either due to a failure, or the instance being shut down. This is a temporary status and the instance will move to either PROVISIONING or TERMINATED.
  • TERMINATED - The instance either failed for some reason or was shutdown. This is a permanent status, and the only way to repair the instance is to delete and recreate it.

Example

$ gcutil --project=my-project getinstance myinstance
+------------------------+--------------------------------------------------------------------------------------------+
|        property        |                                       value                                                |
+------------------------+--------------------------------------------------------------------------------------------+
| name                   | myinstance                                                                                 |
| description            |                                                                                            |
| creation-time          | 2013-01-18T11:15:54.054-08:00                                                              |
| machine                | n1-standard-1                                                                              |
| image                  | projects/debian-cloud/global/images/debian-7-wheezy-vYYYYMMDD                              |
| zone                   | <zone>                                                                                     |
| tags-fingerprint       | 42WmSpB8rSM=                                                                               |
| metadata-fingerprint   | 42WmSpB8rSM=                                                                               |
| status                 | RUNNING                                                                                    |
| status-message         |                                                                                            |
|                        |                                                                                            |
| disk                   | 0                                                                                          |
|   type                 | PERSISTENT                                                                                 |
|   mode                 | READ_WRITE                                                                                 |
|   deviceName           | pd1                                                                                        |
|   source               | http://www.googleapis.com/compute/v1/projects/<project-id>/zones/<zone>/disks/<disk> |
|                        |                                                                                            |
| network-interface      |                                                                                            |
|   network              | default                                                                                    |
|   ip                   | 00.000.000.000                                                                             |
|   access-configuration | External NAT                                                                               |
|     type               | ONE_TO_ONE_NAT                                                                             |
|     external-ip        | 000.000.00.000                                                                             |
|                        |                                                                                            |
| metadata               |                                                                                            |
| fingerprint            | 42WmSpB8rSM=                                                                               |
|                        |                                                                                            |
| tags                   |                                                                                            |
| fingerprint            | 42WmSpB8rSM=                                                                               |
+------------------------+--------------------------------------------------------------------------------------------+

Connecting to an Instance Using ssh

By default, you can always connect to an instance using ssh. This is useful so you can manage and configure your instances beyond the basic configuration enabled by gcutil or the REST API. The easiest way to ssh into an instance is to use gcutil --project=<project-id< ssh <instance-name> from your local computer. For example, the following command ssh'es into an instance named myinst:

$ gcutil --project=my-project ssh myinst

You can use ssh directly without using the gcutil wrapper, if you want, as described in "Using standard ssh", although this is usually less convenient.

Setting Up Your ssh Keys

Before you can access your instance using ssh, you need to set up your ssh keys. Every time you run gcutil addinstance or gcutil ssh, gcutil checks for key files in the location listed below. If you already set up ssh keys when you added another instance in the same project, you won't need to set up keys again. If there are no existing key files, gcutil will:

  1. Prompt you for a passphrase to encrypt your new keys

    As part of the key generation process, gcutil asks for a passphrase to encrypt the keys:

    WARNING: You don't have a public ssh key for Google Compute Engine. Creating one now...
    Enter passphrase (empty for no passphrase):
    Enter same passphrase again:
    

    You are not required to provide a passphrase but if you don't, the keys on your local machine will be unencrypted. We recommend providing a passphrase to encrypt your keys.

  2. Generate your keys using ssh-keygen

    gcutil creates local files to store your public and private key, and copies your public key to the project. By default, gcutil stores ssh keys in the following files on your local computer:

    • $HOME/.ssh/google_compute_engine - Your private key
    • $HOME/.ssh/google_compute_engine.pub - Your public key

    Once gcutil copies your public key to the project, the new key will be added to the VM shortly thereafter. If you want to use existing keys that are stored in a different location, specify the files using the --private_key_file and --public_key_file flags. Note that your instance must have these additional keys installed.

    Note: You need editor permissions to the containing project in order set public keys for that project. For more information, review the documentation for user access permissions.

You can also install multiple public keys during instance creation by calling:

gcutil addinstance --authorized_ssh_keys=username1:/path/to/keyfile1,username2:/path/to/keyfile2,...

If you want to call the API directly, you can specify multiple keys using the metadata field, with each user separated by a newline character:

"metadata": {
      "kind": "compute#metadata",
      "items": [
        {
          "key": "sshKeys",
          "value": "user1:ssh-rsa 123456787..\nuser2:ssh-rsa abcdef0123..\nuser3:ssh-rsa 456789123"
        },...

Being able to add multiple keys is useful for adding multiple users to an instance at startup time, but it also limits the set of ssh keys to be exactly those you specified.

That's it! You have set up ssh access to your instances. Note that this process sets up ssh keys for your local computer, but if you want to ssh in from more than one computer, you should read ssh'ing from different clients.

ssh'ing from different clients

If you are not using the gcutil ssh command to connect to your instance or if you are connecting to the instance from a different computer, then you must copy your local public/private keypair to the machine from which you want to ssh into that instance. If you forget your passphrase, or are working on a machine that does not have a copy of your keys, you will not be able to ssh into the instance.

ssh'ing Using standard ssh

You can use standard ssh rather than gcutil ssh with the following syntax:

ssh -o UserKnownHostsFile=/dev/null -o CheckHostIP=no -o  StrictHostKeyChecking=no \
 -i <key-file> -o LogLevel=QUIET -A  -p 22 <username>@<ipaddress>

Important Flags and Parameters:

key-file
[Required] The file where the keys are stored on the computer e.g. ~/.ssh/google_compute_engine
username
[Required] The username to log in that instance. Typically, this is the username of the local user running gcutil.
ipaddress
[Required] The external IP address of the instance.

ssh'ing From One Instance to Another

If your instance doesn't have an externally-visible IP address, you can still ssh into it by ssh'ing into an instance on the network with an external address, then from there ssh'ing into the internal-only instance from your externally-visible instance. You might need to do this if you've started an instance without an external IP address either intentionally or by accident.

To ssh from one instance to another:

  1. Start ssh-agent using the following command to manage your keys for you:

    eval `ssh-agent`

  2. Call ssh-add to load the gcutil keys from your local computer into the agent, and use them for all ssh commands for authentication:

    ssh-add ~/.ssh/google_compute_engine

  3. Log into an instance with an external IP address:

    gcutil --project=<project-id> ssh <instance-name>

  4. From this externally-addressable instance, you can now log into any other instance on the same network by calling ssh <instance-name>
  5. When you are done, call exit repeatedly to log out of each instance in turn.
  6. You can continue to simply ssh into your internal instances through your external instance until you close your command window, which will close the ssh-agent context.

Example

me@local:~$ eval `ssh-agent`
Agent pid 17666
me@local:~$ ssh-add ~/.ssh/google_compute_engine
Identity added: /home/user/.ssh/google_compute_engine (/home/user/.ssh/google_compute_engine)
me@local:~$ gcutil --project=my-project ssh myinst
INFO: Running command line: ssh -o UserKnownHostsFile=/dev/null -o CheckHostIP=no -o StrictHostKeyChecking=no -i /home/user/.ssh/google_compute_engine -o LogLevel=QUIET -A -p 22 user@255.255.122.238 --
Linux myinst 2.6.39-gcg-yyyymmdd1612 #18 SMP .... x86_64 GNU/Linux
....
me@myinst:~$ ssh myinst2
....
1 package can be updated.
0 updates are security updates.
me@myinst2:~$ exit
logout
Connection to myinst2 closed.
me@myinst:~$

Root Access and Instance Administrators

For security reasons, the standard Google images do not provide the ability to ssh in directly as root. The instance creator and any users that were added using the --authorized_ssh_keys flag or the metadata sshKeys value are automatically administrators to the account, with the ability to run sudo without requiring a password.

Although it is not recommended, advanced users can modify /etc/ssh/sshd_config and restart sshd to change this policy.

Setting Instance Scheduling Options

By default, Google Compute Engine automatically manages the scheduling decisions for your instances. For example, if your instance is terminated due to a system or hardware failure, Compute Engine automatically restarts that instance. Instance scheduling options let you change this automatic behavior.

Maintenance Behavior

Periodically, Google performs scheduled maintenance on the infrastructure underlying your running instance. Compute Engine automatically moves your instance away from the underlying infrastructure before it is taken offline for the scheduled maintenance. For more information, see Scheduled Maintenance.

Compute Engine has two ways to move your instances during maintenance events:

  • Live migrate

    Note: This option is only available for instances in US and Asia zones.

    If you choose this option, Google Compute Engine will automatically migrate your instance away from the maintenance event, and your instance remains running during the migration. Your instance may experience a short period of decreased performance, although generally most instances should not notice any difference. This option is ideal for instances that require constant uptime, and can tolerate a short period of decreased performance.

    When Google Compute Engine migrates your instance, it reports a system event that is published to the your list of zone operations. You can review this event by performing gcutil listzoneoperations event or by viewing the list of operations in the Google Developers Console, or through an API request. The event will appear with the following text:

    compute.instances.migrateOnHostMaintenance
  • Terminate and Restart

    If you choose this option, Google Compute Engine will signal your instance to shut down, wait for a short period of time for your instance to shut down cleanly, terminate the instance, and restart it away from the maintenance event. This option is ideal for instances that demand constant, maximum performance, and your overall application is built to handle instance failures or reboots.

    When Google Compute Engine terminates and reboots your instances, it reports a system event that is published to the list of zone operations. You can review this event by performing gcutil listzoneoperations event or by viewing the list of operations in the Google Developers Console, or through an API request. The event will appear with the following text:

    compute.instances.terminateOnHostMaintenance

    You should also be aware that if you select this option, your instances may experience more frequent terminate-and-reboot events, even outside the normal two week maintenance windows, and plan accordingly. When your instance reboots, it will use the same persistent boot disk as before.

Persistent disks are preserved in both migrate or terminate cases. For the terminate and reboot case, your persistent disk will be briefly detached from the instance while it is being rebooted, and then reattached once the instance is restarted.

See How to Set Scheduling Options below for the default maintenance behavior values and also how to change this setting on existing instances.

Automatic Restart

You can set up Google Compute Engine to automatically restart an instance if it is taken offline by a system event, such as a hardware failure or scheduled maintenance event, using the automaticRestart setting. This setting does not apply if the instance is taken offline through a user action, such as calling sudo shutdown.

When Google Compute Engine automatically restarts your instance, it reports a system event that is published to the list of zone operations. You can review this event by performing gcutil listzoneoperations event or by viewing the list of operations in the Google Developers Console, or through an API request. The event will appear with the following text:

compute.instances.automaticRestart 

How to Set Scheduling Options

All instances are configured with default values for onHostMaintenance and automaticRestart settings. The the default setting for instances is to set the onHostMaintenance flag to migrate, in which case Google Compute Engine will migrate the instance around scheduled maintenance events.

If you want to manually set scheduling options of an instance, you can do so when first creating the instance or after the instance is created, using the setScheduling method.

Specifying scheduling options during instance creation

To specify the maintenance behavior and automatic restart settings of a new instance in gcutil, use the --on_host_maintenance and --automatic_restart flags:

gcutil --project=<project-id> addinstance <instance-name> ... [--on_host_maintenance=<behavior>] [--automatic_restart=<restart>]

Important flags and parameters:

<project-id>
[Required] The project ID for this request.
<instance-name>
[Required] The name of the instance for which you would like to update the scheduling options.
--on_host_maintenance=<behavior>
[Optional] Sets the new maintenance behavior for this instance. Valid values are migrate or terminate. migrate indicates that Google Compute Engine should migrate this instance away from scheduled maintenance windows. terminate means that Google Compute Engine should terminate and reboot this instance in response to scheduled maintenance windows. To makes sure Google Compute Engine automatically restarts the instance, set the --automatic_restart flag. Note that instances may be restarted multiple times and could be restarted outside of scheduled maintenance windows.
--automatic_restart=<restart>
[Optional] Indicates if Google Compute Engine should restart an instance if the instance is taken offline due to any system event (such as a maintenance event).

This is a boolean parameter, with either true or false values. This feature won't apply for situations where an instance is taken offline by a user (such as someone running sudo shutdown).

In the API, make a POST request to:

https://www.googleapis.com/compute/v1/projects/<project-id>/global/instances

with the onHostMaintenance and automaticRestart parameters as part of the request body:

{
  "kind": "compute#instance",
  "name": "vm1",
  "description": "Front-end for real-time ingest; don't migrate.",
...
  // User options for influencing this Instance’s life cycle.
  "scheduling": {
    "onHostMaintenance": "migrate",
    "automaticRestart": "true" # specifies that Google Compute Engine should automatically restart your instance
  }
}

For more information, see the instances reference documentation.

Updating scheduling options for an existing instance

To update the scheduling options of an instance, use the gcutil setscheduling command with the same parameters and flags used in the instance creation command above:

gcutil --project=<project-id> setscheduling <instance-name> [--on_host_maintenance=<behavior>] [--automatic_restart=<restart>]

In the API, you can make a request to the following URL:

https://www.googleapis.com/compute/v1/projects/<project-id>/global/instances/setScheduling

The body of your request must contain the new value for the scheduling options:

{
  "onHostMaintenance": "migrate"
  "automaticRestart": "true" # specifies that Google Compute Engine should automatically restart your instance
}

For more information, see the instances : setScheduling reference documentation.

Installing packages and Configuring an Instance

The instance creator has administrator privileges on any instance she adds to a project, and is automatically on the SUDO list.

When you are logged into an instance as the administrator, you can install packages and configure the instance the same way you would a normal Linux box. For example, you can install Apache, as shown here:

rufus@myinst:~$ sudo apt-get update
rufus@myinst:~$ sudo apt-get install apache2
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
[...]

You can move files between your local computer and instance using gcutil push and gcutil pull as described in Copying Files To/From an Instance.

Note that your machine needs access to the Internet to be able to run apt-get. This means that it needs either an external IP address, or access to an Internet proxy.

Scheduled maintenance notice

It is possible to detect when a maintenance event is about to happen through an instance's virtual machine metadata server. A special metadata attribute, maintenance-event, will update its value shortly before the started and again at the end of a maintenance event, allowing you to detect when a scheduled maintenance event is going to happen and when it ends. You can use this information to help automate any scripts or commands you may want to run at those times.

For more information, see the Scheduled maintenance notice section on the Metadata server documentation.

Copying Files To/From an Instance

Use gcutil push to send files to an instance from your local machine, and gcutil pull to copy files from your instance to your local machine. Note that when calling push, the target directory must exist on the instance.

Note: An instance must have an external IP address to be able to push or pull files to/from it.

gcutil --project=<project-id> push <instance-name> <local-file> <remote-target-path>
gcutil --project=<project-id> pull <instance-name> <file1> <file2> ... <local-directory>

Example

# Copy local file named readme.txt up to my instance named myinst
$ gcutil --project=my-project push myinst readme.txt /home/user/.
$ gcutil --project=my-project ssh myinst

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.

user@myinst$ ls /home/user
readme.txt

Detecting if you are running in Google Compute Engine

It is common for systems to want to detect if they are running within a specific cloud environment. To enable this, you can query for the metadata server for a specific header that indicates you are running within Compute Engine. For more information, see Detecting if you are running in Compute Engine.

For images v20131120 and newer, you can request more explicit confirmation using the dmidecode tool. The dmidecode tool can be used to access the DMI/SMBOIS information in /proc/mem directly. Run the following command and the dmidecode tool should print out "Google" to indicate that you are running in Google Compute Engine:

my@myinst:~$ sudo dmidecode -s bios-vendor | grep Google
Google

Enabling network traffic

By default, all new instances have the following connections enabled:

  • Traffic between instances in the same network, over any port and any protocol.
  • Incoming ssh connections (port 22) from anywhere.

Any other incoming traffic to an instance is blocked. You must explicitly assign new firewall rules a network to enable other connections. See Connecting to an instance using ssh to learn how to ssh into your instance, or Networks and Firewalls to learn how instances communicate with each other over IP, and how to set up an externally accessible HTTP connection to an instance.

Using instance tags

You can assign tags to your instances to help coordinate or group instances that may share common traits or attributes. For example, if there are several instances that perform the same task, such as serving a large website, you may consider tagging these instances with a shared word or term. Instance tags are also used by networks and firewalls to identify which instances firewall rules may apply to. Tags are also reflected in the metadata server, so you can use them for applications running on your instances.

To assign tags to a running instance using gcutil, use the gcutil setinstancetags command:

gcutil --project=<project-id> setinstancetags <instance-name> --tags=<tag-1,tag-2,..,tag-n> --fingerprint=<current-fingerprint-hash>

Important Flags and Parameters:

--project=<project-id>
[Required] The project ID of the instance.
<instance-name>
[Required] The name of the instance to set tags.
--tags=<tag-1,tag-2,tag-3>
[Required] A list of tags to apply to the instance. Updating tags is done in a batch request, so you must update the entire list of tags, even if you are only modifying a single tag. For example, if you have a list of tags with values mustard,ketchup,romaine, and you want to remove ketchup, you must respecify the entire tag list, without the ketchup tag:
---tags=mustard,romaine
--fingerprint=<current-fingerprint-hash>
[Required] The current fingerprint hash of the tag. You can grab the fingerprint hash by performing a gcutil getinstance <instance-name> command, and copying the value of the tags-fingerprint field. The fingerprint you supply must match the current fingerprint on the instance. This performs optimistic locking, so that only one user may update the tag list at any one time.

The following example demonstrates how to update the instance tags for an instance:

  1. Get information about this instance and note the metadata fingerprint field:
    $ gcutil --project=myproject getinstance myinstance
    +------------------------+--------------------------------------------------------------------------------------------+
    |        property        |                                       value                                                |
    +------------------------+--------------------------------------------------------------------------------------------+
    | name                   | myinstance                                                                                 |
    | description            |                                                                                            |
    | creation-time          | 2013-01-18T11:15:54.054-08:00                                                              |
    | machine                | n1-standard-1                                                                              |
    | image                  | projects/debian-cloud/global/images/debian-7-wheezy-vYYYYMMDD                              |
    | zone                   | <zone>                                                                                    |
    | tags-fingerprint       | 42WmSpB8rSM=                                                                               |
    | metadata-fingerprint   | 42WmSpB8rSM=                                                                               |
    | status                 | RUNNING                                                                                    |
    | status-message         |                                                                                            |
    |                        |                                                                                            |
    | disk                   | 0                                                                                          |
    |   type                 | PERSISTENT                                                                                 |
    |   mode                 | READ_WRITE                                                                                 |
    |   deviceName           | pd1                                                                                        |
    |   source               | http://www.googleapis.com/compute/v1/projects/<project-id>/zones/<zone>/disks/<disk> |
    |                        |                                                                                            |
    | network-interface      |                                                                                            |
    |   network              | default                                                                                    |
    |   ip                   | 00.000.000.000                                                                             |
    |   access-configuration | External NAT                                                                               |
    |     type               | ONE_TO_ONE_NAT                                                                             |
    |     external-ip        | 000.000.00.000                                                                             |
    |                        |                                                                                            |
    | metadata               |                                                                                            |
    | fingerprint            | 42WmSpB8rSM=                                                                               |
    |                        |                                                                                            |
    | tags                   |                                                                                            |
    | fingerprint            | kFyURcqFcPg=                                                                               |
    |                        | cheese                                                                                     |
    |                        | mustard                                                                                    |
    |                        | romaine                                                                                    |
    +------------------------+--------------------------------------------------------------------------------------------+
    
  2. Next, use that fingerprint to update the metadata entries:
    $ gcutil --project=myproject setinstancetags --tags=mustard,romaine --fingerprint=kFyURcqFcPg= myinstance
    INFO: Waiting for setTags of instance myinstance. Sleeping for 3s
    ....
  3. Run gcutil getinstance again to see your new metadata entries:
    $ gcutil --project=myproject getinstance myinstance
    +------------------------+--------------------------------------------------------------------------------------------+
    |        property        |                                       value                                                |
    +------------------------+--------------------------------------------------------------------------------------------+
    | name                   | myinstance                                                                                 |
    | description            |                                                                                            |
    | creation-time          | 2013-01-18T11:15:54.054-08:00                                                              |
    | machine                | n1-standard-1                                                                              |
    | image                  | projects/debian-cloud/global/images/debian-7-wheezy-vYYYYMMDD                              |
    | zone                   | <zone>                                                                                     |
    | tags-fingerprint       | 42WmSpB8rSM=                                                                               |
    | metadata-fingerprint   | 42WmSpB8rSM=                                                                               |
    | status                 | RUNNING                                                                                    |
    | status-message         |                                                                                            |
    |                        |                                                                                            |
    | disk                   | 0                                                                                          |
    |   type                 | PERSISTENT                                                                                 |
    |   mode                 | READ_WRITE                                                                                 |
    |   deviceName           | pd1                                                                                        |
    |   source               | http://www.googleapis.com/compute/v1/projects/<project-id>/zones/<zone>/disks/<disk> |
    |                        |                                                                                            |
    | network-interface      |                                                                                            |
    |   network              | default                                                                                    |
    |   ip                   | 00.000.000.000                                                                             |
    |   access-configuration | External NAT                                                                               |
    |     type               | ONE_TO_ONE_NAT                                                                             |
    |     external-ip        | 000.000.00.000                                                                             |
    |                        |                                                                                            |
    | metadata               |                                                                                            |
    | fingerprint            | 42WmSpB8rSM=                                                                               |
    |                        |                                                                                            |
    |                        |                                                                                            |
    | tags                   |                                                                                            |
    | fingerprint            | wHsRKmfQcPg=                                                                               |
    |                        | mustard                                                                                    |
    |                        | romaine                                                                                    |
    +------------------------+--------------------------------------------------------------------------------------------+

Resetting an instance

You can perform a hard reset on an instance by using the gcutil resetinstance command or by making a POST request to the following URI:

https://www.googleapis.com/compute/v1/projects/<project-id>/zones/<zone>/instances/<instance-name>/reset

Performing a reset on your instance is similar to pressing the reset button on your computer. Note that your instance remains in RUNNING mode through the reset.

To reset your instance using gcutil:

gcutil --project=<project-id> resetinstance <instance-name>

Important Flags and Parameters:

--project=<project-id>
[Required] The project ID for this request.
<instance-name>
[Required] The name of the instance to reset.

To reset your instance using the client libraries, construct a request to the instances().reset method:

def resetInstance(auth_http, gce_service):
  request = gce_service.instances().reset(project=PROJECT_ID, zone=ZONE_NAME, instance=INSTANCE_NAME)
  response = request.execute(auth_http)

  print response

For more information on this method, see the instances().reset reference documentation.

Shutting down an instance

When you shut down an instance, the following things will happen:

  • The instance will disappear but persistent disk data will be retained until the persistent disk is explicitly deleted.
  • If you perform gcutil deleteinstance, and your instance still has resources attached to it (such as a persistent disk, or an ephemeral or static IP address, the resource will be released to be used by another instance. Additionally, if you have a root persistent disk attached to the instance, gcutil prompts you to decide if you want to keep the persistent disk or delete it with the instance.
  • In contrast, if you run sudo poweroff, Google Compute Engine keeps these resources attached to the instance, even if the instance is marked as TERMINATED. You must manually release these resources or run gcutil deleteinstance to release these resources.

There are two ways to shut down an instance:

  • gcutil deleteinstance (Recommended) - Shuts down the instance and removes it from the list of instances. Requires at least read rights on the project.
    gcutil --project=<project-id> deleteinstance <instance-name-1> [<instance-name-2> <instance-name-3> ...  <instance-name-n>]
  • sudo poweroff - Called when ssh'ed into an instance. Does not require any project rights, but you must still call gcutil deleteinstance at some point afterward, or else the instance will continue to be listed as a member of the project, but will not be accessible.

Restarting an instance

There are two valid ways to restart a currently running instance, manually through gcutil or the instance, or automatically using the automaticRestart flag.

Manually restarting an instance

To manually restart an instance, use one of the following two methods:

  • sudo reboot - Called when ssh'ed into an instance. Wipes the memory and re-initializes the instance with the original metadata, image, and persistent disks. It will not pick up any updated versions of the image, and the instance will retain the same ephemeral IP address.
  • gcutil deleteinstance followed by gcutil addinstance - This is a completely destructive restart, and will initialize the instance with any information passed into gcutil addinstance. You can then select any new images or other resources you'd like to use. The restarted instance will probably have a different IP address. This method potentially swaps the physical machine hosting the instance.

Listing all running instances

You can see a list of all instances in a project by calling gcutil listinstances.

$ gcutil --project=<project-id> listinstances

+------------+---------------+---------------------------------------------------------------+---------+----------------+----------------+--------------------+--------+---------+----------------+
|    name    | machine-type  |                             image                             | network |   network-ip   |  external-ip   |       disks        |  zone  | status  | status-message |
+------------+---------------+---------------------------------------------------------------+---------+----------------+----------------+--------------------+--------+---------+----------------+
| myinstance | n1-standard-1 | projects/debian-cloud/global/images/debian-7-wheezy-vYYYYMMDD | default | 00.000.000.000 | 000.000.00.000 |  <zone>/disks/pd1  | <zone> | RUNNING |                |
+------------+---------------+---------------------------------------------------------------+---------+----------------+----------------+--------------------+--------+---------+----------------+

By default, gcutil provides an aggregate listing of all your resources across all available zones. If you want a list of resources from just a single zone, provide the --zone flag in your request.

$ gcutil --project=<project-id> listinstances --zone=<zone>

Important Flags and Parameters:

--project=<project-id>
[Required] The project ID for this request.
--zone=<zone>
[Required] The zone from which you want to list instances.

In the API, you need to make requests to two different methods to get a list of aggregate resources or a list of resources within a zone. To make a request for an aggregate list, make a GET request to that resource's aggregatedList URI:

https://www.googleapis.com/compute/v1/aggregated/instances

In the client libraries, make a request to the instances().aggregatedList function:

def listAllInstances(auth_http, gce_service):
  request = gce_service.instances().aggregatedList(project=PROJECT_ID)
  response = request.execute(auth_http)

  print response

To make a request for a list of instances within a zone, make a GET request to the following URI:

http://www.googleapis.com/compute/v1/projects/<project-id>/zones/<zone>/instances

In the API client libraries, make a instances().list request:

def listInstances(auth_http, gce_service):
  request = gce_service.instances().list(project=PROJECT_ID,
    zone='<zone>')
  response = request.execute(auth_http)

  print response

Handling instance failures

Unfortunately, individual instances will experience failures from time to time. This can be due to a variety of reasons, including scheduled maintenance, scheduled zone outages (in zones that do not yet support transparent maintenance), unexpected outages, hardware error, or other system failures. As a way to mitigate such situations, you should use persistent disks and back up your data routinely.

If an instance fails, it will be restarted automatically, with the same root persistent disk, metadata, and instance settings as when it failed. To control the automatic restart behavior for an instance, see How to Set Scheduling Options. However, if an instance is terminated for a zone maintenance window it will stay terminated and will not be restarted when the zone exits the maintenance window.

In general, you should design your system to be robust enough that the failure of a single instance should not be catastrophic to your application. For more information, see Designing Robust Systems.

Creating a Custom Image

You can create a custom instance image by customizing a provided image, and then load it onto new instances as they are brought up. See Creating and applying a custom image for details.

Authentication required

You need to be signed in with Google+ to do that.

Signing you in...

Google Developers needs your permission to do that.