Google Cloud Storage

Durable Reduced Availability Storage


Durable Reduced Availability Storage enables you to store data at lower cost, with the tradeoff of lower availability than standard Google Cloud Storage.

Content

  1. Durable Reduced Availability Storage Overview
  2. Creating a Durable Reduced Availability Bucket
  3. Moving Data to a Durable Reduced Availability Bucket

Durable Reduced Availability (DRA) Storage Overview

Durable Reduced Availability storage buckets have lower costs and lower availability, but are designed to have the same durability as standard Google Cloud Storage buckets.

DRA storage is appropriate for applications that are particularly cost-sensitive, or for which some unavailability is acceptable. For example:

  • Data backup - Where high durability is critical, but the highest availability is not required.
  • Batch jobs - Batch jobs can recover from unavailable data, for example by keeping track of the last object that was processed and resuming from that point upon re-starting.

For information about DRA storage pricing, see Pricing and Support.

Google Cloud Storage allows you to enable DRA at the bucket level. As described below, you can specify DRA storage at the time of bucket creation. Once the DRA bucket is created, all bucket and object operations are exactly the same as with the standard storage class. New objects in the bucket are transparently stored with the lower-availability configuration, and your billing for these objects will automatically reflect the lowered prices.

There is currently no support for changing an existing standard bucket into a DRA bucket, or for copying objects from a standard bucket to a DRA bucket or vice-versa. Objects must be downloaded and re-uploaded to the new bucket. Standard data egress/ingress fees apply.

Creating a Durable Reduced Availability Bucket

You enable Durable Reduced Availability Storage at bucket level by issuing a PUT request to Google Cloud Storage. The following examples show how to create a Durable Reduced Availability bucket.

  1. Using gsutil.
    gsutil mb -c DRA gs://<bucketname>/
    
  2. Using curl.
    curl -X PUT --data-binary @dra.txt -H "Authorization: Bearer <auth token>" -H "x-goog-project-id: <project id>" storage.googleapis.com/<bucket name>
    
    Where the dra.txt file contains the following information:
    <CreateBucketConfiguration>
       <LocationConstraint>US</LocationConstraint>
       <StorageClass>DURABLE_REDUCED_AVAILABILITY</StorageClass>
    </CreateBucketConfiguration>
    
  3. Using code. The example uses the boto library that takes care of the PUT request details.
    import logging
    import boto
    
    def createBucketDRA(bucketName, projectId):
       """
       Perform a PUT Bucket operation to create the specified bucket with
       Durable Reduced Availability storage in the specified project.
       """
       try:
          uri = boto.storage_uri(bucketName, "gs")
    
          # Assign the required header values.
          headerValues = {
             'x-goog-api-version': '2',
             'x-goog-project-id': projectId
       	}
    
          # Issue a PUT Bucket request to create a Durable Reduced Availability storage bucket.
          uri.create_bucket(headers=headerValues, location='US', storage_class='DURABLE_REDUCED_AVAILABILITY')
    
       except Exception, e:
          logging.error("createBucketDRA, error occurred: %s", e)
    

To verify that Durable Reduced Availability Storage is enabled you can execute the following command:

gsutil ls -L -b gs://<bucketname>/

Moving Data to a Durable Reduced Availability Bucket

If you want to move data from a Standard Storage to a Durable Reduced Availability Storage bucket, you need to download the data from the Standard bucket to your computer and then upload it to the Durable Reduced Availability bucket. This can be done using the gsutil "daisy chain" copy mode, as follows:

gsutil cp -D -R gs://<standard_bucket>/* gs://<durable_reduced_availability_bucket>

Note that when you perform a daisy chain copy, any ACLs will be lost. You may want to set a default bucket ACL on the destination bucket before uploading the data. Note also that when you perform a daisy chain copy, you may incur network and operation costs for moving the data because the data are downloaded and re-uploaded.

Back to top

Authentication required

You need to be signed in with Google+ to do that.

Signing you in...

Google Developers needs your permission to do that.