Get started with Cloud KMS

Cloud KMS is a cloud-hosted key management service that lets you manage cryptographic keys for your cloud services the same way you do on-premises. It includes support for encryption, decryption, signing, and verification using a variety of key types and sources including Cloud HSM for hardware-backed keys.

In this tutorial, you will learn how to use the advanced features of Cloud Security and Privacy APIs, including:

• Setting up a secure Cloud Storage bucket
• Managing keys and encrypted data using Cloud KMS
• Viewing Cloud Storage audit logs

You will take abridged data from the Enron Corpus, encrypt it, and load it into Cloud Storage.

You will learn

• How to encrypt data and manage encryption keys using Key Management Service (KMS)

You will use

• Cloud KMS
• Cloud Storage
• Cloud SDK

Self-paced environment setup

If you don't already have a Google Account (Gmail or Google Apps), you must create one. Sign-in to Google Cloud Platform console (console.cloud.google.com) and create a new project:

Remember the project ID, a unique name across all Google Cloud projects (the name above has already been taken and will not work for you, sorry!). It will be referred to later in this codelab as PROJECT_ID.

Next, you'll need to enable billing in the Cloud Console in order to use Google Cloud resources.

Running through this codelab shouldn't cost you more than a few dollars, but it could be more if you decide to use more resources or if you leave them running (see "cleanup" section at the end of this document).

New users of Google Cloud Platform are eligible for a $300 free trial.

Start Cloud Shell

In this codelab you will use Cloud Shell, a free virtualized environment running on Google Cloud. From the GCP Console click the Cloud Shell icon on the top right toolbar:

It should only take a few moments to provision and connect to the environment. When it is finished, you should see something like this:

This virtual machine is loaded with all the development tools you'll need. It offers a persistent 5GB home directory, and runs on the Google Cloud, greatly enhancing network performance and authentication. Unless otherwise instructed, run all commands from this shell.

To store data for this tutorial, create a Cloud Storage bucket. Since Cloud Storage bucket names must be globally unique, prefix the bucket name with your project ID:

$ export BUCKET_NAME=${GOOGLE_CLOUD_PROJECT}_enron_emails

Tutorial Variable

This tutorial assumes that the BUCKET_NAME environment variable is set moving forward. If you reset your session or close your terminal, you will need to re-run this command before resuming where you left off.

Then create the bucket:

$ gsutil mb gs://${BUCKET_NAME}

The Enron Corpus is a large database of over 600,000 emails generated by 158 employees of the Enron Corporation. This data has been copied to the Cloud Storage bucket named gs://enron_emails/.

Download one of the email files locally:

$ gsutil cp gs://enron_emails/allen-p/inbox/1. . # <-- don't forget the dot!

Inspect the downloaded file to verify that it is, in fact, an email:

$ tail 1.

The email will contain a message like:

Attached is the Delta position for 1/18...
# ...

In addition to emails, the bucket also contains images. This tutorial uses both the plaintext emails and images.

Before you can use Cloud KMS, you must first enable the service in your project. This only needs to be done once per project. To enable the Cloud KMS service, run the following command:

$ gcloud services enable cloudkms.googleapis.com \
    --project "${GOOGLE_CLOUD_PROJECT}"

It can take up to a minute to enable. The command will report success when it finishes.

Create a Cloud KMS Key Ring. In Cloud KMS, a Key Ring is a logical collection of cryptographic keys. The Key Ring contains metadata about the keys such as their location. Create a Key Ring named my-keyring in the global region:

$ gcloud kms keyrings create "my-keyring" \
    --location "global"

Already Exists

If you have done this tutorial (or other Cloud KMS tutorials) previously, you may see an error that "my-keyring" already exists. You can ignore this error and continue to the next step as you have already created the Key Ring.

Now create a Crypto Key named enron-emails with the purpose encryption inside the Key Ring you just created.

$ gcloud kms keys create "enron-emails" \
    --location "global" \
    --keyring "my-keyring" \
    --purpose "encryption"

Open the Cryptographic Keys Web UI and view the newly created resources. This is available under IAM & Admin in the main menu.

You can view and manage Key Rings and Crypto Keys in the Cloud Console.

Encrypt the contents of the email we downloaded earlier using Cloud KMS. This tutorial uses the gcloud command line tool, but you can also encrypt data using the Cloud KMS API.

$ gcloud kms encrypt \
    --location "global" \
    --keyring "my-keyring" \
    --key "enron-emails" \
    --plaintext-file ./1. \
    --ciphertext-file ./1.enc

This will create a new file on disk 1.enc, which contains the encrypted file contents. Now that the data is encrypted with Cloud KMS, upload the encrypted file to the Cloud Storage bucket.

$ gsutil cp ./1.enc gs://${BUCKET_NAME}

Cloud KMS is integrated with Google Cloud Identity and Access Management (IAM). Cloud KMS IAM roles are largely divided into two categories:

• Permission to manage keys
• Permission to use keys

For example, the roles/cloudkms.admin role allows the holder to create Key Rings, Crypto Keys, set IAM policies, and perform management-related operations. The roles/cloudkms.cryptoKeyEncrypterDecrypter grants the holder the encrypt and decrypt data, but it does not include management permissions.

IAM roles are inherited by their parent resource. If someone has the roles/cloudkms.admin role on the Google Cloud project, they are an admin of all keys in that project. If they have roles/cloudkms.admin on a Key Ring, they are an admin of all keys in that Key Ring. If they have roles/cloudkms.admin on a single Crypto Key, they are only an admin of that key.

For this exercise, use your identity:

$ export MY_IDENTITY=[e-mail] # or [e-mail]

Note

In this tutorial, you are a project owner, which already includes the required permissions to manage keys and to encrypt & decrypt data. The steps below are not necessary, but we do so to demonstrate how to manage permissions. See Cloud KMS Permissions and Roles for more information about the predefined roles.

Assign the identity the ability to manage Cloud KMS resources in the Key Ring created above using the gcloud command line tool:

$ gcloud kms keyrings add-iam-policy-binding "my-keyring" \
    --location "global" \
    --member "user:${MY_IDENTITY}" \
    --role "roles/cloudkms.admin"

The roles/cloudkms.admin role does not include permission to use keys, only to manage them. To grant your identity the ability to encrypt and decrypt data using the enron-emails Crypto Key:

$ gcloud kms keys add-iam-policy-binding "enron-emails" \
    --location "global" \
    --keyring "my-keyring" \
    --member "user:${MY_IDENTITY}" \
    --role "roles/cloudkms.cryptoKeyEncrypterDecrypter"

Note that the first command grants the roles/cloudkms.admin role on the Key Ring, which would include any child Crypto Key resources. The second command grants roles/cloudkms.cryptoKeyEncrypterDecrypter, but only on the enron-emails Crypto Key.

You can also view and assign roles in the Cloud KMS UI in the Cloud Console.

In addition to encrypting a single file, you can also encrypt a collection of files with some scripting. This example streams all emails for allen-p, encrypts them, and uploads the resulting encrypted values to the Cloud Storage bucket:

DIR="gs://enron_emails/allen-p"
for file in $(gsutil ls ${DIR}/**); do
  ENC_NAME="$(basename ${file}).enc"
  gsutil cat ${file} \
  | \

  gcloud kms encrypt \
    --location "global" \
    --keyring "my-keyring" \
    --key "enron-emails" \
    --plaintext-file - \
    --ciphertext-file - \
    | \

    gsutil -q cp - gs://${BUCKET_NAME}/${ENC_NAME}
    echo "Copied ${ENC_NAME}"
done

This iterates over all the files in the given directory in the sample Cloud Storage bucket, encrypts them using Cloud KMS, and uploads them to your Google Cloud Storage.

After the script completes, you can view the encrypted files in the Cloud Storage UI. You should see something like this:

Note

Cloud Storage supports Server Side Encryption with Cloud KMS, which automates this entire process. This tutorial is for demonstration purposes only.

Cloud Audit Logging has two log streams – Admin Activity Logs and Data Access Logs – which are generated by Google Cloud services to help you answer the question of "who did what, where, and when?" within your Google Cloud Platform projects.

To view the activity for any resource in KMS, click the key ring and select View Activity. This will take you to the Cloud Activity UI, where you should see the creation and all modifications made to the KeyRing.

You have successfully encrypted data using Cloud KMS and stored encrypted data in Cloud Storage.

Cleanup

Release the resources created during this tutorial. Delete the Cloud Storage bucket created earlier:

$ gsutil rm -r gs://${BUCKET_NAME}

Cloud KMS resources cannot be deleted. However, you can destroy the key material so it cannot be used again:

$ gcloud kms keys versions destroy "1" \
    --location "global" \
    --key "enron-emails" \
    --keyring "my-keyring"

We covered

• Using Cloud IAM to manage Cloud KMS permissions
• Using Cloud KMS to encrypt data
• Using Cloud Storage to store encrypted data
• Using Cloud Audit Logging to view all activity for keys and key rings

Next steps

• Use Cloud KMS to encrypt a column in a database like BigQuery
• Set an automatic rotation schedule on your Cloud KMS keys

Learn more

• Learn more about how to rotate encryption keys
• Read about envelope encryption
• Read about Server Side Encryption for Cloud Storage buckets
• Post questions and find answers on Stackoverflow under the google-cloud-kms tag

License

This work is licensed under a Creative Commons Attribution 2.0 Generic License.