# Machine Learning Glossary: Google Cloud

This page contains Google Cloud glossary terms. For all glossary terms,
[click here](https://developers.google.com/machine-learning/glossary).


## A

<br />


## accelerator chip

#GoogleCloud

<br />

A category of specialized hardware components designed to perform key
computations needed for deep learning algorithms.

Accelerator chips (or just **accelerators**, for short) can significantly
increase the speed and efficiency of training and inference tasks
compared to a general-purpose CPU. They are ideal for training
neural networks and similar computationally intensive tasks.

Examples of accelerator chips include:

- Google's Tensor Processing Units ([**TPUs**](https://developers.google.com/machine-learning/glossary/googlecloud#TPU)) with dedicated hardware for deep learning.
- NVIDIA's GPUs which, though initially designed for graphics processing, are designed to enable parallel processing, which can significantly increase processing speed.


## B

<br />


## batch inference

#GoogleCloud

<br />

The process of [**inferring**](https://developers.google.com/machine-learning/glossary#inference) predictions on multiple
[**unlabeled examples**](https://developers.google.com/machine-learning/glossary#unlabeled_example) divided into smaller
subsets ("batches").

Batch inference can take advantage of the parallelization features of
[**accelerator chips**](https://developers.google.com/machine-learning/glossary/googlecloud#accelerator-chip). That is, multiple accelerators
can simultaneously infer predictions on different batches of unlabeled
examples, dramatically increasing the number of inferences per second.

See [Production ML systems: Static versus dynamic
inference](https://developers.google.com/machine-learning/crash-course/production-ml-systems/static-vs-dynamic-inference)
in Machine Learning Crash Course for more information.


## C

<br />


## Cloud TPU

#TensorFlow #GoogleCloud

<br />

A specialized hardware accelerator designed to speed up machine
learning workloads on Google Cloud.


## D

<br />


## device

#TensorFlow #GoogleCloud

<br />

An overloaded term with the following two possible definitions:

1. A category of hardware that can run a TensorFlow session, including CPUs, GPUs, and [**TPUs**](https://developers.google.com/machine-learning/glossary/googlecloud#TPU).
2. When training an ML model on [**accelerator chips**](https://developers.google.com/machine-learning/glossary/googlecloud#accelerator-chip) (GPUs or TPUs), the part of the system that actually manipulates [**tensors**](https://developers.google.com/machine-learning/glossary#tensor) and [**embeddings**](https://developers.google.com/machine-learning/glossary#embedding_layer). The device runs on accelerator chips. In contrast, the [**host**](https://developers.google.com/machine-learning/glossary/googlecloud#host) typically runs on a CPU.


## H

<br />


## host

#TensorFlow #GoogleCloud

<br />

When training an ML model on [**accelerator chips**](https://developers.google.com/machine-learning/glossary/googlecloud#accelerator-chip)
(GPUs or [**TPUs**](https://developers.google.com/machine-learning/glossary/googlecloud#TPU)), the part of the system
that controls both of the following:

- The overall flow of the code.
- The extraction and transformation of the input pipeline.

The host typically runs on a CPU, not on an accelerator chip; the
[**device**](https://developers.google.com/machine-learning/glossary/googlecloud#device) manipulates [**tensors**](https://developers.google.com/machine-learning/glossary#tensor) on the
accelerator chips.


## M

<br />


## mesh

#TensorFlow #GoogleCloud

<br />

In ML parallel programming, a term associated with assigning the data and
model to TPU chips, and defining how these values will be sharded or replicated.

Mesh is an overloaded term that can mean either of the following:

- A physical layout of TPU chips.
- An abstract logical construct for mapping the data and model to the TPU chips.

In either case, a mesh is specified as a [**shape**](https://developers.google.com/machine-learning/glossary#shape).


## S

<br />


## shard

#TensorFlow #GoogleCloud

<br />

A logical division of the [**training set**](https://developers.google.com/machine-learning/glossary#training_set) or the
[**model**](https://developers.google.com/machine-learning/glossary#model). Typically, some process creates shards by dividing
the [**examples**](https://developers.google.com/machine-learning/glossary#example) or [**parameters**](https://developers.google.com/machine-learning/glossary#parameter) into (usually)
equal-sized chunks. Each shard is then assigned to a different machine.

Sharding a model is called [**model parallelism**](https://developers.google.com/machine-learning/glossary#model-parallelism);
sharding data is called [**data parallelism**](https://developers.google.com/machine-learning/glossary#data-parallelism).


## T

<br />


## Tensor Processing Unit (TPU)

#TensorFlow #GoogleCloud

<br />

An application-specific integrated circuit (ASIC) that optimizes the
performance of machine learning workloads. These ASICs are deployed as
multiple [**TPU chips**](https://developers.google.com/machine-learning/glossary/googlecloud#TPU_chip) on a [**TPU device**](https://developers.google.com/machine-learning/glossary/googlecloud#TPU_device).


## TPU

#TensorFlow #GoogleCloud

<br />

Abbreviation for [**Tensor Processing Unit**](https://developers.google.com/machine-learning/glossary/googlecloud#TPU).


## TPU chip

#TensorFlow #GoogleCloud

<br />

A programmable linear algebra accelerator with on-chip high bandwidth memory
that is optimized for machine learning workloads.
Multiple TPU chips are deployed on a [**TPU device**](https://developers.google.com/machine-learning/glossary/googlecloud#TPU_device).


## TPU device

#TensorFlow #GoogleCloud

<br />

A printed circuit board (PCB) with multiple [**TPU chips**](https://developers.google.com/machine-learning/glossary/googlecloud#TPU_chip),
high bandwidth network interfaces, and system cooling hardware.


## TPU node

#TensorFlow #GoogleCloud

<br />

A TPU resource on Google Cloud with a specific
[**TPU type**](https://developers.google.com/machine-learning/glossary/googlecloud#TPU_type). The TPU node connects to your
[VPC Network](https://cloud.google.com/vpc/docs/) from a
[peer VPC network](https://cloud.google.com/vpc/docs/vpc-peering).
TPU nodes are a resource defined in the
[Cloud TPU API](https://cloud.google.com/tpu/docs/reference/rest/v1/projects.locations.nodes).


## TPU Pod

#TensorFlow #GoogleCloud

<br />

A specific configuration of [**TPU devices**](https://developers.google.com/machine-learning/glossary/googlecloud#TPU_device) in a Google
data center. All of the devices in a TPU Pod are connected to one another
over a dedicated high-speed network. A TPU Pod is the largest configuration of
[**TPU devices**](https://developers.google.com/machine-learning/glossary/googlecloud#TPU_device) available for a specific TPU version.


## TPU resource

#TensorFlow #GoogleCloud

<br />

A TPU entity on Google Cloud that you create, manage, or consume. For
example, [**TPU nodes**](https://developers.google.com/machine-learning/glossary/googlecloud#TPU_node) and [**TPU types**](https://developers.google.com/machine-learning/glossary/googlecloud#TPU_type) are
TPU resources.


## TPU slice

#TensorFlow #GoogleCloud

<br />

A TPU slice is a fractional portion of the [**TPU devices**](https://developers.google.com/machine-learning/glossary/googlecloud#TPU_device) in
a [**TPU Pod**](https://developers.google.com/machine-learning/glossary/googlecloud#TPU_Pod). All of the devices in a TPU slice are connected
to one another over a dedicated high-speed network.


## TPU type

#TensorFlow #GoogleCloud

<br />

A configuration of one or more [**TPU devices**](https://developers.google.com/machine-learning/glossary/googlecloud#TPU_device) with a specific
TPU hardware version. You select a TPU type when you create
a [**TPU node**](https://developers.google.com/machine-learning/glossary/googlecloud#TPU_node) on Google Cloud. For example, a `v2-8`
TPU type is a single TPU v2 device with 8 cores. A `v3-2048` TPU type has 256
networked TPU v3 devices and a total of 2048 cores. TPU types are a resource
defined in the
[Cloud TPU API](https://cloud.google.com/tpu/docs/reference/rest/v1/projects.locations.acceleratorTypes).


## TPU worker

#TensorFlow #GoogleCloud

<br />

A process that runs on a host machine and executes machine learning programs
on [**TPU devices**](https://developers.google.com/machine-learning/glossary/googlecloud#TPU_device).


## V

<br />


## Vertex

#GoogleCloud #generativeAI Google Cloud's platform for AI and machine learning. Vertex provides tools and infrastructure for building, deploying, and managing AI applications, including access to [**Gemini**](https://developers.google.com/machine-learning/glossary#Gemini) models.

<br />