# Machine Learning Glossary: TensorFlow This page contains TensorFlow glossary terms. For all glossary terms, [click here](https://developers.google.com/machine-learning/glossary). ## C
## Cloud TPU #TensorFlow #GoogleCloud
A specialized hardware accelerator designed to speed up machine learning workloads on Google Cloud. ## D
## Dataset API (tf.data) #TensorFlow
A high-level [**TensorFlow**](https://developers.google.com/machine-learning/glossary/tensorflow#TensorFlow) API for reading data and transforming it into a form that a machine learning algorithm requires. A `tf.data.Dataset` object represents a sequence of elements, in which each element contains one or more [**Tensors**](https://developers.google.com/machine-learning/glossary/tensorflow#tensor). A `tf.data.Iterator` object provides access to the elements of a `Dataset`. ## device #TensorFlow #GoogleCloud
An overloaded term with the following two possible definitions: 1. A category of hardware that can run a TensorFlow session, including CPUs, GPUs, and [**TPUs**](https://developers.google.com/machine-learning/glossary/tensorflow#TPU). 2. When training an ML model on [**accelerator chips**](https://developers.google.com/machine-learning/glossary#accelerator-chip) (GPUs or TPUs), the part of the system that actually manipulates [**tensors**](https://developers.google.com/machine-learning/glossary/tensorflow#tensor) and [**embeddings**](https://developers.google.com/machine-learning/glossary#embedding_layer). The device runs on accelerator chips. In contrast, the [**host**](https://developers.google.com/machine-learning/glossary/tensorflow#host) typically runs on a CPU. ## E
## eager execution #TensorFlow
A TensorFlow programming environment in which [**operations**](https://developers.google.com/machine-learning/glossary/tensorflow#Operation) run immediately. In contrast, operations called in [**graph execution**](https://developers.google.com/machine-learning/glossary/tensorflow#graph_execution) don't run until they are explicitly evaluated. Eager execution is an [imperative interface](https://wikipedia.org/wiki/Imperative_programming), much like the code in most programming languages. Eager execution programs are generally far easier to debug than graph execution programs. ## Estimator #TensorFlow
A deprecated TensorFlow API. Use [**tf.keras**](https://developers.google.com/machine-learning/glossary/tensorflow#tf.keras) instead of Estimators. ## F
## feature engineering #fundamentals #TensorFlow
A process that involves the following steps: 1. Determining which [**features**](https://developers.google.com/machine-learning/glossary#feature) might be useful in training a model. 2. Converting raw data from the dataset into efficient versions of those features. For example, you might determine that `temperature` might be a useful feature. Then, you might experiment with [**bucketing**](https://developers.google.com/machine-learning/glossary#bucketing) to optimize what the model can learn from different `temperature` ranges. Feature engineering is sometimes called [**feature extraction**](https://developers.google.com/machine-learning/glossary#feature_extraction) or [**featurization**](https://developers.google.com/machine-learning/glossary#featurization). #### Click the icon for additional notes about TensorFlow. In TensorFlow, feature engineering often means converting raw log file entries to [**tf.Example**](https://developers.google.com/machine-learning/glossary/tensorflow#tf.Example) protocol buffers. See also [tf.Transform](https://github.com/tensorflow/transform). *** ** * ** *** See [Numerical data: How a model ingests data using feature vectors](https://developers.google.com/machine-learning/crash-course/numerical-data/feature-vectors) in Machine Learning Crash Course for more information. ## feature spec #TensorFlow
Describes the information required to extract [**features**](https://developers.google.com/machine-learning/glossary#feature) data from the [**tf.Example**](https://developers.google.com/machine-learning/glossary/tensorflow#tf.Example) protocol buffer. Because the tf.Example protocol buffer is just a container for data, you must specify the following: - The data to extract (that is, the keys for the features) - The data type (for example, float or int) - The length (fixed or variable) ## G
## graph #TensorFlow
In TensorFlow, a computation specification. Nodes in the graph represent operations. Edges are directed and represent passing the result of an operation (a [**Tensor**](https://developers.google.com/machine-learning/glossary/tensorflow#tensor)) as an operand to another operation. Use [**TensorBoard**](https://developers.google.com/machine-learning/glossary/tensorflow#TensorBoard) to visualize a graph. ## graph execution #TensorFlow
A TensorFlow programming environment in which the program first constructs a [**graph**](https://developers.google.com/machine-learning/glossary/tensorflow#graph) and then executes all or part of that graph. Graph execution is the default execution mode in TensorFlow 1.x. Contrast with [**eager execution**](https://developers.google.com/machine-learning/glossary/tensorflow#eager_execution). ## H
## host #TensorFlow #GoogleCloud
When training an ML model on [**accelerator chips**](https://developers.google.com/machine-learning/glossary#accelerator-chip) (GPUs or [**TPUs**](https://developers.google.com/machine-learning/glossary/tensorflow#TPU)), the part of the system that controls both of the following: - The overall flow of the code. - The extraction and transformation of the input pipeline. The host typically runs on a CPU, not on an accelerator chip; the [**device**](https://developers.google.com/machine-learning/glossary/tensorflow#device) manipulates [**tensors**](https://developers.google.com/machine-learning/glossary/tensorflow#tensor) on the accelerator chips. ## L
## Layers API (tf.layers) #TensorFlow
A TensorFlow API for constructing a [**deep**](https://developers.google.com/machine-learning/glossary#deep_model) neural network as a composition of layers. The Layers API lets you build different types of [**layers**](https://developers.google.com/machine-learning/glossary#layer), such as: - `tf.layers.Dense` for a [**fully-connected layer**](https://developers.google.com/machine-learning/glossary#fully_connected_layer). - `tf.layers.Conv2D` for a convolutional layer. The Layers API follows the [**Keras**](https://developers.google.com/machine-learning/glossary#Keras) layers API conventions. That is, aside from a different prefix, all functions in the Layers API have the same names and signatures as their counterparts in the Keras layers API. ## M
## mesh #TensorFlow #GoogleCloud
In ML parallel programming, a term associated with assigning the data and model to TPU chips, and defining how these values will be sharded or replicated. Mesh is an overloaded term that can mean either of the following: - A physical layout of TPU chips. - An abstract logical construct for mapping the data and model to the TPU chips. In either case, a mesh is specified as a [**shape**](https://developers.google.com/machine-learning/glossary#shape). ## metric #TensorFlow #Metric
A statistic that you care about. An [**objective**](https://developers.google.com/machine-learning/glossary#objective) is a metric that a machine learning system tries to optimize. ## N
## node (TensorFlow graph) #TensorFlow
An operation in a TensorFlow [**graph**](https://developers.google.com/machine-learning/glossary/tensorflow#graph). ## O
## operation (op) #TensorFlow
In TensorFlow, any procedure that creates, manipulates, or destroys a [**Tensor**](https://developers.google.com/machine-learning/glossary/tensorflow#tensor). For example, a matrix multiply is an operation that takes two Tensors as input and generates one Tensor as output. ## P
## Parameter Server (PS) #TensorFlow
A job that keeps track of a model's [**parameters**](https://developers.google.com/machine-learning/glossary#parameter) in a distributed setting. ## Q
## queue #TensorFlow
A TensorFlow [**Operation**](https://developers.google.com/machine-learning/glossary/tensorflow#Operation) that implements a queue data structure. Typically used in I/O. ## R
## rank (Tensor) #TensorFlow
The number of dimensions in a [**Tensor**](https://developers.google.com/machine-learning/glossary/tensorflow#tensor). For example, a scalar has rank 0, a vector has rank 1, and a matrix has rank 2. Not to be confused with [**rank (ordinality)**](https://developers.google.com/machine-learning/glossary#rank_ordinality). ## root directory #TensorFlow
The directory you specify for hosting subdirectories of the TensorFlow checkpoint and events files of multiple models. ## S
## SavedModel #TensorFlow
The recommended format for saving and recovering TensorFlow models. SavedModel is a language-neutral, recoverable serialization format, which enables higher-level systems and tools to produce, consume, and transform TensorFlow models. See the [Saving and Restoring section](https://www.tensorflow.org/guide/saved_model) of the TensorFlow Programmer's Guide for complete details. ## Saver #TensorFlow
A [TensorFlow object](https://www.tensorflow.org/api_docs/python/tf/compat/v1/train/Saver) responsible for saving model checkpoints. ## shard #TensorFlow #GoogleCloud
A logical division of the [**training set**](https://developers.google.com/machine-learning/glossary#training_set) or the [**model**](https://developers.google.com/machine-learning/glossary#model). Typically, some process creates shards by dividing the [**examples**](https://developers.google.com/machine-learning/glossary#example) or [**parameters**](https://developers.google.com/machine-learning/glossary#parameter) into (usually) equal-sized chunks. Each shard is then assigned to a different machine. Sharding a model is called [**model parallelism**](https://developers.google.com/machine-learning/glossary#model-parallelism); sharding data is called [**data parallelism**](https://developers.google.com/machine-learning/glossary#data-parallelism). ## summary #TensorFlow
In TensorFlow, a value or set of values calculated at a particular [**step**](https://developers.google.com/machine-learning/glossary#step), usually used for tracking model metrics during training. ## T
## Tensor #TensorFlow
The primary data structure in TensorFlow programs. Tensors are N-dimensional (where N could be very large) data structures, most commonly scalars, vectors, or matrixes. The elements of a Tensor can hold integer, floating-point, or string values. ## TensorBoard #TensorFlow
The dashboard that displays the summaries saved during the execution of one or more TensorFlow programs. ## TensorFlow #TensorFlow
A large-scale, distributed, machine learning platform. The term also refers to the base API layer in the TensorFlow stack, which supports general computation on dataflow graphs. Although TensorFlow is primarily used for machine learning, you may also use TensorFlow for non-ML tasks that require numerical computation using dataflow graphs. ## TensorFlow Playground #TensorFlow
A program that visualizes how different [**hyperparameters**](https://developers.google.com/machine-learning/glossary#hyperparameter) influence model (primarily neural network) training. Go to to experiment with TensorFlow Playground. ## TensorFlow Serving #TensorFlow
A platform to deploy trained models in production. ## Tensor Processing Unit (TPU) #TensorFlow #GoogleCloud
An application-specific integrated circuit (ASIC) that optimizes the performance of machine learning workloads. These ASICs are deployed as multiple [**TPU chips**](https://developers.google.com/machine-learning/glossary/tensorflow#TPU_chip) on a [**TPU device**](https://developers.google.com/machine-learning/glossary/tensorflow#TPU_device). ## Tensor rank #TensorFlow
See [**rank (Tensor)**](https://developers.google.com/machine-learning/glossary/tensorflow#rank_Tensor). ## Tensor shape #TensorFlow
The number of elements a [**Tensor**](https://developers.google.com/machine-learning/glossary/tensorflow#tensor) contains in various dimensions. For example, a `[5, 10]` Tensor has a shape of 5 in one dimension and 10 in another. ## Tensor size #TensorFlow
The total number of scalars a [**Tensor**](https://developers.google.com/machine-learning/glossary/tensorflow#tensor) contains. For example, a `[5, 10]` Tensor has a size of 50. ## tf.Example #TensorFlow
A standard [protocol buffer](https://developers.google.com/protocol-buffers/) for describing input data for machine learning model training or inference. ## tf.keras #TensorFlow
An implementation of [**Keras**](https://developers.google.com/machine-learning/glossary#Keras) integrated into [**TensorFlow**](https://developers.google.com/machine-learning/glossary/tensorflow#TensorFlow). ## TPU #TensorFlow #GoogleCloud
Abbreviation for [**Tensor Processing Unit**](https://developers.google.com/machine-learning/glossary/tensorflow#TPU). ## TPU chip #TensorFlow #GoogleCloud
A programmable linear algebra accelerator with on-chip high bandwidth memory that is optimized for machine learning workloads. Multiple TPU chips are deployed on a [**TPU device**](https://developers.google.com/machine-learning/glossary/tensorflow#TPU_device). ## TPU device #TensorFlow #GoogleCloud
A printed circuit board (PCB) with multiple [**TPU chips**](https://developers.google.com/machine-learning/glossary/tensorflow#TPU_chip), high bandwidth network interfaces, and system cooling hardware. ## TPU node #TensorFlow #GoogleCloud
A TPU resource on Google Cloud with a specific [**TPU type**](https://developers.google.com/machine-learning/glossary/tensorflow#TPU_type). The TPU node connects to your [VPC Network](https://cloud.google.com/vpc/docs/) from a [peer VPC network](https://cloud.google.com/vpc/docs/vpc-peering). TPU nodes are a resource defined in the [Cloud TPU API](https://cloud.google.com/tpu/docs/reference/rest/v1/projects.locations.nodes). ## TPU Pod #TensorFlow #GoogleCloud
A specific configuration of [**TPU devices**](https://developers.google.com/machine-learning/glossary/tensorflow#TPU_device) in a Google data center. All of the devices in a TPU Pod are connected to one another over a dedicated high-speed network. A TPU Pod is the largest configuration of [**TPU devices**](https://developers.google.com/machine-learning/glossary/tensorflow#TPU_device) available for a specific TPU version. ## TPU resource #TensorFlow #GoogleCloud
A TPU entity on Google Cloud that you create, manage, or consume. For example, [**TPU nodes**](https://developers.google.com/machine-learning/glossary/tensorflow#TPU_node) and [**TPU types**](https://developers.google.com/machine-learning/glossary/tensorflow#TPU_type) are TPU resources. ## TPU slice #TensorFlow #GoogleCloud
A TPU slice is a fractional portion of the [**TPU devices**](https://developers.google.com/machine-learning/glossary/tensorflow#TPU_device) in a [**TPU Pod**](https://developers.google.com/machine-learning/glossary/tensorflow#TPU_Pod). All of the devices in a TPU slice are connected to one another over a dedicated high-speed network. ## TPU type #TensorFlow #GoogleCloud
A configuration of one or more [**TPU devices**](https://developers.google.com/machine-learning/glossary/tensorflow#TPU_device) with a specific TPU hardware version. You select a TPU type when you create a [**TPU node**](https://developers.google.com/machine-learning/glossary/tensorflow#TPU_node) on Google Cloud. For example, a `v2-8` TPU type is a single TPU v2 device with 8 cores. A `v3-2048` TPU type has 256 networked TPU v3 devices and a total of 2048 cores. TPU types are a resource defined in the [Cloud TPU API](https://cloud.google.com/tpu/docs/reference/rest/v1/projects.locations.acceleratorTypes). ## TPU worker #TensorFlow #GoogleCloud
A process that runs on a host machine and executes machine learning programs on [**TPU devices**](https://developers.google.com/machine-learning/glossary/tensorflow#TPU_device).