TensorFlow and Earth Engine

TensorFlow is an open source ML platform that supports advanced ML methods such as deep learning. This page describes TensorFlow specific features in Earth Engine. Although TensorFlow models are developed and trained outside Earth Engine, the Earth Engine API provides methods for exporting training and testing data in TFRecord format and importing/exporting imagery in TFRecord format. See the TensorFlow examples page for more information about how to develop pipelines for using TensorFlow with data from Earth Engine. See the TFRecord page to learn more about how Earth Engine writes data to TFRecord files.

ee.Model

The ee.Model package handles interaction with TensorFlow backed machine learning models.

Interacting with models hosted on AI Platform

A new ee.Model instance can be created with ee.Model.fromAiPlatformPredictor(). This is an ee.Model object that packages Earth Engine data into tensors, forwards them as predict requests to Google AI Platform then automatically reassembles the responses into Earth Engine data types. Note that depending on the size and complexity of your model and its inputs, you may wish to adjust the minimum node size of your AI Platform model to accommodate a high volume of predictions.

Earth Engine requires AI Platform models to use TensorFlow's SavedModel format. Before a hosted model can interact with Earth Engine, its inputs/outputs need to be compatible with the TensorProto interchange format, specifically serialized TensorProtos in base64. To make this easier, the Earth Engine CLI has the model prepare command that wraps an existing SavedModel in the required operations to convert input/output formats.

Image Predictions

Use model.predictImage() to make predictions on an ee.Image using a hosted model. The return type of predictImage() is an ee.Image which can be added to the map, used in other computations, exported, etc. Earth Engine will automatically tile the input bands and adjust the output projection for scale changes and overtiling as needed. (See the TFRecord doc for more information on how tiling works). Note that Earth Engine will always forward 3D tensors to your model even when bands are scalar (the last dimension will be 1).

Nearly all convolutional models will have a fixed input projection (that of the data on which the model was trained). In this case, set the fixInputProj parameter to true in your call to ee.Model.fromAiPlatformPredictor(). When visualizing predictions, use caution when zooming out on a model that has a fixed input projection. This is for the same reason as described here. Specifically, zooming to a large spatial scope can result in requests for too much data and may manifest as slowdowns or rejections by AI Platform.