Debugging guide

Earth Engine is unlike traditional image processing, GIS or other desktop software used for geospatial data analysis. Algorithms you create in Earth Engine run in the Google cloud, distributed over many computers. Debugging can be challenging because errors can occur either in the client-side JavaScript code or the server-side execution of the coded instructions, and result from scaling problems as well as syntactic or logical errors. The bits of the program that are running somewhere in the cloud are not available to inspect, unless you ask for them. This document presents debugging strategies, tools and solutions to help you resolve common errors and debug Earth Engine scripts.

Syntax errors

Syntax errors in client-side JavaScript are generally caught by the Code Editor, which alerts you to their presence with a on the left side of the editor panel. Hovering on the with the cursor will provide a clue about the source of the error. For example, Figure 1 illustrates what happens when a list is incorrectly coded. Specifically, without a closing bracket (]), the list is not correctly specified, a problem the Code Editor explains as Expected ']' to match '[' from line 1 and saw ';'.

Debugging_01_syntax_error.png
Figure 1. The Code Editor provides information about syntactic errors in JavaScript.

Client-side errors

Despite syntactically correct JavaScript, there may be errors associated with the consistency or logic of the script. The following examples demonstrate errors from using a variable and method that don't exist:

Error — this code doesn't work!

// Load a Sentinel-2 image.
var image = ee.Image('USGS/SRTMGL1_003');

// Error: "bandNames" is not defined in this scope.
var display = image.visualize({bands: bandNames, min: 0, max: 9000});

// Error: image.selfAnalyze is not a function
var silly = image.selfAnalyze();

The first error informs you that the bandNames variable is not defined in the scope in which it's referenced. As a solution, set the variable, or provide a list argument for the bands parameter. The second error demonstrates what happens when the non-existent selfAnalyze() function is called. Since that isn't a real method on images, the error tells you it's not a function. In both cases, the error is descriptive of the problem.

Casting

The "...is not a function" error may result from Earth Engine not knowing the type of a variable. Common manifestations of this problem result from:

  • Doing something to an object returned by first() (the type of the elements in a collection is unknown).
  • Doing something to an object returned by get() (the type of element stored in a property is unknown).
  • Doing something to a function argument (in the function) when the type of the argument is unknown.

For an example of the former:

Error — this code doesn't work!

var collection = ee.FeatureCollection('USDOS/LSIB_SIMPLE/2017');

// Error: collection.first(...).area is not a function
var area = collection.first().area();

The solution in all cases is to cast the object of unknown type with the constructor of the known type. Continuing the previous example, the solution is to cast to ee.Feature:

Solution — use a cast!

Code Editor (JavaScript)

var area = ee.Feature(collection.first()).area();

(It's worth noting that you can safely call any method on Element here because that's what Earth Engine thinks it is).

Avoid mixing client and server functions

The following example is less obvious:

Error — this code doesn't do what you want

// Don't mix EE objects and JavaScript objects:
var image = ee.Image('USGS/SRTMGL1_003');
var nonsense = image + 2;

// You can print this, but it's not what you were hoping for.
print(nonsense);

// Error: g.eeObject.name is not a function
Map.addLayer(nonsense);

Supposing the author of this code intended to add 2 to every pixel in the image, this is not the right way to do it. Specifically, this code wrongly mixes a server-side object (image) with a client-side JavaScript operator (+). The results may be surprising. In the first case, printing of nonsense, JavaScript will perform the requested operation (+) by converting both image and 2 to strings, then concatenating them. The resultant string is unintended. In the second case, adding nonsense to the map, the cryptic g.eeObject.name is not a function error is displayed because the object being added to the map, nonsense, is a string, not an EE object. To avoid possibly unintended results and uninformative errors, don't mix server objects and functions with JavaScript objects, primitives or functions. The solution is this example is to use a server function:

Solution — use a server function!

Code Editor (JavaScript)

Map.addLayer(image.add(2));

For more detailed explanation of client vs. server in Earth Engine, see this page and/or this tutorial.

Browser lock

Aw, snap! If you ever experience an error that results in the Chrome browser locking, you may see a window like Figure 2.

Debugging_02_browser_lock.png
Figure 2. Browser lock.

When this occurs, it is often a result of JavaScript running in the client taking too long, or waiting for something from Earth Engine. Two common sources of this error are for-loops and/or getInfo() in your code, with the worst-case scenario of getInfo() inside a for-loop. For-loops can cause the browser to lock because the code runs on your machine. On the other hand, getInfo() synchronously requests the result of a computation from Earth Engine, blocking until the result is received. If the computation takes a long time, the blocking could cause your browser to lock. Avoid both for-loops and getInfo(). See this page for details.

Server-side errors

Despite logical consistency in the client JavaScript, there may be bugs which only become apparent at run time on the server. The following example demonstrates what happens when trying to get a band that doesn't exist:

Error — this code doesn't work!

// Load a Sentinel-2 image.
var s2image = ee.Image('COPERNICUS/S2/20160625T100617_20160625T170310_T33UVR');

// Error: Image.select: Pattern 'nonBand' did not match any bands.
print(s2image.select(['nonBand']));

In this example, the error informs you that there is no band named nonBand. The possibly obvious solution is to specify a band name that does exist. You can discover the band names by printing the image and inspecting it in the console, or by printing the list of band names returned by image.bandNames().

Immutability

Server-side objects you create in Earth Engine are immutable. (Any ee.Object is a server side Object). That means that if you want to make a change to the object, you have to save the changed state into a new variable. For example, this won't work to set a property on the Sentinel-2 image:

Error — this code doesn't do what you want!

var s2image = ee.Image('COPERNICUS/S2/20160625T100617_20160625T170310_T33UVR');
s2image.set('myProperty', 'This image is not assigned to a variable');

// This will not result in an error, but will not find 'myProperty'.
print(s2image.get('myProperty')); // null

In this example, s2image.set() returns a copy of the image with the new property, but the image stored in the s2image variable is unchanged. You need to save the image returned by s2image.set() in a new variable. For example:

Solution — capture the result in a variable!

Code Editor (JavaScript)

s2image = s2image.set('myProperty', 'OK');
print(s2image.get('myProperty')); // OK

Mapped functions

Another context in which client and server functions don't mix is in mapped functions. Specifically, the operations specified by the mapped function run in the cloud, so client functions such as print(), getInfo() or any method on Map, Chart or Export won't work in mapped functions. For example:

Error — this code doesn't work!

var collection = ee.ImageCollection('MODIS/006/MOD44B');

// Error: A mapped function's arguments cannot be used in client-side operations
var badMap3 = collection.map(function(image) {
  print(image);
  return image;
});

This somewhat cryptic error results from the process Earth Engine uses to turn this code into a set of instructions that can be run on Google servers. Specifically, it means that Earth Engine can't find a server-side function called print(), because there isn't one. More generally, client-side functions and control structures cannot be used to operate on the argument image passed to the mapped function. To avoid this error, avoid the use of client-side functions in mapped functions. See this page to learn more about the distinction between client and server functions.

Mapped functions have additional requirements, all of which must be met to avoid errors. For example, mapped functions must return something. Although the Code Editor detects this problem and issues the error, it is specific to mapped functions that run on the server:

Error — this code doesn't work!

var collection = ee.ImageCollection('MODIS/006/MOD44B');

// Error: User-defined methods must return a value.
var badMap1 = collection.map(function(image) {
  // Do nothing.
});

The possibly obvious solution is to return something. But it can't return just any type of thing. Specifically, functions mapped over an ImageCollection or FeatureCollection must return an Image or Feature. For example, you can't return a date from a function mapped over an ImageCollection:

Error — this code doesn't work!

var collection = ee.ImageCollection('MODIS/006/MOD44B');

var badMap2 = collection.map(function(image) {
  return image.date();
});

// Error: Collection.map: A mapped algorithm must return a Feature or Image.
print(badMap2);

To avoid this, return the input image with a new property set. Then, if you need a list of the dates of the images in the collection, you can use aggregate_array():

Solution — set a property!

Code Editor (JavaScript)

var collection = ee.ImageCollection('MODIS/006/MOD44B');

var okMap2 = collection.map(function(image) {
  return image.set('date', image.date());
});
print(okMap2);

// Get a list of the dates.
var datesList = okMap2.aggregate_array('date');
print(datesList);

Scaling errors

Though a script may be valid JavaScript, without logical errors, and represent a valid set of instructions for the server, in parallelizing and executing the computation, the resultant objects may be too big, too numerous, or take too long to compute. In this case, you will get an error indicating that the algorithm can't be scaled. These errors are generally the most difficult to diagnose and resolve. Examples of this type of error include:

  • Computation timed out
  • Too many concurrent aggregations
  • User memory limit exceeded
  • An internal error has occurred

Improving the scaling of your code will let you get results faster, and also improve the availability of computing resources for all users. Each type of error is discussed below, following a brief aside about reduceRegion(), a commonly used function that is notorious for being able to cause every type of scaling error.

reduceRegion()

Although reduceRegion() greedily consumes enough pixels to trigger an exciting variety of errors, there are also parameters intended to control the computation, so you can overcome the errors. For example, consider the following inadvisable reduction:

Error — this code doesn't work!

var absurdComputation = ee.Image(1).reduceRegion({
  reducer: 'count',
  geometry: ee.Geometry.Rectangle([-180, -90, 180, 90], null, false),
  scale: 100,
});

// Error: Image.reduceRegion: Too many pixels in the region.
//        Found 80300348117, but only 10000000 allowed.
print(absurdComputation);

This silly example is just for demonstration. The purpose of this error is to ask you whether you really want to reduce 80300348117 (that's 80 billion) pixels. If not, increase the scale (pixel size in meters) accordingly, or set bestEffort to true, to recompute a larger scale automatically. See the reduceRegion() page for more details about these parameters.

Computation timed out

Suppose you need all those pixels in your computation. If so, you can increase the maxPixels parameter to allow the computation to succeed. However, it's going to take Earth Engine some time to finish the computation. As a result, a “computation timed out” error might be thrown:

Bad — don't do this!

var ridiculousComputation = ee.Image(1).reduceRegion({
  reducer: 'count',
  geometry: ee.Geometry.Rectangle([-180, -90, 180, 90], null, false),
  scale: 100,
  maxPixels: 1e11
});

// Error: Computation timed out.
print(ridiculousComputation);

What this error means is that Earth Engine waited about five minutes before stopping the computation. Exporting allows Earth Engine to perform the computation in an environment with longer allowable running times (but not more memory). As the return value from reduceRegion() is a dictionary, you can use the dictionary to set the properties of a feature with null geometry:

Good — use Export!

Code Editor (JavaScript)

Export.table.toDrive({
  collection: ee.FeatureCollection([
    ee.Feature(null, ridiculousComputation)
  ]),
  description: 'ridiculousComputation',
  fileFormat: 'CSV'
});

Too many concurrent aggregations

The "aggregations" part of this error refers to operations that are spread out over multiple machines (such as reductions that span more than one tile). Earth Engine has limits in place to prevent too many such aggregations from being run concurrently. In this example, the "Too many concurrent aggregations" error is triggered by a reduction within a map:

Bad — don't do this!

var collection = ee.ImageCollection('LANDSAT/LT05/C02/T1')
    .filterBounds(ee.Geometry.Point([-123, 43]));

var terribleAggregations = collection.map(function(image) {
  return image.set(image.reduceRegion({
    reducer: 'mean',
    geometry: image.geometry(),
    scale: 30,
    maxPixels: 1e9
  }));
});

// Error: Quota exceeded: Too many concurrent aggregations.
print(terribleAggregations);

Assuming that the purpose of this code is to get image statistics for each image, one possible solution is to Export the result. For example, using the fact that an ImageCollection is also a FeatureCollection, the metadata associated with the images can be exported as a table:

Good — use Export!

Code Editor (JavaScript)

Export.table.toDrive({
  collection: terribleAggregations,
  description: 'terribleAggregations',
  fileFormat: 'CSV'
});

User memory limit exceeded

One way your algorithms get parallelized in Earth Engine is by splitting the inputs into tiles, running the same computation separately on each tile, then combining the results. As a consequence, all of the inputs necessary to compute an output tile have to fit into memory. For example, when the input is an image with many bands, that could end up taking a lot of memory if all the bands are used in the computation. To demonstrate, this example uses too much memory by forcing (unnecessarily) an entire image collection into a tile:

Bad — don't do this!

var memoryHog = ee.ImageCollection('LANDSAT/LT05/C02/T1').select('B.')
  .toArray()
  .arrayReduce(ee.Reducer.mean(), [0])
  .arrayProject([1])
  .arrayFlatten([['B1', 'B2', 'B3', 'B4', 'B5', 'B6', 'B7']])
  .reduceRegion({
    reducer: 'mean',
    geometry: ee.Geometry.Point([-122.27, 37.87]).buffer(1000),
    scale: 1,
    bestEffort: true,
  });

// Error: User memory limit exceeded.
print(memoryHog);

This very bad code demonstrates one reason to not use arrays unless you really need to (see also the 'Avoid converting type unnecessarily' section). When that collection is converted to a gigantic array, the array has to be loaded into memory all at once. Because it's a long time series of images, the array is large and won't fit in memory.

One possible solution is to set the tileScale parameter to a higher value. Higher values of tileScale result in tiles smaller by a factor of tileScale^2. For example, the following allows the computation to succeed:

Code Editor (JavaScript)

var smallerHog = ee.ImageCollection('LANDSAT/LT05/C02/T1').select('B.')
  .toArray()
  .arrayReduce(ee.Reducer.mean(), [0])
  .arrayProject([1])
  .arrayFlatten([['B1', 'B2', 'B3', 'B4', 'B5', 'B6', 'B7']])
  .reduceRegion({
    reducer: 'mean',
    geometry: ee.Geometry.Point([-122.27, 37.87]).buffer(1000),
    scale: 1,
    bestEffort: true,
    tileScale: 16
  });

print(smallerHog);

However, the much preferred solution is to not use arrays unnecessarily, so you don't need to fiddle with tileScale at all:

Good — avoid Arrays!

Code Editor (JavaScript)

var okMemory = ee.ImageCollection('LANDSAT/LT05/C02/T1').select('B.')
  .mean()
  .reduceRegion({
    reducer: 'mean',
    geometry: ee.Geometry.Point([-122.27, 37.87]).buffer(1000),
    scale: 1,
    bestEffort: true,
  });

print(okMemory);

Unless necessary to resolve a memory error, you should not set tileScale as smaller tiles also result in larger parallelization overhead.

Internal errors

You may encounter an error that looks like:

If you get this error, please click the "Report error" link which appears in the console. You can also Send feedback from the Help button. This error can result from logical errors in your script that only become obvious at runtime or a problem with the inner workings of Earth Engine. In either case, the error is uninformative and should be reported so that it can be fixed.

Internal errors include a request ID, like the following:

These strings act as unique identifiers to help the Earth Engine team to identify specific issues. Please include this string in bug reports.

Debugging methods

You coded up your analysis, clicked Run and got an error. Now what? This section describes general debugging techniques to isolate the problem and fix it.

Suppose you have a very complex analysis that produces an error. If it not obvious where the error originates, a good initial strategy is to print or visualize intermediate objects and inspect them to ensure the structure of the object is consistent with the logic in your script. Specifically, you can inspect pixel values of layers added to the map with the Inspector tab of the Code Editor. If you print something, be sure to expand its properties with the zippies (▶). Some things to check include:

  • Band names. Do image band names match your code?
  • Pixel values. Does your data have the right range? Is it masked appropriately?
  • Null. Is anything null that shouldn't be?
  • Sizes. Is the size zero when it shouldn't be?

aside()

It can be onerous to put every intermediate step in an analysis into a variable so that it can be printed and inspected. To print intermediate values from a long chain of function calls, you can use the aside() method. For example:

Code Editor (JavaScript)

var image = ee.Image(ee.ImageCollection('COPERNICUS/S2')
    .filterBounds(ee.Geometry.Point([-12.294402, 168.830071]))
    .aside(print)
    .filterDate('2011-01-01', '2016-12-31')
    .first());

Just remember, aside(print) is still calling print(), and it will still fail in mapped functions. You can also use aside with user defined functions. For example:

Code Editor (JavaScript)

var composite = ee.ImageCollection('LANDSAT/LC08/C02/T1_TOA')
    .filterBounds(ee.Geometry.Point([106.9155, 47.9177]))
    .map(function(image) {
      return image.addBands(image.normalizedDifference(['B5', 'B4']));
    })
    .aside(Map.addLayer, {bands: ['B4', 'B3', 'B2'], max: 0.3}, 'collection')
    .qualityMosaic('nd');

Map.setCenter(106.9155, 47.9177, 11);
Map.addLayer(composite, {bands: ['B4', 'B3', 'B2'], max: 0.3}, 'composite');

Running a function on first()

Printing and visualizing are useful for debugging when available, but you're debugging a function mapped over a collection, then you can't print() in the function, as described in this section. In this case, it's useful to isolate problematic elements in the collection and test the mapped function on an individual element. When you're testing the function without mapping it, you can use print statements to understand the problem. Consider the following example:

Error — this code doesn't work!

var image = ee.Image('COPERNICUS/S2/20150821T111616_20160314T094808_T30UWU');

var someFeatures = ee.FeatureCollection([
  ee.Feature(ee.Geometry.Point([-2.0256, 48.4374])),
  ee.Feature(ee.Geometry.Point([-2.8084, 48.3727])),
  ee.Feature(ee.Geometry.Point([-1.2277, 48.2932])),
  ee.Feature(ee.Geometry.Point([-1.7372, 48.6511])),
]);

var problem = someFeatures.map(function(feature) {

  var dictionary = image.reduceRegion({
    reducer: 'first',
    geometry: feature.geometry(),
    scale: 10,
  });

  return feature.set({
    result: ee.Number(dictionary.get('B5'))
                .divide(dictionary.get('B4'))
  });
});

// Error in map(ID=2):
//  Number.divide: Parameter 'left' is required.
print(problem);

To debug this, it is instructional to examine the error. Fortunately, this helpful error informs you that there is a problem with the feature with ID=2. To investigate further, it's useful to refactor the code a bit. Specifically, you can't have print statements in the function when it's mapped over a collection, as described in this section. The debugging goal is to isolate the problematic feature, and run the function with some print statements in it. Using the same image and features as above:

Code Editor (JavaScript)

// Define a function to be mapped over the collection.
var functionToMap = function(feature) {

  var dictionary = image.reduceRegion({
    reducer: 'first',
    geometry: feature.geometry(),
    scale: 10,
  });

  // Debug:
  print(dictionary);

  return feature.set({
    result: ee.Number(dictionary.get('B5'))
                .divide(dictionary.get('B4'))
  });
};

// Isolate the feature that's creating problems.
var badFeature = ee.Feature(someFeatures
    .filter(ee.Filter.eq('system:index', '2'))
    .first());

// Test the function with print statements added.
functionToMap(badFeature);

// Inspect the bad feature in relation to the image.
Map.centerObject(badFeature, 11);
Map.addLayer(badFeature, {}, 'bad feature');
Map.addLayer(image, {bands: ['B4', 'B3', 'B2'], max: 3000}, 'image');

Now, because the function is run only on one feature, you can put a print call inside. Inspect the printed object to discover (ah ha!) that the object returned by reduceRegion() has nulls for every band. That explains why the division is failing: because you can't divide null by null. Why is it null in the first place? To investigate, add the input image and the bad feature to the map, and center on the bad feature. In doing so, you discover that the problem is due to the point being outside the bounds of the image. Based on this discovery, the debugged code is:

Code Editor (JavaScript)

var functionToMap = function(feature) {
  var dictionary = image.reduceRegion({
    reducer: 'first',
    geometry: feature.geometry(),
    scale: 10,
  });
  return feature.set({
    result: ee.Number(dictionary.get('B5'))
                .divide(dictionary.get('B4'))
  });
};

var noProblem = someFeatures
    .filterBounds(image.geometry())
    .map(functionToMap);

print(noProblem);

Profiler

The profiler is part of the Code Editor (see this page for details on how to activate the Profiler tab). For a demonstration, choose one of the example scripts (e.g. Scripts > Examples > Image > Central Pivot Irrigation Detector). Click Run with profiler and watch the profiler tab. Note that the computational and memory resources for the operations required by your script are displayed in sorted order. Look for the entries at the top of the profiler for information about the most resource intensive operations. For long-running or inefficient scripts, the entries at the top of the profiler provide clues about where to focus efforts to optimize the script. Important note: the profiler itself influences the performance of the script, so you should only run it when necessary.