Increase your range with Geospatial Depth

Geospatial Depth Hero

The ARCore Depth API now supports Geospatial Depth, which automatically increases the range and speed of the Depth API when Streetscape Geometry is also enabled. When in a location with VPS coverage and with Streetscape Geometry enabled, the output images from the Depth API include terrain and building geometry retrieved in the area out to 65 meters from the current position. This depth data retrieved from geometry is merged with local depth observations and gets updated as the user moves to a new location.

ARCore Depth API calls now provide both local observations from the camera as well as buildings and terrain from Streetscape Geometry, merged into a single depth image.

Device compatibility

Geospatial Depth is available on all devices that support the Depth API. This feature does not require a supported hardware depth sensor, such as a time-of-flight (ToF) sensor. However, the Depth API makes use of any supported hardware sensors that a device may have.

Performance impact

Geospatial Depth introduces a small one-time computation at the start of the session to integrate Streetscape Geometry into the depth representation when initially downloaded, but otherwise does not measurably increase depth computation cost.

Depth range

Without Geospatial Depth, typical ranges in depth images are around 20-30 meters away, with the density and accuracy of depth observations reduced beyond that range. With Geospatial Depth enabled, it is typical to see densely sampled depth values reach the maximum of 65.535 meters, even with a small amount of initial movement.

Use cases

The ARCore Depth API can be used for all existing use-cases already supported. With Geospatial Depth, depth images obtained in VPS-supported locations will be populated with long-range depth faster than before, enabling use-cases targeting long-range depth in outdoor environments. Some use-cases include:

  • Building-scale occlusion of virtual content and other visual effects
  • Outdoor navigation
  • Distance measurements

Limitations

Geospatial Depth is only supported in areas that support VPS localization and Streetscape Geometry. In other areas, the ARCore Depth API will perform as normal without Geospatial values.

Prerequisites

Make sure that you understand fundamental AR concepts and how to configure an ARCore session before proceeding.

Enable Geospatial Depth

In a new ARCore session, check whether a user's device supports Depth and the Geospatial API. Not all ARCore-compatible devices support the Depth API due to processing power constraints.

To save resources, depth is disabled by default on ARCore. Enable depth mode to have your app use the Depth API. Additionally, enable Geospatial mode and Streetscape Geometry to use Geospatial Depth.

Java

Config config = session.getConfig();

// Check whether the user's device supports the Depth API.
boolean isDepthSupported = session.isDepthModeSupported(Config.DepthMode.AUTOMATIC);
boolean isGeospatialSupported =
    session.isGeospatialModeSupported(Config.GeospatialMode.ENABLED);
if (isDepthSupported && isGeospatialSupported) {
  // These three settings are needed to use Geospatial Depth.
  config.setDepthMode(Config.DepthMode.AUTOMATIC);
  config.setGeospatialMode(Config.GeospatialMode.ENABLED);
  config.setStreetscapeGeometryMode(Config.StreetscapeGeometryMode.ENABLED);
}
session.configure(config);

Kotlin

val config = session.config

// Check whether the user's device supports the Depth API.
val isDepthSupported = session.isDepthModeSupported(Config.DepthMode.AUTOMATIC)
val isGeospatialSupported = session.isGeospatialModeSupported(Config.GeospatialMode.ENABLED)
if (isDepthSupported && isGeospatialSupported) {
  // These three settings are needed to use Geospatial Depth.
  config.depthMode = Config.DepthMode.AUTOMATIC
  config.geospatialMode = Config.GeospatialMode.ENABLED
  config.streetscapeGeometryMode = Config.StreetscapeGeometryMode.ENABLED
}
session.configure(config)

Once Geospatial Depth is enabled, depth images can be accessed through existing API calls as described in the depth developer guide.

Java

// Retrieve the depth image for the current frame, if available.
Image depthImage = null;
try {
  depthImage = frame.acquireDepthImage16Bits();
  // Use the depth image here.
} catch (NotYetAvailableException e) {
  // This means that depth data is not available yet.
  // Depth data will not be available if there are no tracked
  // feature points. This can happen when there is no motion, or when the
  // camera loses its ability to track objects in the surrounding
  // environment.
} finally {
  if (depthImage != null) {
    depthImage.close();
  }
}

Kotlin

// Retrieve the depth image for the current frame, if available.
try {
  frame.acquireDepthImage16Bits().use { depthImage ->
    // Use the depth image here.
  }
} catch (e: NotYetAvailableException) {
  // This means that depth data is not available yet.
  // Depth data will not be available if there are no tracked
  // feature points. This can happen when there is no motion, or when the
  // camera loses its ability to track objects in the surrounding
  // environment.
}

What’s next