Depth Perception

How it works

Depth Perception gives an application the ability to understand the distance to objects in the real world. To implement Depth Perception, manufacturers of Tango devices can choose among common depth technologies, including Structured Light, Time of Flight, and Stereo. Structured Light and Time of Flight require the use of an infrared (IR) projector and IR sensor; Stereo does not.

Depth Perception can be useful in a number of ways:

  • A game might want to detect when the user is approaching a wall or other object in the area and have that be part of the gameplay.

  • By combining Depth Perception with Motion Tracking, the device can measure distances between points in an area that aren't in the same frame.

  • Depth data can be associated with the color image data to look up the color for a point cloud for texturing or meshing.

Usability tips

Current devices are designed to work best indoors at moderate distances (0.5 to 4 meters). This configuration gives good depth at a distance while balancing power requirements for IR illumination and depth processing. It may not be ideal for close-range object scanning or gesture detection.

For a device that relies on viewing IR light using its camera, there are some situations where accurate Depth Perception is difficult. Areas lit with light sources high in IR like sunlight or incandescent bulbs, or objects that do not reflect IR light, cannot be scanned well.

Point clouds

The Tango APIs provide a function to get depth data in the form of a point cloud. This format gives (x, y, z) coordinates for as many points in the scene as are possible to calculate. Each dimension is a floating point value recording the position of each point in meters in the coordinate frame of the depth-sensing camera.

For more information, see the TangoPointCloud struct API reference.