C Support Library Tutorial

This page describes how to use features of the Tango Support Library in C. The Tango Support Library C reference pages describe the available functions in more detail.

Installation instructions

Download

If you followed our instructions in Getting Started with the Tango C API, then you already have the Support Library on your machine. The library lives in the tango_support_api folder in the github repository. Otherwise, you should download the Tango C Support Library from our Downloads page.

In your Android.mk file, add the following lines to import both the client and Support Library:

$(call import-add-path,$(PROJECT_ROOT))
$(call import-module,tango_client_api)
$(call import-module,tango_support_api)

Support Library tutorial

The Support Library contains functions to help you manage and use data from the Tango APIs. Current features of the library include support for managing callback data and objects and plane fitting to depth data.

Initializing the Support Library

Before you can use the Support Library, you must initialize it:

Managing callback data

The Support Library helps users manage point cloud and image buffers efficiently in their applications without performing unneeded copies or blocking the underlying Tango service by performing long-running operations in callbacks.

You can use helper functions to allocate and copy depth data stored in the TangoPointCloud data structures. (The same maximum point cloud size must be consistently applied across user allocations.)

Initialize the Support Library:

TangoSupport_initialize(TangoService_getPoseAtTime, TangoService_getCameraIntrinsics)

Create a Point Cloud Manager:

if (point_cloud_manager_ == nullptr) {
  int32_t max_point_cloud_elements;
  ret = TangoConfig_getInt32(tango_config_, "max_point_cloud_elements",
                             &max_point_cloud_elements);
  if (ret != TANGO_SUCCESS) {
    LOGE("Failed to query maximum number of point cloud elements.");
    std::exit(EXIT_SUCCESS);
  }

  ret = TangoSupport_createPointCloudManager(max_point_cloud_elements,
                                             &point_cloud_manager_);
  if (ret != TANGO_SUCCESS) {
    std::exit(EXIT_SUCCESS);
  }
}

When there is a new point cloud, update the Point Cloud Manager:

void onPointCloudAvailableRouter(void* context,
                                 const TangoPointCloud* point_cloud) {
  tango_point_cloud::PointCloudApp* app =
      static_cast<tango_point_cloud::PointCloudApp*>(context);
  app->onPointCloudAvailable(point_cloud);
}

namespace tango_point_cloud {
void PointCloudApp::onPointCloudAvailable(const TangoPointCloud* point_cloud) {
  TangoSupport_updatePointCloud(point_cloud_manager_, point_cloud);
}

Whenever you want to use the point cloud data, fetch it from the Point Cloud Manager:

TangoPointCloud* point_cloud = nullptr;
TangoSupport_getLatestPointCloud(point_cloud_manager_, &point_cloud);
if (point_cloud == nullptr) {
  return;
}

When you're done, free the Point Cloud Manager:

if (point_cloud_manager_ != nullptr) {
  TangoSupport_freePointCloudManager(point_cloud_manager_);
  point_cloud_manager_ = nullptr;
}

Plane fitting

A common use case in augmented reality (AR) applications is placing a virtual object on real world surfaces like walls, floors, tabletops, etc. This uses a combination of Depth Perception and Motion Tracking. The TangoSupport_fitPlaneModelNearPoint helper function fits a plane model to the depth data near a specified screen location, and returns the plane equation and the intersection point with the camera ray.

Initialize the Support Library:

TangoSupport_initialize(TangoService_getPoseAtTime, TangoService_getCameraIntrinsics)

This example fits a plane to the depth data near the center of the camera image:

double out_intersection_point[3];
double out_plane_equation[4];

double uv[2] = {0.5, 0.5};

if(TangoSupport_fitPlaneModelNearPoint(point_cloud, point_cloud_translation,
                                       point_cloud_orientation,
                                       uv, intersection_point, display_rotation,
                                       color_camera_translation,
                                       color_camera_orientation,
                                       out_intersection_point,
                                       out_plane_equation) != TANGO_SUCCESS) {
  LOGE(“Error fitting plane model”);
}

The inputs color_camera_rotation and color_camera_orientation are the transform from the desired output frame to the color camera frame when the uv coordinates are chosen. The inputs point_cloud_translation and point_cloud_orientation are the transform from the desired output frame to the point cloud at the time the point cloud was acquired. The returned intersection and plane model use this desired coordinate system. The cpp_plane_fitting_example on GitHub demonstrates how to generate the inputs and transform the outputs into world coordinates.

Depth data with interpolation

There are sometimes gaps in the depth data coming from the sensor. The Support Library provides a few interpolation algorithms to try to fill these gaps when querying for depth at a particular point in the image.

Nearest neighbor interpolation is fast, but tends to have more artifacts such as blocky areas or sudden changes in depth.

Bilateral interpolation is much slower than nearest neighbor, but creates smooth transitions between depth readings and takes the color image into account. When interpolating for the whole image, you can use an approximate verison of bilateral interpolation, which is faster but less accurate.

Nearest neighbor interpolation - single point

This example attempts to find a depth reading near the center of the camera image:

const float uv_coordinates[2] = {0.5f, 0.5f};
float out_point[3];
int is_valid_point;

if (TangoSupport_getDepthAtPointNearestNeighbor(point_cloud,
                                                point_cloud_translation,
                                                point_cloud_orientation,
                                                uv_coordinates,
                                                display_rotation,
                                                color_camera_translation,
                                                color_camera_orientation,
                                                out_point) != TANGO_SUCCESS) {
  LOGE("Error finding depth at the point.");
}
Bilateral interpolation - single point

This example attempts to find a depth reading near the center of the camera image:

TangoSupportDepthInterpolator* depth_interpolator;
TangoSupport_createDepthInterpolator(TangoSupportDepthInterpolator** depth_interpolator);

const float uv_coordinates[2] = {0.5f, 0.5f};
float out_point[3];
int is_valid_point;

if (TangoSupport_getDepthAtPointBilateral(depth_interpolator, point_cloud,
                                          point_cloud_translation, point_cloud_orientation,
                                          depth_interpolator, uv_coordinates,
                                          display_rotation, color_camera_translation,
                                          color_camera_orientation, out_point) != TANGO_SUCCESS) {
  LOGE("Error finding depth at the point.");
}

TangoSupport_freeDepthInterpolator(depth_interpolator);

Nearest neighbor interpolation - whole image

This example attempts to interpolate depth across the entire image using nearest neighbor interpolation.

TangoSupportDepthInterpolator* depth_interpolator;
TangoSupport_createDepthInterpolator(&camera_intrinsics, &depth_interpolator);

TangoSupportDepthBuffer depth_buffer;
TangoSupport_initializeDepthBuffer(kImageWidth, kImageHeight, &depth_buffer);

if (TangoSupport_upsampleImageNearestNeighbor(depth_interpolator,
                                              point_cloud,
                                              color_camera_T_point_cloud,
                                              &depth_buffer) != TANGO_SUCCESS) {
  LOGE("Error upsampling the image.");
}

TangoSupport_freeDepthBuffer(&depth_buffer);
TangoSupport_freeDepthInterpolator(depth_interpolator);

Bilateral interpolation - whole image

This example attempts to interpolate depth across the entire image using bilateral interpolation. Doing bilateral interpolation across the entire image can be slow. Setting approximate to a non-zero value would use an approximate version of bilateral interpolation, which is faster but less accurate.

TangoSupportDepthInterpolator* depth_interpolator;
TangoSupport_createDepthInterpolator(&camera_intrinsics, &depth_interpolator);

TangoSupportDepthBuffer depth_buffer;
TangoSupport_initializeDepthBuffer(kImageWidth, kImageHeight, &depth_buffer);

int approximate = 0;
if (TangoSupport_upsampleImageBilateral(depth_interpolator, approximate,
                                        point_cloud, image_buffer,
                                        color_camera_T_point_cloud, &depth_buffer)
                                        != TANGO_SUCCESS) {
  LOGE("Error upsampling the image.");
}

TangoSupport_freeDepthBuffer(&depth_buffer);
TangoSupport_freeDepthInterpolator(depth_interpolator);

Enviar comentarios sobre…