Augmented Images Developer Guide for Android NDK

Learn how to use Augmented Images in your own apps.

Create an image database

Create an ArAugmentedImageDatabase to store reference images. There are two ways:

  • Create an empty database

    ArAugmentedImageDatabase* ar_augmented_image_database = nullptr;
    ArAugmentedImageDatabase_create(ar_session_, &ar_augmented_image_database);
    
  • Read from an image database file. Refer to the AugmentedImage C sample app for util::LoadFileFromAssetManager.

    std::string database_buffer;
    util::LoadFileFromAssetManager(asset_manager_, "sample_database.imgdb",
                               &database_buffer);
    uint8_t* raw_buffer = reinterpret_cast<uint8_t*>(&database_buffer.front());
    
    ArAugmentedImageDatabase* ar_augmented_image_database = nullptr;
    const ArStatus status = ArAugmentedImageDatabase_deserialize(
        ar_session_, raw_buffer, database_buffer.size(),
        &ar_augmented_image_database);
    

    Database files can be created either with the Image Tool or by calling ArAugmentedImageDatabase_serialize().

Add images to database

This step is optional if your desired reference images have already been loaded from the database file. To add an image at runtime, call ArAugmentedImageDatabase_addImage() as shown below. Refer to the AugmentedImage C sample app for functions in the util namespace.

int32_t width, height, stride, index;
uint8_t* image_pixel_buffer = nullptr;
constexpr const char kSampleImageName[] = "default.jpg";
bool load_image_result = util::LoadImageFromAssetManager(
    kSampleImageName, &width, &height, &stride, &image_pixel_buffer);

uint8_t* grayscale_buffer = nullptr;
util::ConvertRgbaToGrayscale(image_pixel_buffer, width, height, stride,
                             &grayscale_buffer);

int32_t grayscale_stride = stride / 4;
const ArStatus status = ArAugmentedImageDatabase_addImage(
    ar_session_, ar_augmented_image_database, kSampleImageName,
    grayscale_buffer, width, height, grayscale_stride, &index);

// If the physical size of the image is known, you can instead use
//     ArStatus ArAugmentedImageDatabase_addImageWithPhysicalSize
// This will improve the initial detection speed. ARCore will still actively
// estimate the physical size of the image as it is viewed from multiple
// viewpoints.

delete[] image_pixel_buffer;
delete[] grayscale_buffer;

The integer and name values are used later to identify which reference image was detected.

Enable image tracking

Configure your ARCore session to track images by registering the image database:

ArConfig_setAugmentedImageDatabase(ar_session_, ar_config,
                               ar_augmented_image_database);
const ArStatus status = ArSession_configure(ar_session_, ar_config);

During the session, ARCore uses feature points from the camera image to match against those in the image database.

To get the matched images, poll for updated ArAugmentedImages in your frame update loop.

// Update loop, in onDrawFrame
ArTrackableList* updated_image_list = nullptr;
ArTrackableList_create(ar_session_, &updated_image_list);
ArFrame_getUpdatedTrackables(
    ar_session_, ar_frame_, AR_TRACKABLE_AUGMENTED_IMAGE, updated_image_list);

int32_t image_list_size;
ArTrackableList_getSize(ar_session_, updated_image_list, &image_list_size);

for (int i = 0; i < image_list_size; ++i) {
  ArTrackable* ar_trackable = nullptr;
  ArTrackableList_acquireItem(ar_session_, updated_image_list, i,
                              &ar_trackable);
  ArAugmentedImage* image = ArAsAugmentedImage(ar_trackable);

  ArTrackingState tracking_state;
  ArTrackable_getTrackingState(ar_session_, ar_trackable, &tracking_state);

  int image_index;
  ArAugmentedImage_getIndex(ar_session_, image, &image_index);

  if (tracking_state == AR_TRACKING_STATE_TRACKING) {
    util::ScopedArPose scopedArPose(ar_session_);
    ArAugmentedImage_getCenterPose(ar_session_, image,
                                   scopedArPose.GetArPose());

    ArAnchor* image_anchor = nullptr;
    const ArStatus status = ArTrackable_acquireNewAnchor(
        ar_session_, ar_trackable, scopedArPose.GetArPose(),
        &image_anchor);

    // For example, you can now render content at the image anchor, choosing
    // content based on the image index (or name).
  }
}

Send feedback about...