Render your AR app using Vulkan on Android SDK (Kotlin/Java)

  • ARCore can provide the camera image as an Android hardware buffer for efficient Vulkan rendering when TextureUpdateMode.EXPOSE_HARDWARE_BUFFER is enabled.

  • This hardware buffer can be accessed using Frame.getHardwareBuffer() and bound to a Vulkan VkImage for rendering.

  • Vulkan rendering with ARCore requires Android API level 27 or higher and device support for the VK_ANDROID_external_memory_android_hardware_buffer extension.

  • The hello_ar_vulkan_c sample app demonstrates how to implement Vulkan rendering with ARCore.

  • To ensure your app is only available on compatible devices, declare the Vulkan feature requirement in your app's manifest.

When the Config.TextureUpdateMode is set to TextureUpdateMode.EXPOSE_HARDWARE_BUFFER, ARCore will provide an Android hardware buffer when Session.update() is called. This hardware buffer can be bound to a Vulkan VkImage.

View the sample application

Vulkan rendering support is demonstrated in the hello_ar_vulkan_c sample app.

Enable the hardware buffer output mode

The configured Config.TextureUpdateMode determines how ARCore will update the camera texture. When it is set to TextureUpdateMode.EXPOSE_HARDWARE_BUFFER, ARCore will provide the camera image through a HardwareBuffer.

Configure the session to use TextureUpdateMode.EXPOSE_HARDWARE_BUFFER:

Java

Config config = session.getConfig();
config.setTextureUpdateMode(Config.TextureUpdateMode.EXPOSE_HARDWARE_BUFFER);
session.configure(config);

Kotlin

session.configure(
  session.config.apply { textureUpdateMode = Config.TextureUpdateMode.EXPOSE_HARDWARE_BUFFER }
)

Obtain the hardware buffer

When TextureUpdateMode.EXPOSE_HARDWARE_BUFFER is enabled, use Frame.getHardwareBuffer() to get the hardware buffer:

Java

try {
  HardwareBuffer buffer = frame.getHardwareBuffer();
  // Use the buffer object in your rendering.
} catch (NotYetAvailableException e) {
  // The hardware buffer is not ready yet.
}

Kotlin

try {
  val buffer = frame.hardwareBuffer
  // Use the buffer object in your rendering.
} catch (e: NotYetAvailableException) {
  // The hardware buffer is not ready yet.
}

Use the hardware buffer during Vulkan rendering

See vulkan_handler.cc for an example of how to render an AR application using Vulkan.

Supported devices

Vulkan rendering support is only available on Android API levels 27 and above. Additionally, the device must support the VK_ANDROID_external_memory_android_hardware_buffer extension.

Require Vulkan in your app's manifest

Google Play uses <uses-feature> declared in your app manifest to filter your app from devices that don't meet its hardware and software feature requirements. Devices using Vulkan 1.0 might not support the required extension, but devices compatible with Vulkan 1.1 must have the required extension starting in Android 10 (API level 29).