LiteRT in Google Play services CompiledModel C++ API

The CompiledModel API is the recommended way to run machine learning (ML) models from C++ with the LiteRT in Google Play services runtime. It lets you run your model on CPU or GPU through a single API, without bundling the LiteRT runtime in your APK.

For general information about the CompiledModel C++ API, see the CompiledModel C++ API guide.

Note: If your app uses the older C or C++ Interpreter API with Google Play services, see the Interpreter C and C++ APIs guide.

Prerequisites: CMake 3.20 or later.

1. Update your build configuration

(1) Add the Google Play services LiteRT dependency to your app's build.gradle file:

implementation "com.google.android.gms:play-services-litert:17.0.0"
// Optional: For Kotlin coroutines Task.await() support
implementation "org.jetbrains.kotlinx:kotlinx-coroutines-play-services:1.10.2"

(2) Enable the Prefab feature to access the native SDK from your CMake script by updating the android block of your module's build.gradle file:

buildFeatures {
  prefab = true
}

(3) Update your app's CMakeLists.txt to configure C++20, Abseil, FlatBuffers, and the litert_gms_cc_api target imported from the AAR:

cmake_minimum_required(VERSION 3.20)
project("litert-cc-sample")

# The LiteRT C++ SDK requires C++20.
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

include(FetchContent)
FetchContent_Declare(
        flatbuffers
        URL https://github.com/google/flatbuffers/archive/refs/tags/v25.9.23.zip
)
FetchContent_Declare(
        absl
        URL https://github.com/abseil/abseil-cpp/archive/refs/tags/20260817.0.zip
)
set(FLATBUFFERS_BUILD_FLATC OFF)
set(FLATBUFFERS_BUILD_TESTS OFF)
set(FLATBUFFERS_INSTALL OFF)
set(ABSL_ENABLE_INSTALL ON)
FetchContent_MakeAvailable(flatbuffers)
FetchContent_MakeAvailable(absl)

add_library(flatbuffers::flatbuffers ALIAS flatbuffers)

find_package(litert_compiledmodel_gms_client REQUIRED CONFIG)
get_target_property(litert_cc_sdk_dir
        litert_compiledmodel_gms_client::litert_compiledmodel_gms_client
        INTERFACE_INCLUDE_DIRECTORIES)
add_subdirectory(${litert_cc_sdk_dir} litert_cc_sdk_build)

add_library(litert-jni SHARED
        litert_jni.cc)

target_link_libraries(litert-jni
        litert_gms_cc_api::litert_gms_cc_api
        absl::span
        android
        log)

2. Initialize the LiteRT native handle

Before calling the LiteRT C++ API from native code, obtain a CompiledModelNativeHandle in your Kotlin or Java code using LiteRt.initializeNative(context), and pass the CompiledModelNativeHandle instance to your JNI method:

Kotlin

import com.google.android.gms.litert.LiteRt
import com.google.android.gms.litert.client.CompiledModelNativeHandle
import kotlinx.coroutines.tasks.await
...
val handle: CompiledModelNativeHandle =
    LiteRt.initializeNative(context).await()

Java

import com.google.android.gms.litert.LiteRt;
import com.google.android.gms.litert.client.CompiledModelNativeHandle;
import com.google.android.gms.tasks.Tasks;
...
// Callback pattern is recommended for Java to avoid blocking the thread.
// Tasks.await() is used here for simplicity.
CompiledModelNativeHandle handle =
    Tasks.await(LiteRt.initializeNative(context));

3. Create a CompiledModel and run inference in C++ (CPU)

In your C++ JNI code, include "litert/abi/litert.h" along with the standard LiteRT C++ headers. Call GmsLiteRtInitializeEnvironment(env, handle) (passing the CompiledModelNativeHandle jobject) to initialize a litert::Environment backed by Google Play services, then create a litert::CompiledModel with litert::HwAccelerators::kCpu and pre-allocate input and output litert::TensorBuffers.

Important: Create the litert::Environment, litert::CompiledModel, and litert::TensorBuffers once off the main thread and reuse them across inference calls (such as camera frames). Model compilation is expensive—do not recreate the Environment or CompiledModel on every Run() call.

#include <jni.h>
#include <vector>

#include "absl/types/span.h"
#include "litert/abi/litert.h"
#include "litert/cc/litert_buffer_ref.h"
#include "litert/cc/litert_common.h"
#include "litert/cc/litert_compiled_model.h"
#include "litert/cc/litert_environment.h"
#include "litert/cc/litert_expected.h"
#include "litert/cc/litert_tensor_buffer.h"
...
// 1. Initialize the Environment and CompiledModel once off the main thread
litert::Expected<litert::Environment> litert_env =
    GmsLiteRtInitializeEnvironment(env, j_native_initialization_handle);
if (!litert_env) {
  return;
}

litert::Expected<litert::CompiledModel> compiled_model =
    litert::CompiledModel::Create(*litert_env, model_buffer,
                                  litert::HwAccelerators::kCpu);
if (!compiled_model) {
  return;
}

litert::Expected<std::vector<litert::TensorBuffer>> input_buffers =
    compiled_model->CreateInputBuffers();
litert::Expected<std::vector<litert::TensorBuffer>> output_buffers =
    compiled_model->CreateOutputBuffers();
if (!input_buffers || !output_buffers) {
  return;
}

// 2. Run inference (reuse the compiled model and buffers across calls)
(*input_buffers)[0].Write<float>(absl::MakeConstSpan(input_data));
compiled_model->Run(*input_buffers, *output_buffers);
std::vector<float> output_data(output_size);
(*output_buffers)[0].Read<float>(absl::MakeSpan(output_data));

4. Hardware acceleration

With the CompiledModel C++ API in Google Play services, you can accelerate model inference across hardware accelerators:

• CPU acceleration: Supported by default (litert::HwAccelerators::kCpu).
• GPU acceleration: Supported on devices with a compatible GPU (litert::HwAccelerators::kGpu).
• NPU acceleration: Available in standalone LiteRT; support in Google Play services is coming soon.

GPU acceleration

LiteRT's GPU accelerator is built on the open-source ML Drift engine.

To use GPU acceleration with the CompiledModel C++ API in Google Play services, first enable GPU accelerator support in LiteRtInitializationOptions when requesting the CompiledModelNativeHandle with LiteRt.initializeNative(context, initOptions) in Kotlin or Java:

Kotlin

import com.google.android.gms.litert.LiteRt
import com.google.android.gms.litert.LiteRtDsl.initializationOptions
import com.google.android.gms.litert.client.CompiledModelNativeHandle
import kotlinx.coroutines.tasks.await
...
val initOptions = initializationOptions {
  setEnableGpu(true)
}
val handle: CompiledModelNativeHandle =
    LiteRt.initializeNative(context, initOptions).await()

Java

import com.google.android.gms.litert.LiteRt;
import com.google.android.gms.litert.client.CompiledModelNativeHandle;
import com.google.android.gms.litert.client.LiteRtInitializationOptions;
import com.google.android.gms.tasks.Tasks;
...
LiteRtInitializationOptions initOptions =
    LiteRtInitializationOptions.builder()
        .setEnableGpu(true)
        .build();
// Callback pattern is recommended for Java to avoid blocking the thread.
// Tasks.await() is used here for simplicity.
CompiledModelNativeHandle handle =
    Tasks.await(LiteRt.initializeNative(context, initOptions));

Then, in your C++ code, configure litert::Options with litert::HwAccelerators::kGpu | litert::HwAccelerators::kCpu to automatically fall back to CPU if GPU acceleration is unavailable on the device (or litert::HwAccelerators::kGpu to restrict execution to GPU only) and pass it to litert::CompiledModel::Create():

#include "litert/cc/litert_common.h"
#include "litert/cc/litert_compiled_model.h"
#include "litert/cc/litert_options.h"
...
// Initialize the LiteRT Environment from the handle with GPU support enabled
litert::Expected<litert::Environment> litert_env =
    GmsLiteRtInitializeEnvironment(env, j_native_initialization_handle);
if (!litert_env) {
  return 0;
}

// Configure GPU acceleration with CPU fallback
litert::Expected<litert::Options> compilation_options =
    litert::Options::Create();
if (!compilation_options) {
  return 0;
}
compilation_options->SetHardwareAccelerators(
    litert::HwAccelerators::kGpu | litert::HwAccelerators::kCpu);

litert::Expected<litert::CompiledModel> compiled_model =
    litert::CompiledModel::Create(*litert_env, model_buffer,
                                  *compilation_options);