Migrate to LiteRT from TensorFlow Lite

TensorFlow Lite is now LiteRT. LiteRT is the primary runtime for Google's high-performance on-device AI. While existing TensorFlow Lite packages will remain functional, all future feature updates and performance enhancements will be exclusive to LiteRT. Because LiteRT fully supports the TensorFlow Lite Interpreter API, migrating requires only a package name update—no logic changes are necessary.

Two API generations: v1 and v2

LiteRT ships in two API generations, which map to different migration paths:

  • LiteRT v1 is the classic TensorFlow Lite Interpreter API. LiteRT is a drop-in replacement, so moving from TensorFlow Lite requires only a package name change.
  • LiteRT v2 adds the CompiledModel API, which provides turn-key hardware accelerator selection, the latest GPU acceleration through ML Drift, NPU support (Google Tensor, Qualcomm, and MediaTek), zero-copy I/O buffers, and asynchronous execution. See On-device Inference with LiteRT for details.

Choose your migration path

  • Keep the Interpreter API (fastest migration): Swap your TensorFlow Lite package for the matching LiteRT package. No code changes are required. Follow the platform guides in the following sections.
  • Adopt the v2 CompiledModel API (recommended for new work): Update your inference code to unlock the latest GPU and NPU acceleration. See Upgrade to the LiteRT v2 API.

For package name changes, see the following migration guides for specific platforms.

Migrate on Android

To migrate an Android application using Tensorflow Lite, replace the dependency from org.tensorflow:tensorflow-lite to com.google.ai.edge.litert. The LiteRT Maven repository includes the following packages:

You can make this change in your build.gradle dependencies:

dependencies {
  ...
  implementation 'com.google.ai.edge.litert:litert:2.1.0'
}

Play Services

LiteRT in the Play Services runtime continues to use the play-services-tflite dependency. If your app is already using the Play Services runtime with TensorFlow Lite, you don't need to make any code changes.

To use LiteRT in Play Services, add the following to your build.gradle dependencies:

dependencies {
...
    // LiteRT dependencies for Google Play services
    implementation 'com.google.android.gms:play-services-tflite-java:16.5.0'
    // Optional: include LiteRT Support Library
    implementation 'com.google.android.gms:play-services-tflite-support:16.5.0'
...
}

Migrate with Python

To migrate Python code using Tensorflow Lite, replace the PIP package from tflite-runtime to ai-edge-litert.

Install LiteRT with PIP:

$ python3 -m pip install ai-edge-litert

Import LiteRT with the following:

from ai_edge_litert.interpreter import Interpreter
interpreter = Interpreter(model_path=args.model_file)

Other Libraries

The Swift and Objective-C SDKs for iOS, C++ SDK, Task Library, and Model Maker library continues to exist in the TensorFlow Lite packages. Applications using these SDKs or libraries shouldn't migrate to LiteRT.

Upgrade to the LiteRT v2 API

Swapping packages keeps your app running with the classic Interpreter API. To take advantage of the latest GPU and NPU acceleration, upgrade your inference code to the LiteRT v2 CompiledModel API.

The v2 API replaces the single Interpreter object with a few composable objects:

  • litert::Environment: the global runtime context.
  • litert::Model: a hardware-agnostic representation of your model.
  • litert::CompiledModel: an executable optimized for a specific hardware target (CPU, GPU, or NPU).
  • litert::TensorBuffer: an abstraction over tensor memory (CPU, GPU, or NPU) that enables zero-copy I/O.

API mapping

Task TensorFlow Lite (Interpreter) LiteRT v2 (CompiledModel)
Load a model FlatBufferModel::BuildFromFile Model::CreateFromFile / Model::CreateFromBuffer
Select hardware Interpreter::ModifyGraphWithDelegate Accelerator option to CompiledModel::Create
Allocate I/O Interpreter::AllocateTensors CompiledModel::CreateInputBuffers / CreateOutputBuffers
Fill inputs memcpy into typed_input_tensor TensorBuffer::Write (or wrap an existing hardware buffer)
Run inference Interpreter::Invoke CompiledModel::Run (or RunAsync)
Read outputs typed_output_tensor Read from the output TensorBuffer

Example: CPU inference

The following TensorFlow Lite code runs a model on the CPU:

// Load the model and initialize the runtime.
std::unique_ptr<tflite::FlatBufferModel> model =
    tflite::FlatBufferModel::BuildFromFile("mymodel.tflite");
tflite::ops::builtin::BuiltinOpResolver resolver;
tflite::InterpreterBuilder builder(*model, resolver);
std::unique_ptr<tflite::Interpreter> interpreter;
builder(&interpreter);

// Allocate tensors and fill the inputs.
interpreter->AllocateTensors();
float* input = interpreter->typed_input_tensor<float>(0);
std::memcpy(input, kTestInput, kTestInputSize * sizeof(float));

// Run inference and read the output.
interpreter->Invoke();
float* output = interpreter->typed_output_tensor<float>(0);

The equivalent LiteRT v2 code is:

// Load the model and initialize the runtime.
auto env = litert::Environment::Create({});
auto model = litert::Model::CreateFromFile("mymodel.tflite");
auto compiled_model = litert::CompiledModel::Create(*env, *model);

// Preallocate I/O buffers so they can be reused across inference runs.
auto input_buffers = compiled_model->CreateInputBuffers();
auto output_buffers = compiled_model->CreateOutputBuffers();

// Fill the model inputs.
input_buffers[0].Write<float>(
    absl::MakeConstSpan(kTestInput, kTestInputSize));

// Run inference.
compiled_model->Run(input_buffers, output_buffers);

// Access the model output.
auto lock_and_addr =
    litert::TensorBufferScopedLock::Create<const float>(output_buffers[0]);
const float* output = lock_and_addr->second;

To run on GPU or NPU, pass an accelerator option (for example, kLiteRtHwAcceleratorGpu or kLiteRtHwAcceleratorNpu) to CompiledModel::Create. The rest of the code is unchanged because TensorBuffer encapsulates device memory.

Next steps

Samples