LiteRT is Google's on-device inference engine for deploying ML and GenAI
models on edge platforms, with tools to convert, optimize, and run them.

The latest LiteRT 2.x release introduces the `CompiledModel` API,
a modern runtime interface designed to maximize hardware acceleration. While the
`Interpreter` API (formerly TensorFlow Lite) remains available for backward
compatibility, the `CompiledModel` API is the recommended choice for
developers seeking state-of-the-art performance in on-device AI applications.
[1 Prerequisites](https://developers.google.com/edge/litert/overview#prerequisites) [2 Install packages](https://developers.google.com/edge/litert/overview#install-litert) [3 Obtain model](https://developers.google.com/edge/litert/overview#choose-model) [4 Run \& verify](https://developers.google.com/edge/litert/overview#integrate-model) [5 Accelerate](https://developers.google.com/edge/litert/overview#choose-backend) Docs: [Android](https://developers.google.com/edge/litert/android) [Web](https://developers.google.com/edge/litert/web) [LiteRT-LM](https://developers.google.com/edge/litert-lm/overview) [Migration](https://developers.google.com/edge/litert/migration)

> [!IMPORTANT]
> **Looking to deploy large language models (LLMs) or generative AI?**   
> For conversational streaming, dynamic KV caching, and tool calling with open-weight models (such as **Gemma 4** , **Llama 3.2** , and **Qwen 3.5** ), jump directly to the [LiteRT-LM Getting Started Guide](https://developers.google.com/edge/litert-lm/overview). Per-language code samples live in the [LiteRT-LM API guide](https://developers.google.com/edge/litert-lm/api_overview); install with `pip install litert-lm-api`.  
> To deploy perception models (vision, audio, embeddings) using the `CompiledModel` API, continue with the 5 steps in this guide.

## Development workflow

LiteRT runs inference entirely on-device on Android, iOS, Web, IoT, and on
desktop/laptop. On every platform, the workflow follows the same five steps,
with links to detailed instructions.

### 1. Prerequisites

Before developing with LiteRT, verify that your workstation environment meets
the following minimum toolchain and SDK requirements:

| Platform / Toolchain | Minimum Requirements | Recommended Tooling | Target Accelerators |
|---|---|---|---|
| **Python** | Python 3.9--3.12 | `pip` 23.0+ or `uv` | CPU, GPU, Qualcomm NPU |
| **Android (Kotlin / C++)** | Android Studio Ladybug+ (2024.2.1+) Min SDK 24+ (Android 7.0+) | Android NDK r26a+ (for C++) | CPU, GPU (OpenCL/OpenGL), NPU |
| **Web (JavaScript / TS)** | Modern browser with WebGPU support (Chrome 113+, Edge 113+, Safari 17.4+) | Node.js 18+ (bundler / npm) | WebGPU, WASM (CPU), WebNN (NPU) |
| **C++ (Native \& Embedded)** | CMake 3.19+ C++17 compiler (GCC 10+, Clang 12+, MSVC 2019+) | Ninja build system | CPU (XNNPACK), GPU (OpenCL/Vulkan), NPU |
| **iOS (Swift)** | Xcode 15+ Deployment target iOS 15.0+ / macOS 12.0+ | CocoaPods 1.12+ or Swift Package Manager | CPU, GPU (Metal), Core ML |

### 2. Install LiteRT packages

Install LiteRT SDK packages across your target platforms:

### Python

    # Platform: Linux, macOS, Windows (Python 3.9+)
    # Core LiteRT (Vision, Audio, CompiledModel)
    pip install ai-edge-litert

    # LiteRT-LM (On-device LLMs & Generative AI)
    pip install litert-lm-api

Command-line tools install separately --- see the **CLI Tools** tab.

### Android (Kotlin)

    // Platform: Android (API 24+) | In app/build.gradle.kts
    dependencies {
        // Core LiteRT runtime with CompiledModel API.
        implementation("com.google.ai.edge.litert:litert:2.3.0")
        // Optional: include for GPU acceleration
        implementation("com.google.ai.edge.litert:litert-gpu:2.3.0")

        // LiteRT-LM runtime for on-device LLMs & Gemma
        implementation("com.google.ai.edge.litertlm:litertlm-android:latest.release")
    }

### iOS (Swift)

    // Platform: iOS 15+, macOS 12+

    // LiteRT-LM ships a versioned Swift package. In Package.swift:
    .package(url: "https://github.com/google-ai-edge/LiteRT-LM", from: "0.16.0")

    // In target dependencies:
    .product(name: "LiteRTLM", package: "LiteRT-LM")

    // LiteRT Runtime iOS distribution is still pre-release:
    // the LiteRTSwift CocoaPod publishes nightly builds only, and
    // Swift Package Manager support is in progress.
    pod 'LiteRTSwift'   // nightly builds

> [!NOTE]
> **Choosing an iOS path.** LiteRT-LM is the production-ready option on Apple platforms today. For perception models on iOS, track the [C++ SDK](https://developers.google.com/edge/litert/next/cpp_sdk) or build the runtime from source until the versioned Swift distribution lands.

### Web (JavaScript)

    # Platform: Web (Browsers & Node.js)
    # Core LiteRT.js runtime for vision & audio models
    npm install @litertjs/core

    # LiteRT-LM for on-device LLM token streaming
    npm install @litert-lm/core

### C++

    # Platform: Linux, Android NDK, macOS, Windows
    # In CMakeLists.txt (CMake 3.19+)
    # Link prebuilt LiteRT C++ SDK (see /edge/litert/next/cpp_sdk)
    target_link_libraries(${PROJECT_NAME} PRIVATE
        litert_cc_api
    )

### CLI Tools

    # Platform: Linux, macOS, Windows
    # LiteRT Model Tools CLI (conversion, quantization, benchmarking)
    pip install litert-cli-nightly

    # LiteRT-LM CLI (on-device LLM prompt testing & serving)
    pip install litert-lm

### 3. Obtain a model

A LiteRT model is represented in an efficient portable format known as
[FlatBuffers](https://google.github.io/flatbuffers/), which uses the `.tflite`
file extension.

#### Quickstart: Download a sample model

To test the code examples in [Step 4](https://developers.google.com/edge/litert/overview#integrate-model), download a
pre-trained MobileNetV2 image classification model (`.tflite`):

    curl -L -o mobilenet_v2.tflite \
      https://storage.googleapis.com/tfweb/litertjs_demo_models/mobilenetv2/torchvision_mobilenet_v2.tflite

This provides a ready-to-run model (ImageNet 1,000-class classification,
input shape `1x3x224x224` `float32`) for immediate verification.

> [!NOTE]
> **Bringing your own model?** Run steps 4 and 5 with this sample first --- the same buffer and accelerator code applies to any `.tflite` file. Then swap in your own from [Use your own model](https://developers.google.com/edge/litert/overview#explore-models), which covers pre-trained catalogs and PyTorch / TensorFlow / JAX conversion.

### 4. Run inference and verify results

LiteRT lets you run ML models entirely on-device with high performance
across Android, iOS, Web, desktop, and IoT platforms.

> [!NOTE]
> **Tip --- Connecting Sensor Streams to `CompiledModel` Input Buffers:**
>
> - **Vision (Camera to Tensor):** When processing camera frames using Android CameraX or iOS AVFoundation, copy decoded pixel buffers (or use zero-copy `AHardwareBuffer` using `TensorBuffer`) directly into `inputBuffers`. See the complete camera loop in the [Image Segmentation Kotlin App](https://github.com/google-ai-edge/litert-samples/tree/main/samples/litert/image_segmentation/kotlin_npu).
> - **Audio (Microphone to Tensor):** For continuous streaming audio, capture 16 kHz PCM chunks using `AudioRecord`, write them into an audio ring buffer, and pass windowed tensors to `inputBuffers`. See the complete microphone ingestion pipeline in the [Speech Recognition Kotlin App](https://github.com/google-ai-edge/litert-samples/tree/main/samples/litert/speech_recognition).

The following tabbed examples demonstrate the complete lifecycle---loading the `mobilenet_v2.tflite` model downloaded in Step 3, allocating buffers, running inference, and inspecting output classifications:

### Python

    # Platform: Linux, macOS, Windows • Accelerators: CPU / GPU / NPU
    import numpy as np
    from ai_edge_litert.compiled_model import CompiledModel, HardwareAccelerator

    # 1. Load model and compile for target accelerator (CPU / GPU / NPU)
    model = CompiledModel.from_file("mobilenet_v2.tflite", HardwareAccelerator.CPU)

    # 2. Preallocate input and output memory buffers
    input_buffers = model.create_input_buffers(0)
    output_buffers = model.create_output_buffers(0)

    # 3. Populate input buffer with test tensor (1x3x224x224 float32 for MobileNetV2)
    input_data = np.zeros((1, 3, 224, 224), dtype=np.float32)
    input_buffers[0].write(input_data)

    # 4. Execute on-device inference
    model.run_by_index(0, input_buffers, output_buffers)

    # 5. Read output tensor and inspect classification logits (1,000 ImageNet classes)
    output = output_buffers[0].read(1000, np.float32)
    top_class = np.argmax(output)
    print("LiteRT Python inference completed successfully!")
    print(f"Output shape: {output.shape}")
    print(f"Top predicted class index: {top_class}, score: {output[top_class]:.4f}")

### Android (Kotlin)

    // Platform: Android (API 24+) • Accelerators: Qualcomm / MediaTek NPU, GPU, CPU
    import android.util.Log
    import com.google.ai.edge.litert.Accelerator
    import com.google.ai.edge.litert.CompiledModel

    // 1. Initialize and compile model (bundle mobilenet_v2.tflite into app/src/main/assets/)
    // For files outside assets, use CompiledModel.create(file.absolutePath, options)
    CompiledModel.create(
        context.assets,
        "mobilenet_v2.tflite",
        // Target accelerator: CPU, GPU, or NPU
        CompiledModel.Options(Accelerator.CPU)
    ).use { compiledModel ->
        // 2. Preallocate input buffer with test image tensor (1x3x224x224 float32)
        val inputBuffers = compiledModel.createInputBuffers()
        val inputData = FloatArray(1 * 3 * 224 * 224) { 0.0f }
        inputBuffers[0].writeFloat(inputData)

        // 3. Run inference (automatically creates and returns output buffers)
        val outputBuffers = compiledModel.run(inputBuffers)

        // 4. Read and verify classification logits (1,000 ImageNet classes)
        val output = outputBuffers[0].readFloat()
        val topClass = output.indices.maxByOrNull { output[it] } ?: -1
        Log.d("LiteRT", "Inference completed successfully!")
        Log.d("LiteRT", "Output logits size: ${output.size} classes")
        Log.d("LiteRT", "Top predicted class: $topClass (score: ${output[topClass]})")
    }

### iOS (Swift)

    // Platform: iOS 15+, macOS 12+ • Accelerators: Apple Silicon GPU / Neural Engine / CPU
    import Foundation
    import LiteRT

    // 1. Initialize environment (defaults to CPU acceleration)
    let env = try Environment()

    // 2. Compile model from app bundle (add mobilenet_v2.tflite to Xcode bundle resources)
    guard let modelPath = Bundle.main.path(forResource: "mobilenet_v2", ofType: "tflite") else {
      fatalError("Model file not found in bundle")
    }
    let compiledModel = try CompiledModel(filePath: modelPath, environment: env)

    // 3. Allocate and populate input buffers (1x3x224x224 float32)
    let inputBuffers = try compiledModel.createInputBuffers()
    let outputBuffers = try compiledModel.createOutputBuffers()
    let inputData = [Float](repeating: 0.0, count: 1 * 3 * 224 * 224)
    try inputBuffers[0].write(inputData)

    // 4. Execute inference
    try compiledModel.run(inputs: inputBuffers, outputs: outputBuffers)

    // 5. Read and inspect output classification results (1,000 classes)
    let output: [Float] = try outputBuffers[0].read()
    print("LiteRT Swift inference completed successfully!")
    print("Output logits count: \(output.count) classes")
    if let maxIdx = output.indices.max(by: { output[$0] < output[$1] }) {
      print("Top predicted class: \(maxIdx) (score: \(output[maxIdx]))")
    }

### Web (JavaScript)

    // Platform: Web (Browsers & Node.js) • Accelerators: WebGPU / WebAssembly
    import {loadLiteRt, loadAndCompile, Tensor} from '@litertjs/core';

    // 1. Initialize LiteRT WebAssembly runtime
    await loadLiteRt('https://cdn.jsdelivr.net/npm/@litertjs/core/wasm/');

    // 2. Load and compile public model targeting WebGPU acceleration
    // (WebGPU requires HTTPS / secure context; fallback to CPU if unavailable)
    const model = await loadAndCompile(
      'https://storage.googleapis.com/tfweb/litertjs_demo_models/mobilenetv2/torchvision_mobilenet_v2.tflite',
      { accelerator: 'webgpu' }
    );

    // 3. Prepare input tensor (1x3x224x224 float32)
    const inputData = new Float32Array(1 * 3 * 224 * 224);
    const inputTensor = new Tensor(inputData, [1, 3, 224, 224]);

    // 4. Run inference with input tensors
    const outputs = await model.run([inputTensor]);

    // 5. Read 1,000-class output logits and inspect results
    const output = await outputs[0].data();
    let topClass = 0;
    for (let i = 1; i < output.length; i++) {
      if (output[i] > output[topClass]) topClass = i;
    }
    console.log('LiteRT.js inference completed successfully!');
    console.log('Output logits length:', output.length);
    console.log(`Top predicted class: ${topClass} (score: ${output[topClass].toFixed(4)})`);

    // 6. Clean up model and tensor resources to prevent WebGPU/WASM memory leaks
    inputTensor.delete();
    outputs.forEach(tensor => tensor.delete());
    model.delete();

### C++

    // Platform: Linux, Android NDK, macOS, Windows • Accelerators: CPU / GPU / NPU
    #include <algorithm>
    #include <iostream>
    #include <vector>
    #include "absl/types/span.h"
    #include "litert/cc/litert_compiled_model.h"
    #include "litert/cc/litert_environment.h"
    #include "litert/cc/litert_macros.h"

    litert::Expected<void> RunInference() {
      // 1. Initialize LiteRT runtime environment
      LITERT_ASSIGN_OR_RETURN(auto env, litert::Environment::Create({}));

      // 2. Load model and compile for CPU acceleration
      LITERT_ASSIGN_OR_RETURN(
          auto compiled_model,
          litert::CompiledModel::Create(env, "mobilenet_v2.tflite",
                                        litert::HwAccelerators::kCpu));

      // 3. Preallocate input and output buffers
      LITERT_ASSIGN_OR_RETURN(auto input_buffers,
                              compiled_model.CreateInputBuffers());
      LITERT_ASSIGN_OR_RETURN(auto output_buffers,
                              compiled_model.CreateOutputBuffers());

      // 4. Fill input buffer with test image tensor (1x3x224x224 float32)
      std::vector<float> input_data(1 * 3 * 224 * 224, 0.0f);
      LITERT_RETURN_IF_ERROR(
          input_buffers[0].Write<float>(absl::MakeConstSpan(input_data)));

      // 5. Invoke model inference
      LITERT_RETURN_IF_ERROR(
          compiled_model.Run(input_buffers, output_buffers));

      // 6. Read and verify classification output logits (1,000 classes)
      std::vector<float> output_data(1000);
      LITERT_RETURN_IF_ERROR(
          output_buffers[0].Read<float>(absl::MakeSpan(output_data)));

      auto max_it = std::max_element(output_data.begin(), output_data.end());
      int top_class = std::distance(output_data.begin(), max_it);
      std::cout << "LiteRT C++ inference completed successfully!" << std::endl;
      std::cout << "Output logits size: " << output_data.size() << " classes" << std::endl;
      std::cout << "Top predicted class: " << top_class << " (score: " << *max_it << ")" << std::endl;

      return {};
    }

    int main() {
      if (auto status = RunInference(); !status) {
        std::cerr << "Inference failed: " << status.Error() << std::endl;
        return 1;
      }
      return 0;
    }

### 5. Choose a hardware backend

The Step 4 examples all target the CPU. To accelerate the same model, change
only the accelerator you pass when compiling --- `HardwareAccelerator.GPU` in
Python, `Accelerator.GPU` in Kotlin, `Options.setHardwareAccelerators([.gpu])`
in Swift, `litert::HwAccelerators::kGpu` in C++, or `{accelerator: 'webgpu'}` on
the Web. Nothing else changes: the `CompiledModel`
API handles delegate setup and falls back automatically when an op is
unsupported. See the [on-device inference guide](https://developers.google.com/edge/litert/inference) for more details.

|   | Android | iOS / macOS | Web | Linux | Windows | IoT |
|---|---|---|---|---|---|---|
| CPU | XNNPACK | XNNPACK | XNNPACK | XNNPACK | XNNPACK | XNNPACK |
| GPU | OpenGL OpenCL | Metal WebGPU | WebGPU | WebGPU OpenCL | WebGPU OpenCL | WebGPU |
| NPU | Samsung Exynos Google Tensor MediaTek Qualcomm | - | - | Qualcomm | Intel Qualcomm | Qualcomm |

> [!NOTE]
> **Next steps:** For complete project configuration, dependency coordinates (Gradle, pip, npm, CMake), and end-to-end sample apps, see the platform quickstarts:
>
> - [Run on Android (Kotlin / C++)](https://developers.google.com/edge/litert/android)
> - [Run on Web (LiteRT.js)](https://developers.google.com/edge/litert/web)
> - [Run on Desktop (Python)](https://developers.google.com/edge/litert/next/python)
> - [Run on iOS \& macOS (Swift / C++)](https://developers.google.com/edge/litert/ios/quickstart)
> - [Run on Linux, Windows \& Embedded (C++)](https://developers.google.com/edge/litert/next/cpp)

## Use your own model

Once the quickstart runs, swap in a model of your own. Start from a pre-trained
catalog, or convert a model you already have:

- **Pre-trained models:** Ready-to-use models optimized for on-device execution.

  | Domain | Workload | Source \& Model Catalog |
  |---|---|---|
  | **Vision** | Segmentation, Classification, Detection | [Hugging Face Vision Models](https://huggingface.co/collections/litert-community/image-classification-models) [Kaggle DeepLabV3](https://www.kaggle.com/models/tensorflow/deeplabv3) |
  | **Audio** | Speech Recognition (ASR), Sound ID | [Hugging Face ASR Models](https://huggingface.co/collections/litert-community/asr) [Kaggle Audio Models](https://www.kaggle.com/models?framework=tfLite&query=audio) |
  | **GenAI** | LLMs, Multimodal, Embeddings | [LiteRT Hugging Face Hub](https://huggingface.co/litert-community/models) [Gemma Family Collection](https://huggingface.co/collections/litert-community/gemma-family) |

- **Convert custom models:** If you have custom-trained models in PyTorch, TensorFlow, or JAX, convert them directly into `.tflite`:

  | Framework | Supported Domains | Conversion Tool | Interactive Tutorial |
  |---|---|---|---|
  | **PyTorch** | TorchVision, TorchAudio, Hugging Face | [litert-torch](https://developers.google.com/edge/litert/conversion/pytorch/overview) | [NPU AOT Compilation Colab](https://github.com/google-ai-edge/litert-samples/blob/main/samples/litert/colab/LiteRT_AOT_Compilation_Tutorial.ipynb) |
  | **TensorFlow** | Keras, TF Hub | [TensorFlow Lite Converter](https://developers.google.com/edge/litert/conversion/tensorflow/convert_tf) | [TF Conversion Guide](https://developers.google.com/edge/litert/conversion/tensorflow/convert_tf) |
  | **JAX** | Custom JAX modules | [JAX Converter](https://developers.google.com/edge/litert/conversion/jax/overview) | [JAX Conversion Guide](https://developers.google.com/edge/litert/conversion/jax/overview) |

> [!NOTE]
> **Advanced: Model Quantization**
>
> AI Edge Quantizer quantizes converted LiteRT models. Use it to get the best performance from resource-intensive models, such as GenAI models. See more details in the [AI Edge Quantizer documentation](https://github.com/google-ai-edge/ai-edge-quantizer).

## Build a reference app

Explore end-to-end reference applications built with the `CompiledModel` API
across vision, audio, and generative AI:

| Modality | Task | Hardware Acceleration | Sample Implementation |
|---|---|---|---|
| **Vision** | **Image Segmentation** | CPU, GPU, NPU (JIT \& AOT) | • [Kotlin Android App (CameraX)](https://github.com/google-ai-edge/litert-samples/tree/main/samples/litert/image_segmentation/kotlin_npu) • [C++ App (Async Execution)](https://github.com/google-ai-edge/litert-samples/tree/main/samples/litert/image_segmentation/c%2B%2B_segmentation) |
| **Audio** | **Speech Recognition (ASR)** | CPU, NPU | • [Kotlin Android App (Real-time Mic Stream)](https://github.com/google-ai-edge/litert-samples/tree/main/samples/litert/speech_recognition) |
| **GenAI** | **Semantic Similarity** | CPU, GPU, NPU | • [EmbeddingGemma C++ Demo](https://github.com/google-ai-edge/litert-samples/tree/main/samples/litert/semantic_similarity/build_from_source) |

## Advanced: author custom models and ops

When pre-trained or converted models do not meet your performance or
architecture requirements, the **Generative API** (`litert-torch`) provides
PyTorch building blocks for composing or fine-tuning Transformer models (such
as [Gemma](https://github.com/google-ai-edge/litert-torch/blob/main/litert_torch/generative/examples/gemma) and [TinyLlama](https://github.com/google-ai-edge/litert-torch/blob/main/litert_torch/generative/examples/tiny_llama)) with mobile-friendly KV-caching and
built-in support for `.tflite` conversion. See the [Generative API Guide](https://developers.google.com/edge/litert/conversion/pytorch/genai).

## Additional documentation and support

- **Supported operators:** the `CompiledModel` API supports a wide range of
  operators across hardware accelerators. See the [operators guide](https://developers.google.com/edge/litert/inference#operators).

- [**LiteRT-Samples GitHub Repo**](https://github.com/google-ai-edge/litert-samples/tree/main/samples/litert) for more LiteRT sample apps.

- **For existing users of TensorFlow Lite** , see the [migration guide](https://developers.google.com/edge/litert/migration).

- [**LiteRT Tools page**](https://developers.google.com/edge/litert/models/measurement) for performance profiling, error reporting, and more.

- Blogs and announcements: Visit the
  [LiteRT news and blogs page](https://developers.google.com/edge/litert/new_blogs/news_blogs) for product
  announcements, silicon partner deep dives, and community articles, including
  [Mastering Edge AI on Raspberry Pi with LiteRT and Gemma](https://developers.googleblog.com/mastering-edge-ai-on-raspberry-pi-with-litert-and-gemma/).