LiteRT August Release

The following are the release notes for the LiteRT August release.

LiteRT Runtime (v2.2.0)

API and Platform updates

Rust Integration - Published google-ai-edge-litert crate on crates.io enabling C++ SDK capabilities in Rust
ABI Compatibility - Added ABI version tracking to key modules (libLiteRT.so, GPU accelerator, Dispatch, Compiler) facilitating smoother upgrades across 2.2.x binary versions.

API Updates

LiteRT.js - LiteRT Tensor API support and features for recurrent KV cache swapping; and JIT/tracing support for custom tensor graphs.
New Data Type - Added built-in support for F8E4M3FN and F8E5M2 tensor elements

New HW support

Performance and Model optimizations

LLM throughput improvements - added ROPE composite Ops for GPU, resulting in ~5% faster decode performance
Multiple optimizations for memory efficiency and GPU execution

Validation / Testing Infrastructure

Faster test registration
New multi-criteria validation framework (ConformanceSpec) for mixed-precision verification.

Enables PyTorch model conversion to LiteRT with specialized pipelines for vision and generative models.

Vision & Generative Pipelines

Gemma 4 & ASR Support - Architecture export support for Gemma 4 and ASR models in the export_hf pipeline.
Dynamic Context Length - Added --enable_gpu_dynamic_prefill and --enable_gpu_dynamic_cache flags, with automated stop token handling.
GPU Optimizations (Experimental) - Support for optimized GPU attention custom op, FP16/FP32 mixed precision, and Jinja template transpilation.

NPU & Acceleration

NPU Export Pipeline (Experimental) - Added end-to-end NPU export colab, sampling executor, post-training quantization, and LiteRT-LM NPU compiler.

Converter Legalizations & Fixes

Ops & Types - Added 1D transposed convolution, FFT lowering, and torch.uint8 support.
Bug Fixes - Fixed InlineConstsContext cache collisions for tensors sharing data_ptr with distinct offsets, and resolved BPE de-tokenization issues.

Manages model compression, low-bit weight quantization, and quantization transformations.

Algorithms & Recipes

New Quantization Modes - Added GPTQ support, 2-bit quantization with buffer packing, blockwise scale recovery recipes (4-bit and 8-bit), and int4 Hadamard transforms.
Calibration & Validation - Added profiler-based XNNPACK calibration, CalibrationInterpreter API, multi-metric validation, and progress reporting.

Tooling & Integration

CLI & Formats - Introduced the aeq.py CLI and built-in .litertlm container support for end-to-end LLM quantization.

Performance & Memory Optimizations

Memory Efficiency - Replaced RAM serialization with disk-backed memory mapping (mmap_utils), zero-copy in-place ops, chunked quantization, and immediate memory reclamation to prevent OOM on large LLMs.

Packaging & Fixes

Updated packaging to pyproject.toml and uv; fixed non-constant weight casting crashes and input double-quantization.

v0.16.0 / v0.16.1

C API Prebuilts - Added the first versioned C API shared library prebuilts for all supported platforms, enabling direct integration into applications and language bindings without building shared libraries.
Experimental YNNPACK Delegate - Added the experimental YNNPACK delegate, enabled for Linux ARM64 builds in the LiteRT-LM CLI and Python API.
Bug Fixes (v0.16.1) - Fixed a Windows JVM crash with updated build flags for Kotlin and Maven on Windows.

Apple Foundation Framework (v0.15.0)

Apple FM Adapter - Shipped a new Apple Foundation framework adapter directly integrating Apple's backend with the LiteRT-LM runtime for fast text and multimodal (vision and audio) execution featuring Gemma 4 models, thanks to john-rocky's contributions.
Concurrency & Stability - Resolved re-entrancy issues on concurrent engine initializations.

LiteRT-LM CLI (v0.15.0)

Centralized config.json System - Introduced configuration file support for the LiteRT-LM CLI to define global defaults and per-model settings (such as specialized hardware backends, thinking budgets, and context window sizes) without needing to pass verbose command-line flags.
Strict Parameter Precedence - Execution parameters follow a clear hierarchy: Runtime arguments > Model-specific config > Global defaults > Model metadata > Engine fallbacks. For more details, see the CLI Configuration guide.

JavaScript API & Web Application (LiteRT-LM.js) (v0.15.0)

Gemma 4 Web Support - Gemma 4 12B, Gemma 4 26B A4B, and Gemma 4 31B are officially supported on web.
Memory Efficiency - Added a new use_autosized_ringbuffers option that improves long context memory efficiency.
Tool Calling Capabilities - Added AutoToolChat, which automatically runs tool calls that the model emits.

Versions & Dependencies - Upgraded to CLI version 0.2.0, updated LiteRT stable dependencies to 2.2.0, bumped the QAIRT SDK, and changed the default binary version to nightly.
Features & Fixes - Switched from OAuth to Application Default Credentials (ADC) login, refactored Android file pushing logic, and resolved issues with desktop run outputs, tensor shapes, and QNN extraction.
Documentation & Tutorials - Updated documentation and added links to the LiteRT CLI 101 workshop codelab.
Testing & CI - Added a dry_run flag for release workflows and updated test configurations for speculative decoding and static quantization.