Google MLIR compiler for Coral NPU

The initial version 1.0 of the open-source compiler is released on GitHub in 2026 at https://github.com/google-coral/coralnpu-compiler. The compiler transforms high-level ML models into Coral NPU binaries.

Using progressive lowering across MLIR dialects, the compiler translates models from JAX and LiteRT (formerly TensorFlow Lite) into optimized intermediate representations, partitions workloads between the host CPU and the Coral NPU core, and links embedded RISC-V executables packaged for execution on hardware or simulators.

The MLIR compiler, runtime, and simulation stack are designed to ship as a turnkey package for Coral NPU. You can use these tools to compile and run reference models, from MobileNet V2 and YOLOv8n to compact generative models such as Gemma 3 (270M), directly on the MPACT-CoralNPU simulator, Verilator, Spike, or FPGA targets.

Key features:

  • An open-source and fully accessible toolchain, intended to serve as a reference for Coral NPU adopters
  • Simple developer experience: Python scriptable, using PJRT (Pluggable Just-In-Time Runtime) plug-in
  • Low-maintenance extensibility: adopters can design with abstractions that allow tailoring of the compiler to specific NPU derivatives without significant upstream maintenance
  • Performance profiling: enable easy profiling and provide warnings for degraded performance
  • Build/find tools for model export into MLIR
  • JAX frontend via PJRT plugin and AOT via StableHLO path
  • JAX/StableHLO → Linalg frontend lowering pipeline (non-optimized)
  • LiteRT (TFLite) via TOSA path
  • TFLite/TOSA → Linalg frontend lowering pipeline (non-optimized)
  • Global graph optimizations (CSE, DCE, constant folding) and fine-tuning for Coral NPU matrix execution engine
  • Auto-vectorization and polyhedral compilation optimizations for matrix workloads
  • Basic tiling and fusion for vector and matrix engines
  • Heterogeneous dispatch partitioning (Coral NPU versus host CPU)
  • Host CPU fallback execution for unsupported operations
  • Coral NPU compiler target is based on the 32-bit RISC-V CPU supported by LLVM
  • LLVM RISC-V backend with Coral NPU RVV vector and VME matrix unit support
  • Coral NPU IREE target plug-in and linker integration
  • Coral NPU HAL driver for simulators

PJRT is an open interface for device runtime and compilers that simplifies ML hardware and framework integration. An implementation of the PJRT API is called a PJRT plug-in, which is usually a Python package for seamless ML model developer experience.

Coral compiler pipeline

CoralMLIR compiler

* The llvm-cpu compiler target uses these options:

--iree-hal-target-device=local
--iree-hal-local-target-device-backends=llvm-cpu
--iree-llvmcpu-target-triple=aarch64-unknown-none

HAL runtime architecture

The compiler includes a custom IREE Hardware Abstraction Layer (HAL) named coralnpu:

  • Driver and device: Manages the hardware enumeration and the active connection to the NPU core.
  • Allocator: Handles AXI memory-mapped regions to ensure zero-copy shared access between the host CPU and Coral NPU.
  • Executable: Bundles RISCV-32 machine code with metadata for entry points.
  • Simulator support: Integrated with the Mpact-CoralNPU simulator, allowing end-to-end execution on x86-64 hosts without physical development boards.

Extensibility for adopters

The Coral NPU compiler is structured so as to allow silicon partners and hardware integrators to adapt the toolchain to custom SoC configurations without maintaining a forked compiler stack:

  • Custom dialects: Adopters can add low-level assembly or abstract dialects for proprietary matrix units while leveraging the base Coral NPU infrastructure.
  • Flexible HAL: The coralnpu driver is implemented independently of standard IREE "local-sync" to allow proprietary memory or signal extensions.