Overview

  • Coral NPU provides easy-to-use, standards-based developer tools supporting multiple ML frameworks, leveraging the existing RISC-V ecosystem for simplified development.

  • It is an ideal hardware accelerator for PyTorch, JAX, or TensorFlow Lite, and its open-source architecture allows for customization to facilitate software development.

  • Software development with Coral NPU is primarily focused on deploying ML models on small devices and building MCU micro-applications for edge computing using ML kernels.

  • Coral NPU offers several software development advantages including compiler support simplicity, embedded processing capabilities when programmed in C, and resources for ML micro-kernels.

Coral NPU offers standards-based, easy-to-use developer tools that support multiple ML frameworks. The tools leverage existing compiler and library support from the broader RISC-V ecosystem, simplifying development.

Coral NPU is the ideal hardware accelerator for running code developed with PyTorch, JAX, or TensorFlow Lite. Because Coral NPU is an open-source architecture, it can also be customized to facilitate software development.

Guidance for software development falls into two main categories:

  • Deploying ML models on small devices such as AR glasses and other wearables that have ambient environment sensing devices.
  • Building MCU micro-applications for edge applications, using ML kernels.

Coral NPU's software development advantages

Development feature Advantages
Coral NPU is designed to be simple for compilers to support even with ML domain specialization.
  • Compute requirements in ML models are dominated by matrix multiplication.
  • Outer-product multiplication circuit enables N*M*K MACs/cycle compute, with typical values of N=8, M=8, K=4, N*M*K=256 (MAC = multiply-accumulate). This can be parameterized to suit the application domain.
  • Convolution, etc., can be expressed using matmuls as sub-operations.
  • In a fixed-function accelerator, only matmuls are performed, with limited control flow.
When programmed in C, Coral NPU:
  • Is an embedded processor capable of single-context compute offload
  • Allows for C++ intrinsics for bootstrapping workloads
For ML micro-kernels, Coral NPU has:
  • Math library containing a combination of hand-tuned and templated Operators (ML)
  • TFLM implementations of common ML kernels
  • IREE support via baseline RISC-V scalar with Operator kernels
For North Star:
  • Graph refactoring to extract optimal memory layouts/paddings
  • IREE polyhedral compilation and LLVM auto-vectorization
  • Push-button / high-leverage tooling

Getting started

To get started developing software for the Coral NPU, read the following topics: