Compilation FAQs

Troubleshoot and resolve common errors to ensure your AI models compile for Google Tensor.

  • Q: Why does the LiteRT AOT compiler crash with an INTERNAL error at compiler_plugin.cc when compiling?

    A: The Tensor SDK is optimized by default for the newer Tensor_G5 architecture. When compiling larger, complex models for any Google Tensor versions other than Tensor_G5, the compiler encounters memory sharding and truncation constraints that cause a SIGABRT crash, even if all operations are technically supported.

  • Q: How do I fix the compilation crashes due to memory constraints?

    A: To successfully compile and run your model on a Google Tensor TPU other than Tensor_G5, you can use this workaround:

    Fix the Compiler Crash with compilation flags

    When running the AOT compiler, you must explicitly configure it to adapt to the memory constraints of the Google Tensor version you're using. Add these flags to your compilation target:

    • google_tensor_truncation_type="half"
    • google_tensor_sharding_intensity="maximum"

    See Compile Models with Google Tensor for more information.

  • Q: Which version of the ai-edge-litert should I use with the AOT compilation toolchain?

    A: For the AOT compilation process, you can choose from the following approaches:

    • Use a nightly LiteRT package (use a particular version)
     ai-edge-litert-nightly==2.2.0.dev20260530
    • Use a stable LiteRT package (use an environment variable)
     ai-edge-litert==2.1.5

  • Q: Where can I find the on-device NPU runtime libraries for Google Tensor?

    A: For Tensor G5, the recommended approach is to build the dispatch library using LiteRT 2.1.5 release. You can also download the precompiled runtime files from LiteRT Release 2.1.5.

    Refer to the following compatibility matrix to find the correct source code to build:

    Compatibility information

    Target SoC LiteRT version Android version
    Stable Nightly
    Tensor G5 ai-edge-litert==2.1.5 (May 2026) ai-edge-litert-nightly==2.2.0.dev20260530 (May 2026) 26Q1

  • Q: How do I build a Google Tensor dispatch library?

    A: If you specifically want to build the Google Tensor dispatch library from source, use the following steps:

    1. Set up the environment Run the helper script from the workspace directory to build the litert_build_env Docker image and perform the initial verification build:

      ./docker_build/build_with_docker.sh

      Q: Why does the Docker build fail with java.net.MalformedURLException: no protocol during the Android sdkmanager step?

      A: This happens when you build on a non-corporate machine without a proxy. The hermetic_build.Dockerfile and build_with_docker.sh scripts explicitly forward an empty HTTP_PROXY string, which crashes the Android sdkmanager. To fix this, remove or comment out the ENV HTTP_PROXY=${HTTP_PROXY} block in the Docker file and avoid passing empty --build-args in the shell script.

    2. Run the custom Bazel build in a preserved container: Once the Docker image is ready, run a new container manually to compile the Google Tensor dispatch library:

      docker run --name litert_build_container \
  --user $(id -u):$(id -g) \
  -e HOME=/litert_build \
  -e USER=$(id -un) \
  -v $(pwd):/litert_build \
  litert_build_env \
  bazel build -c opt --config=android_arm64 \    litert/vendors/google_tensor/dispatch:dispatch_api.so

    3. Copy the shared library to your host machine's root directory

      docker cp litert_build_container:/litert_build/bazel-bin/litert/vendors/google_tensor/dispatch/libLiteRtDispatch_GoogleTensor.so .

    4. Clean up the container

      docker rm -f litert_build_container

  • Q: Where can I find the on-device NPU runtime libraries for Google Tensor?

    A: For Tensor G5, the recommended approach is to build the dispatch library using LiteRT 2.1.5 release. You can also download the precompiled runtime files from the LiteRT Release 2.1.5.

  • Q: How do I fix an invalid input model error (example, "File efficientnet_b7.tflite does not exist")?

    A: This occurs when the compiler cannot find your model. Verify that your input model is located in the expected folder by running ls ${folder_name}.

  • Q: What should I do if compilation fails due to an invalid input file?

    A: This error triggers if your input file is not a .tflite file, or if the .tflite file is invalid or corrupted. Ensure you are using a properly formatted, valid .tflite file and retry the compilation.

  • Q: How do I resolve the error stating "The compiler does not support layers with dynamic shapes"?

    A: The compiler requires static shapes. To fix this, you must convert any layers in your model that use dynamic shapes into static shapes. Once updated, retry the compilation.

  • Q: Why does the compiler say it "does not support the specified device"?

    A: This happens if the target device name provided is either incorrect or unsupported. Double-check the exact device name you specified and retry the compilation using a supported device.

  • Q: How do I fix a "missing required flag" error?

    A: The compiler requires certain mandatory parameters to run. Review your compilation commands, identify the missing flag, ensure it is properly specified, and retry the compilation.

  • Q: What if the compilation fails with an unknown error, or the error message is vague?

    A: You may encounter unexpected or unhelpful error messages due to the system's limited error-reporting support. To bypass these issues, try running the compilation again using different combinations of compilation options, or attempt to optimize your model prior to compiling.

  • Q: Can the compilation process affect my model's performance?

    A: Yes, compilation may introduce numerical issues that can affect the overall accuracy of certain models.