Image generation task guide

The MediaPipe Image Generator is a lightweight text-to-image generation task. The task supports text-to-image generation with pre-trained diffusion models on mobile GPU. It also supports conditioned text-to-image generation through a lightweight plugin model.

Install and run the Image Generator demo app

These instructions require downloading an open source model from an external repository. You are responsible for confirming that you can comply with any applicable license terms. These open sourced models are not a Google service.

Step 1. Identify a compatible checkpoint file. The model checkpoint must exactly match the runwayml/stable-diffusion-v1-5 EMA-only model format. To quickly ensure you have the correct file, check that the model file is 4.27GB.

Here are a couple of example diffusion checkpoints that are verified to work with this demo app:

Step 2. Use the image_generator_converter to convert the checkpoint into on-device format.

Install the necessary dependencies

$ pip install torch typing_extensions numpy Pillow requests pytorch_lightning absl-py

Run the convert.py script.

$ python3 convert.py --ckpt_path <ckpt_path> --output_path <output_path>

Step 3. Push the content of the <output_path> folder to the Android device.

$ adb shell rm -r /data/local/tmp/image_generator/ # Remove any previously loaded weights
$ adb shell mkdir -p /data/local/tmp/image_generator/
$ adb push <output_path>/. /data/local/tmp/image_generator/bins

Step 4. Install the Android demo app: pre-built apk.

$ adb install imagegenerator.apk