<br />

The MediaPipe Universal Embedder task combines the features of individual audio, text, and
image embedders into a single unified service, while also adding support for
video embeddings. It runs on the on-device LiteRT-LM runtime to produce
high-performance multimodal embeddings without any server round-trips. You can
use the Universal Embedder to vectorize text, images, and audio samples, or run
multimodal embedding extraction on lists of composite contents to capture
semantic meaning across modalities.


<iframe class="mp-demo-frame mp-demo-frame--studio" aria-label="Universal Embedder interactive Web SDK demo" loading="lazy" data-src="https://developers.devsite.google/frame/edge/mediapipe/demos/embed/universal_embedder.jshtml" allow="camera *; microphone *"></iframe>

[Open in Web Demos](https://google-ai-edge.github.io/mediapipe-samples-web/#/retrieval/universal_embedder)[Edit in StackBlitz](https://stackblitz.com/github/google-ai-edge/mediapipe-samples-web?file=src/tasks/universal-embedder.ts)

<br />

## Quickstart

Install the SDK for your platform, initialize the task with a model, and run
inference.

### Web

npm`npm install @mediapipe/tasks-retrieval`[Web demo](https://google-ai-edge.github.io/mediapipe-samples-web/#/retrieval/universal_embedder)·[Edit in StackBlitz](https://stackblitz.com/github/google-ai-edge/mediapipe-samples-web?file=src/tasks/universal-embedder.ts)·[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_web.md.txt)

```typescript
import { FilesetResolver, UniversalEmbedder } from "@mediapipe/tasks-retrieval";

// 1. Create the task
const MODEL_URL = "https://huggingface.co/litert-community/embeddinggemma-2-text-vision-440m-litert-lm/resolve/main/embeddinggemma-2-text-vision-440m.litertlm";

const retrieval = await FilesetResolver.forRetrievalTasks(
  "https://cdn.jsdelivr.net/npm/@mediapipe/tasks-retrieval@latest/wasm"
);
const universalEmbedder = await UniversalEmbedder.createFromOptions(retrieval, {
  baseOptions: { modelAssetPath: MODEL_URL },
});

// 2. Run inference across modalities and compare cosine similarity
const textResult = await universalEmbedder.embedText("A photo of a monument");
const imageBytes = new Uint8Array(await (await fetch("monument.jpg")).arrayBuffer());
const imageResult = await universalEmbedder.embedImage(imageBytes);
const similarity = UniversalEmbedder.cosineSimilarity(
  textResult.embeddings[0],
  imageResult.embeddings[0]
);
```

### Android (Kotlin)

Gradle`implementation("com.google.mediapipe:tasks-retrieval:latest.release")`[Sample code](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/universal_embedder/android)·[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_android.md.txt)

```kotlin
import com.google.mediapipe.tasks.core.BaseOptions
import com.google.mediapipe.tasks.core.Delegate
import com.google.mediapipe.tasks.retrieval.universalembedder.UniversalEmbedder
import com.google.mediapipe.tasks.retrieval.universalembedder.UniversalEmbedderOptions

// 1. Create the task
val baseOptions = BaseOptions.builder()
    .setModelAssetPath("embeddinggemma-2-text-vision-440m.litertlm")
    .build()
val options = UniversalEmbedderOptions.builder()
    .setBaseOptions(baseOptions)
    .setTextDelegate(Delegate.GPU)
    .setVisionDelegate(Delegate.GPU)
    .setL2Normalize(true)
    .build()

val universalEmbedder = UniversalEmbedder.createFromOptions(context, options)

// 2. Run inference across modalities and compare cosine similarity
val textResult = universalEmbedder.embedText("A photo of a monument")
val imageResult = universalEmbedder.embedImage(mpImage)
val similarity = UniversalEmbedder.cosineSimilarity(
    textResult.embeddings().first(),
    imageResult.embeddings().first()
)
```

### Android (Java)

Gradle`implementation 'com.google.mediapipe:tasks-retrieval:latest.release'`[Sample code](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/universal_embedder/android)·[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_android.md.txt)

```java
import com.google.mediapipe.tasks.components.containers.EmbeddingResult;
import com.google.mediapipe.tasks.core.BaseOptions;
import com.google.mediapipe.tasks.core.Delegate;
import com.google.mediapipe.tasks.retrieval.universalembedder.UniversalEmbedder;
import com.google.mediapipe.tasks.retrieval.universalembedder.UniversalEmbedderOptions;

// 1. Create the task
BaseOptions baseOptions = BaseOptions.builder()
    .setModelAssetPath("embeddinggemma-2-text-vision-440m.litertlm")
    .build();
UniversalEmbedderOptions options = UniversalEmbedderOptions.builder()
    .setBaseOptions(baseOptions)
    .setTextDelegate(Delegate.GPU)
    .setVisionDelegate(Delegate.GPU)
    .setL2Normalize(true)
    .build();

UniversalEmbedder universalEmbedder = UniversalEmbedder.createFromOptions(context, options);

// 2. Run inference across modalities and compare cosine similarity
EmbeddingResult textResult = universalEmbedder.embedText("A photo of a monument");
EmbeddingResult imageResult = universalEmbedder.embedImage(mpImage);
double similarity = UniversalEmbedder.cosineSimilarity(
    textResult.embeddings().get(0),
    imageResult.embeddings().get(0)
);
```

### iOS

SwiftPM`https://github.com/google-ai-edge/mediapipe → MediaPipeTasksRetrieval`[Package.swift](https://developers.google.com/edge/mediapipe/solutions/setup_ios.md.txt#package_swift)·[Sample code](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/universal_embedder/ios)·[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_ios.md.txt)

```swift
import Foundation
import MediaPipeTasksRetrieval

// 1. Create the task
guard let modelPath = Bundle.main.path(
    forResource: "embeddinggemma-2-text-vision-440m",
    ofType: "litertlm"
) else { return }

let options = UniversalEmbedderOptions()
options.baseOptions.modelAssetPath = modelPath
options.l2Normalize = true

let universalEmbedder = try UniversalEmbedder(options: options)

// 2. Run inference across modalities and compare cosine similarity
let textResult = try universalEmbedder.embed(text: "A photo of a monument")
let imageResult = try universalEmbedder.embed(image: mpImage)
let similarity = try UniversalEmbedder.cosineSimilarity(
    embedding1: textResult.embeddings[0],
    embedding2: imageResult.embeddings[0]
)
```

### Python

pip`pip install mediapipe`[Sample code](https://colab.research.google.com/github/google-ai-edge/mediapipe-samples/blob/main/examples/universal_embedder/python/universal_embedder.ipynb)·[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_python.md.txt)

```python
import mediapipe as mp

BaseOptions = mp.tasks.BaseOptions
ImagePart = mp.tasks.retrieval.ImagePart
TextPart = mp.tasks.retrieval.TextPart
UniversalEmbedder = mp.tasks.retrieval.UniversalEmbedder
UniversalEmbedderOptions = mp.tasks.retrieval.UniversalEmbedderOptions

# 1. Create the task
options = UniversalEmbedderOptions(
    base_options=BaseOptions(model_asset_path="embeddinggemma-2-text-vision-440m.litertlm"),
    l2_normalize=True,
)

# 2. Run inference across modalities and compare cosine similarity
with UniversalEmbedder.create_from_options(options) as universal_embedder:
    text_result = universal_embedder.embed_text("A photo of a monument")
    with open("monument.jpg", "rb") as f:
        image_bytes = f.read()
    image_result = universal_embedder.embed_image(image_bytes)
    composite_result = universal_embedder.embed_content([
        TextPart(text="A photo of a monument"),
        ImagePart(image_bytes=image_bytes),
    ])
    similarity = UniversalEmbedder.cosine_similarity(
        text_result.embeddings[0], image_result.embeddings[0]
    )
```

### Flutter

pub`flutter pub add mediapipe_retrieval`[Live gallery](https://google.github.io/flutter-mediapipe/)·[Sample code](https://github.com/google/flutter-mediapipe/tree/main/samples/retrieval/universal_embedder)·[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_flutter.md.txt)

```dart
import 'dart:typed_data';

import 'package:mediapipe_retrieval/mediapipe_retrieval.dart';

Future<double> captionMatch(Uint8List photo, String caption) async {
  final capabilities = await queryUniversalEmbedderCapabilities();
  final embedder = await UniversalEmbedder.create(
    UniversalEmbedderOptions(
      model: RetrievalModels.embeddingGemma2TextVision,
      // The CPU natively, the GPU in browsers.
      delegate: capabilities.supportedDelegates.first,
    ),
  );
  try {
    final image = await embedder.embedImage(photo);
    final text = await embedder.embedText(caption);
    return UniversalEmbedder.cosineSimilarity(
      image.embeddings.first,
      text.embeddings.first,
    );
  } finally {
    await embedder.dispose();
  }
}
```

## Task details

This section describes the capabilities, inputs, outputs, and configuration
options of this task.

### Features

- **Unified multimodal embedding API** - Extract embeddings from text strings, raw or compressed images, and audio data within a single task lifecycle.
- **On-device execution** - Runs completely on-device on the LiteRT-LM runtime with a shared embedding engine for a small memory footprint.
- **Similarity comparison utilities** - Built-in static utility functions to compute the cosine similarity between raw float arrays or embedding results.
- **Semantic memory bridge** - Integrates directly with `SemanticRetriever` by exposing an `EmbeddingProvider` implementation.

Inputs The Universal Embedder accepts inputs of various modalities, either individually or as a mixed list:

- **Text** (`String`)
- **Image** (`MPImage` / `UIImage` / Compressed bytes --- see [Prepare vision input data](https://developers.google.com/edge/mediapipe/solutions/prepare_data.md.txt#vision))
- **Audio** (`AudioData` / Float samples --- see [Prepare audio input data](https://developers.google.com/edge/mediapipe/solutions/prepare_data.md.txt#audio))
- **Composite content** (A mixed list of any of the preceding objects)
Outputs The Universal Embedder outputs an `EmbeddingResult` containing:

- **Embedding**: The high-dimensional feature vector, either as raw floating-point numbers or scalar-quantized.

### Configuration options

This task has the following configuration options:

### Web

| Option Name | Description | Value Range | Default Value |
|---|---|---|---|
| `baseOptions.modelAssetPath` | Path, URL, or buffer for the `.litertlm` multimodal embedding model. | `String` | Required |
| `baseOptions.delegate` | Default hardware acceleration delegate for model execution. | {`CPU`, `GPU`} | `GPU` |
| `l2Normalize` | Whether to normalize the returned embedding feature vector using L2 normalization. | `Boolean` | `true` |
| `textDelegate / visionDelegate / audioDelegate` | Optional modality-specific hardware acceleration delegates. When unset, defaults to `baseOptions.delegate`. | {`CPU`, `GPU`} | `baseOptions.delegate` |
| `activationDataType` | Precision type used for activations during inference. | {`FLOAT32`, `FLOAT16`, `INT16`, `INT8`} | Model default |
| `maxInputLength` | Maximum input signature sequence length to load from the multisignature model. | `Integer` | Not set |
| `visionTokensPerImage` | Number of vision soft tokens to generate per image. | `Integer` | Not set |

### Android

| Option Name | Description | Value Range | Default Value |
|---|---|---|---|
| `baseOptions.modelAssetPath` | Path, URL, or buffer for the `.litertlm` multimodal embedding model. | `String` | Required |
| `baseOptions.delegate` | Default hardware acceleration delegate for model execution. | {`CPU`, `GPU`} | `CPU` |
| `l2Normalize` | Whether to normalize the returned embedding feature vector using L2 normalization. | `Boolean` | `true` |
| `textDelegate / visionDelegate / audioDelegate` | Optional modality-specific hardware acceleration delegates. When unset, defaults to `baseOptions.delegate`. | {`CPU`, `GPU`} | `baseOptions.delegate` |
| `activationDataType` | Precision type used for activations during inference. | {`FLOAT32`, `FLOAT16`, `INT16`, `INT8`} | Model default |
| `cacheDir` | Writable directory for storing compiled model cache artifacts. | `String` | Not set |
| `maxInputLength` | Maximum input signature sequence length to load from the multisignature model. | `Integer` | Not set |
| `visionTokensPerImage` | Number of vision soft tokens to generate per image. | `Integer` | Not set |

### iOS

| Option Name | Description | Value Range | Default Value |
|---|---|---|---|
| `baseOptions.modelAssetPath` | Path, URL, or buffer for the `.litertlm` multimodal embedding model. | `String` | Required |
| `baseOptions.delegate` | Default hardware acceleration delegate for model execution. | {`CPU`, `GPU`} | `CPU` |
| `l2Normalize` | Whether to normalize the returned embedding feature vector using L2 normalization. | `Boolean` | `true` |
| `textDelegate / visionDelegate / audioDelegate` | Optional modality-specific hardware acceleration delegates. When unset, defaults to `baseOptions.delegate`. | {`CPU`, `GPU`} | `baseOptions.delegate` |
| `activationDataType` | Precision type used for activations during inference. | {`FLOAT32`, `FLOAT16`, `INT16`, `INT8`} | Model default |
| `cacheDirectory` | Writable directory for storing compiled model cache artifacts. | `String` | Not set |
| `maxInputLength` | Maximum input signature sequence length to load from the multisignature model. | `Integer` | Not set |
| `visionTokensPerImage` | Number of vision soft tokens to generate per image. | `Integer` | Not set |

### Python

| Option Name | Description | Value Range | Default Value |
|---|---|---|---|
| `base_options.model_asset_path` | Path, URL, or buffer for the `.litertlm` multimodal embedding model. | `String` | Required |
| `base_options.delegate` | Default hardware acceleration delegate for model execution. | {`CPU`, `GPU`} | `CPU` |
| `l2_normalize` | Whether to normalize the returned embedding feature vector using L2 normalization. | `Boolean` | `True` |
| `text_delegate / vision_delegate / audio_delegate` | Optional modality-specific hardware acceleration delegates. When unset, defaults to `baseOptions.delegate`. | {`CPU`, `GPU`} | `baseOptions.delegate` |
| `activation_data_type` | Precision type used for activations during inference. | {`FLOAT32`, `FLOAT16`, `INT16`, `INT8`} | Model default |
| `cache_dir` | Writable directory for storing compiled model cache artifacts. | `String` | Not set |
| `max_input_length` | Maximum input signature sequence length to load from the multisignature model. | `Integer` | Not set |
| `vision_tokens_per_image` | Number of vision soft tokens to generate per image. | `Integer` | Not set |

### Flutter

| Option Name | Description | Value Range | Default Value |
|---|---|---|---|
| `model / modelPath / modelBytes` | Source of the `.litertlm` multimodal embedding model: a pinned `RetrievalModels` model, a model path, or the model's bytes. Supply exactly one. | `DownloadAsset` / `String` / `Uint8List` | Required |
| `delegate` | Hardware acceleration delegate for model execution. Native platforms run on `Delegate.cpu`; browsers run on `Delegate.gpu` (WebGPU) only. `queryUniversalEmbedderCapabilities()` reports the delegates available on the current platform. There are no modality-specific delegates. | {`Delegate.cpu`, `Delegate.gpu`} | `Delegate.cpu` |
| `l2Normalize` | Whether to normalize the returned embedding feature vector using L2 normalization. | `bool` | `true` |
| `activationDataType` | Precision type used for activations during inference. | {`ActivationDataType.modelDefault`, `float32`, `float16`, `int16`, `int8`} | `ActivationDataType.modelDefault` |
| `cacheDir` | Writable directory for storing compiled model cache artifacts. | `String?` | `null` (not set) |
| `maxInputLength` | Maximum input signature sequence length to load from the multisignature model. `0` keeps the model's default. | `int` \>= 0 | `0` (not set) |
| `visionTokensPerImage` | Number of vision soft tokens to generate per image. `0` keeps the model's default. | `int` \>= 0 | `0` (not set) |

### Task output

Each `UniversalEmbedderResult` (`EmbeddingResult`) returns the projected feature vector in the shared multimodal embedding space, where text and image vectors can be compared directly via `cosineSimilarity`:

| Field | Type / Range | Description |
|---|---|---|
| `floatEmbedding` | `Float32Array` / `float[]` | 768-dimensional L2-normalized float vector in the shared text-and-image space (`embedding` in Python; a `Float32List` in Dart). |
| `headIndex` | `int` | Embedding head index (`0`). |

#### Example `UniversalEmbedderResult` payload

The following example shows a 768-dimensional L2-normalized vector returned by `embedText` or `embedImage`:

```javascript
{
  embeddings: [
    {
      headIndex: 0,
      floatEmbedding: [-0.0341, 0.0518, 0.0192, -0.0284 /* ... 768 L2-normalized floats */]
    }
  ]
}
```

## Models

![Universal Embedder pipeline overview](https://developers.google.com/static/edge/mediapipe/images/solutions/diagrams/universal_embedder.svg)

The Universal Embedder supports the EmbeddingGemma V2 model, which is available
under the [Apache 2.0 License](https://gemma.ai/resources/apache-2/).

### EmbeddingGemma V2

EmbeddingGemma V2 is a multimodal embedding model capable of projecting text,
images, and audio into a single shared vector space. This unified representation
is what allows for accurate cross-modal capabilities, such as semantic image
search based on natural language concepts.

You can download the `.litertlm` model bundles for each modality from the LiteRT
community on Hugging Face:

| Modality | Parameters | Model |
|---|---|---|
| Text | 270M | [`embeddinggemma-2-text-270m.litertlm`](https://huggingface.co/litert-community/embeddinggemma-2-text-270m-litert-lm/resolve/main/embeddinggemma-2-text-270m.litertlm) |
| Text / Vision | 440M | [`embeddinggemma-2-text-vision-440m.litertlm`](https://huggingface.co/litert-community/embeddinggemma-2-text-vision-440m-litert-lm/resolve/main/embeddinggemma-2-text-vision-440m.litertlm) |
| Text / Vision / Audio | 740M | [`embeddinggemma-2-740m.litertlm`](https://huggingface.co/litert-community/embeddinggemma-2-740m-litert-lm/resolve/main/embeddinggemma-2-740m.litertlm) |