Semantic retriever guide for Web

The MediaPipe Semantic Retriever task lets you ingest and search semantic vectors efficiently on-device. This guide shows you how to use the Semantic Retriever for web and JavaScript applications to implement semantic search, allowing your application to retrieve relevant items based on meaning rather than explicit keywords.

For more information about the capabilities, models, and configuration options of this task, see the Overview.

Code example

The example code for Semantic Retriever provides a complete implementation of this task in JavaScript for your reference. This code helps you test this task and get started on building your own on-device search engine. You can view the Semantic Retriever example code on GitHub.

Setup

This section describes key steps for setting up your development environment and code projects specifically to use Semantic Retriever. For general information on setting up your development environment for using MediaPipe tasks, including platform version requirements, see the Setup guide for Web.

JavaScript packages

Semantic Retriever code is available through the @mediapipe/tasks-retrieval package. You can find and download these libraries from links provided in the platform Setup guide.

You can install the required packages with the following code for local staging using the following command:

npm install @mediapipe/tasks-retrieval

If you want to deploy to a server, you can use a content delivery network (CDN) service, such as jsDelivr, to add code directly to your HTML page, as follows:

<head>
  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/tasks-retrieval@latest/retrieval_bundle.mjs" crossorigin="anonymous"></script>
</head>

Model

To generate embeddings that populate the vector store, the MediaPipe Semantic Retriever task is typically coupled with a corresponding embedding model. For multimodal applications for web, the EmbeddingGemma V2 model using a UniversalEmbedder is highly recommended.

For more information, see the Models section of the task overview. You can download a model and host it alongside your application, or reference it directly by URL in the modelAssetPath option.

Create the task

Instead of taking model configuration options directly, the SemanticRetriever groups vector storage (such as in-memory vector storage) and UniversalEmbedder embedding providers using SemanticRetrieverComponents.

import { SemanticRetriever, InMemoryVectorStore, UniversalEmbedder, FilesetResolver } from "@mediapipe/tasks-retrieval";

let semanticRetriever;

async function createRetriever() {
  const retrieval = await FilesetResolver.forRetrievalTasks("https://cdn.jsdelivr.net/npm/@mediapipe/tasks-retrieval@latest/wasm");

  // 1. Initialize the embedding engine / provider
  const embedder = await UniversalEmbedder.createFromOptions(retrieval, {
    baseOptions: {
      modelAssetPath: "https://huggingface.co/litert-community/embeddinggemma-2-text-vision-440m-litert-lm/resolve/main/embeddinggemma-2-text-vision-440m.litertlm"
    }
  });

  // 2. Initialize the semantic retriever
  semanticRetriever = await SemanticRetriever.createFromComponents(retrieval, {
    vectorStore: new InMemoryVectorStore(),
    providers: [embedder.getProvider()]
  });
}

createRetriever();

Prepare data

You can directly load content strings into the retriever. These inputs are passed to the integrated embedding provider automatically, and the generated embeddings are managed safely inside the internal vector store.

The Web API lets you push standard string queries into the index using SemanticRetriever.index().

// Ingest text entries into the vector store
await semanticRetriever.index("How to compute semantic distance on the web");
await semanticRetriever.index("Best vector search libraries for JavaScript");
await semanticRetriever.index("A completely irrelevant document about apples and bananas.");

Run inference

Use the API to search and retrieve related documents.

Single Search Query

Search operations use standard strings (or items). These are seamlessly delegated to the UniversalEmbedder instances for generating the internal search vector natively in WASM/WebGL.

const searchOptions = {
    maxResults: 2
};

// Initiate the semantic retrieval query
const results = await semanticRetriever.retrieve("What is vector search in JS?", searchOptions);

Handle and display results

The retrieval outputs are mapped to internal records.

results.forEach((retrievalResult, index) => {
    console.log(`Rank ${index + 1}:`);
    console.log(`  Similarity Score: ${retrievalResult.score}`);
    console.log(`  Text Data: ${retrievalResult.textData}`);
});