Text-to-speech guide for web

The MediaPipe Text-to-Speech task lets you synthesize natural-sounding speech audio from text in the browser using neural TTS models such as Kokoro-82M. These instructions show you how to use Text-to-Speech for web apps.

You can see this task in action by viewing the demo. For more information about the capabilities, .litertlm model bundles, and supported voices of this task, see the Overview.

Note: Please ensure you process voice data responsibly and in accordance with applicable law.

Code example

The example code for Text-to-Speech provides a complete implementation of this task in TypeScript for your reference. The example runs the task in a web worker, lists the available voices, synthesizes and plays speech, streams speech clause by clause, highlights words as they are spoken, and saves the audio as a WAV file. You can view, run, and edit the Text-to-Speech example using just your web browser. You can view the source code for this example on GitHub.

Setup

This section describes key steps for setting up your development environment and code projects specifically to use Text-to-Speech. For general information on setting up your development environment for using MediaPipe tasks, including platform version requirements, see the Setup guide for web.

JavaScript packages

Text-to-Speech code is available through the MediaPipe @mediapipe/tasks-audio NPM package. You can install the package with the following command:

npm install @mediapipe/tasks-audio

Note: Text-to-Speech requires version 1.2.2 or later of @mediapipe/tasks-audio.

If you want to import the task code through a content delivery network (CDN) service, load the task's Wasm files from the CDN as shown in Create the task.

Model

The MediaPipe Text-to-Speech task requires a .litertlm model bundle. This guide uses kokoro_int8_gpu_fp16.litertlm. See the task overview Models section for the other supported models.

The task downloads the model from the URL that you pass in the options. The model bundles are large, so consider caching them in the browser, for example with the Cache Storage API, and passing the cached model contents to the task as baseOptions.modelAssetBuffer. The example code shows one way to do this.

Create the task

Use the createFromOptions() function to create the task with TextToSpeechOptions. For more information on configuration options, see Configuration options.

import {FilesetResolver, TextToSpeech} from '@mediapipe/tasks-audio';

const audio = await FilesetResolver.forAudioTasks(
    'https://cdn.jsdelivr.net/npm/@mediapipe/tasks-audio@latest/wasm');

const tts = await TextToSpeech.createFromOptions(audio, {
  baseOptions: {
    modelAssetPath:
        'https://huggingface.co/litert-community/Kokoro-82M/resolve/main/kokoro_int8_gpu_fp16.litertlm',
    delegate: 'GPU',
  },
  defaultVoiceId: 'af_heart',
  speakingRate: 1.0,
  cacheDirectory: ':nocache',
});

The GPU delegate uses WebGPU. If the browser doesn't support WebGPU, or the device has no suitable GPU, createFromOptions() rejects, and you can create the task with delegate: 'CPU' instead.

To use PocketTTS or Phonon, pass that model's bundle URL and set defaultVoiceId to one of its voices (or leave it unset). Both models run on CPU, so they ignore the delegate setting.

Loading a model and synthesizing speech are compute-intensive and can block the page's main thread. To keep your user interface responsive, run the task in a web worker and send the audio to the main thread for playback, as shown in the example code.

Configuration options

This task has the following configuration options for web applications:

Task options (TextToSpeechOptions)

Option Name	Description	Value Range	Default Value
baseOptions.modelAssetPath	Path (or URL on Web) to a supported .litertlm model bundle (see Models). Keep the original filename, which the task uses to detect the model type.	String / Path	Required
baseOptions.delegate	Hardware acceleration delegate used by the LiteRT runtime. When set to GPU, the Kokoro acoustic stage executes on CPU (XNNPACK) and the neural vocoder executes on GPU (OpenCL/Metal/WebGPU) with CPU fallback. PocketTTS and Phonon always run on CPU.	"CPU", "GPU"	"CPU"
speakingRate	Speaking rate multiplier applied to synthesized audio (1.0 represents normal speaking speed).	Float (0.1, 4.0]	1.0
targetBucket	Target sequence-length bucket for the Kokoro acoustic/vocoder graph (for example, 64, 128, or 256 tokens). Smaller buckets reduce synthesis latency for short sentences; 0 uses the model's default capacity. PocketTTS and Phonon ignore this option.	0, 64, 128, 256	0
defaultVoiceId	Default voice ID (for example, "af_heart" for Kokoro-82M, "alba" for PocketTTS, or "freya" for Phonon). Can be overridden per utterance using SynthesisOptions.	String (Optional)	The model's default voice
defaultLocale	Optional default BCP 47 language tag (for example, "en-US", "es", "zh-CN"). When omitted, the task automatically infers the language from the selected voice.	BCP 47 String (Optional)	Inferred from voice
cacheDirectory	Writable directory path used to persist unpacked phonetic dictionary assets and LiteRT XNNPACK weight caches across runs for faster cold starts. Used by Kokoro-82M. On the web, set it to ":nocache", because the Wasm virtual file system can't hold the weight cache.	Directory Path (Optional)	""

Per-utterance options (SynthesisOptions)

Option Name	Description	Default Value
voiceId	Overrides the voice profile for a single synthesize call (for example, switching from "af_heart" to "ef_dora" in under 1 ms without reloading model weights).	null / None (uses task default)
locale	Optional BCP 47 locale override for a single utterance. Leave unset (null / None) when switching voices so the engine automatically resolves the voice's target language from the voice ID.	null / None (inferred from voiceId)

On the web, you pass the per-utterance options as a SynthesisOptions object to synthesize() and synthesizeStream().

Inspect available voices

Call listVoices() on the initialized task to enumerate all voices packaged in the loaded .litertlm bundle:

for (const voice of await tts.listVoices()) {
  console.log(`${voice.voiceId} (${voice.displayName}) - ` +
              `locale=${voice.locale}, ${voice.sampleRateHz} Hz`);
}

Run the task

Synthesize speech

Call synthesize() to convert a piece of text into speech in a single call. Multi-sentence text is split into clauses automatically.

const result = await tts.synthesize(
    'Hello from MediaPipe Text to Speech! Multi-sentence inputs are chunked ' +
    'automatically.');

Switch voices

To switch voices or languages on a single utterance, pass the target voiceId. Leave locale unset so the task resolves the language from the voice:

// Speak English with "af_heart", then switch to Spanish with "ef_dora":
const enResult = await tts.synthesize(
    "Good morning! Let's practice Spanish.", {voiceId: 'af_heart'});
const esResult = await tts.synthesize(
    '¡Buenos días! Practiquemos español.', {voiceId: 'ef_dora'});

Streaming speech synthesis

For a lower time to first audio on multi-sentence text, call synthesizeStream(). It returns a TextToSpeechStream that yields an audio chunk for each clause as soon as it's ready, so you can start playing the first clause while the following clauses are synthesized:

const stream = await tts.synthesizeStream(
    'Streaming synthesis plays the first clause immediately. ' +
    'Subsequent clauses synthesize in the background.');
try {
  for await (const chunk of stream) {
    // Play chunk.samples as each clause arrives.
    play(chunk.samples, chunk.sampleRateHz);
  }
} finally {
  stream.close();
}

When streaming tokens incrementally from an LLM, open the stream with an empty initial text and finishInput set to false. Push text as tokens arrive with pushText(), and call finishInput() at the end of the response. Read the audio chunks from the stream while you push text:

const stream = await tts.synthesizeStream('', {voiceId: 'af_heart'},
                                          /* finishInput= */ false);

async function pushLlmResponse() {
  for await (const token of llmTokens) {
    await stream.pushText(token);
  }
  await stream.finishInput();
}

const pushing = pushLlmResponse();
for await (const chunk of stream) {
  play(chunk.samples, chunk.sampleRateHz);
}
await pushing;
stream.close();

Barge-in cancellation

If the user interrupts speech playback ("barge-in"), call tts.cancel() (or stream.cancel()) to abort the active streaming synthesis. The next read from the stream rejects, and you can keep using the task afterwards.

The task runs on the thread that calls it, and it synthesizes each clause when you read it from the stream. A cancel request therefore takes effect between clauses. A synthesize() call runs to completion, so use synthesizeStream() for speech that the user can interrupt. When the task runs in a web worker, handle the cancel message right away, not after the synthesis request that it's meant to stop, as shown in the example code.

let bargeIn = false;  // Set to true when the user interrupts.

try {
  for await (const chunk of await tts.synthesizeStream(longText)) {
    if (bargeIn) {
      tts.cancel();
      break;
    }
    play(chunk.samples, chunk.sampleRateHz);
  }
} catch (e) {
  // Synthesis was cancelled.
}

Handle and play results

synthesize() returns a TextToSpeechResult, and the stream yields one TextToSpeechAudioChunk per clause. Each result contains:

• samples: Float32Array of mono PCM samples in the range [-1.0, 1.0]
• sampleRateHz: sample rate in Hz (24000 for all supported models)
• chunkIndex and isFinal (stream chunks only): the position of the chunk in the stream, and whether it's the last chunk
• wordTimestamps: array of WordTimestamp objects with the word, its position in the input text (startChar, endChar), and when it's spoken (startSec, endSec)

You can play the samples with the Web Audio API:

const audioCtx = new AudioContext();

function play(samples, sampleRateHz) {
  const buffer = audioCtx.createBuffer(1, samples.length, sampleRateHz);
  buffer.copyToChannel(samples, 0);
  const source = audioCtx.createBufferSource();
  source.buffer = buffer;
  source.connect(audioCtx.destination);
  source.start();
}

To play stream chunks back to back without gaps, schedule each chunk to start when the previous one ends, as shown in the example code.

Use the word timestamps to highlight words as they are read aloud. In streaming synthesis, the timestamps and character positions are relative to the start of the stream and the full input text.

for (const word of result.wordTimestamps ?? []) {
  console.log(`${word.startSec.toFixed(2)}s - ${word.endSec.toFixed(2)}s: ` +
              text.slice(word.startChar, word.endChar));
}

Release resources

When you no longer need the task, call close() to release the model and its memory:

tts.close();