Text-to-speech guide for Python

The MediaPipe Text-to-Speech task lets you synthesize natural-sounding speech audio from text on the device using neural TTS models such as Kokoro-82M. These instructions show you how to use Text-to-Speech with Python.

For more information about the capabilities, .litertlm model bundles, and supported voices of this task, see the Overview.

Note: Please ensure you process voice data responsibly and in accordance with applicable law.

Code example

The example code for Text-to-Speech provides a complete implementation of this task in Python for your reference. The example lists the available voices, synthesizes and plays speech, saves it as a WAV file, prints word timestamps, and shows streaming synthesis and incremental text input. You can view, run, and edit the Text-to-Speech example code using just your web browser with Google Colab. You can view the source code for this example, including a command-line version, on GitHub.

Setup

This section describes key steps for setting up your development environment and code projects specifically to use Text-to-Speech. For general information on setting up your development environment for using MediaPipe tasks, including platform version requirements, see the Setup guide for Python.

Packages

The Text-to-Speech task uses the mediapipe pip package. You can install the dependency with the following:

$ python -m pip install mediapipe numpy

Note: Text-to-Speech requires version 1.2.0 or later of mediapipe, and Python 3.10 or later.

Imports

Import the following modules to access the Text-to-Speech task functions:

from mediapipe.tasks.python.audio import text_to_speech
from mediapipe.tasks.python.core import base_options

Model

The MediaPipe Text-to-Speech task requires a .litertlm model bundle. This guide uses kokoro_int8_gpu_fp16.litertlm. See the task overview Models section for the other supported models.

Download the model, store it in a local directory, and keep its original filename, which the task uses to detect the model type:

model_path = '/absolute/path/to/kokoro_int8_gpu_fp16.litertlm'

Create the task

Use the create_from_options function to create the task with TextToSpeechOptions. Pass a writable cache_directory so that unpacked phonetic dictionary data and weight caches persist across runs.

import os
import tempfile

BaseOptions = base_options.BaseOptions
TextToSpeech = text_to_speech.TextToSpeech
TextToSpeechOptions = text_to_speech.TextToSpeechOptions

options = TextToSpeechOptions(
    base_options=BaseOptions(
        model_asset_path=model_path,
        delegate=BaseOptions.Delegate.GPU),
    default_voice_id='af_heart',
    speaking_rate=1.0,
    cache_directory=os.path.join(tempfile.gettempdir(), 'mediapipe_tts_cache'))

with TextToSpeech.create_from_options(options) as tts:
  # The task is initialized. Use it here.
  # ...

If the GPU isn't available, create_from_options raises a RuntimeError, and you can create the task with BaseOptions.Delegate.CPU instead. On Linux, the GPU delegate uses Vulkan, and the user running your application needs access to the GPU device, for example by being a member of the render group.

To use PocketTTS or Phonon, pass that model's bundle path and set default_voice_id to one of its voices (or leave it unset). Both models run on CPU, so they ignore the delegate setting.

For a complete example of creating a Text-to-Speech task, see the code example.

Configuration options

This task has the following configuration options for Python applications:

Task options (TextToSpeechOptions)

Option Name	Description	Value Range	Default Value
base_options.model_asset_path	Path (or URL on Web) to a supported .litertlm model bundle (see Models). Keep the original filename, which the task uses to detect the model type.	String / Path	Required
base_options.delegate	Hardware acceleration delegate used by the LiteRT runtime. When set to GPU, the Kokoro acoustic stage executes on CPU (XNNPACK) and the neural vocoder executes on GPU (OpenCL/Metal/WebGPU) with CPU fallback. PocketTTS and Phonon always run on CPU.	BaseOptions.Delegate.CPU, BaseOptions.Delegate.GPU	BaseOptions.Delegate.CPU
speaking_rate	Speaking rate multiplier applied to synthesized audio (1.0 represents normal speaking speed).	Float (0.1, 4.0]	1.0
target_bucket	Target sequence-length bucket for the Kokoro acoustic/vocoder graph (for example, 64, 128, or 256 tokens). Smaller buckets reduce synthesis latency for short sentences; 0 uses the model's default capacity. PocketTTS and Phonon ignore this option.	0, 64, 128, 256	0
default_voice_id	Default voice ID (for example, "af_heart" for Kokoro-82M, "alba" for PocketTTS, or "freya" for Phonon). Can be overridden per utterance using SynthesisOptions.	String (Optional)	The model's default voice
default_locale	Optional default BCP 47 language tag (for example, "en-US", "es", "zh-CN"). When omitted, the task automatically infers the language from the selected voice.	BCP 47 String (Optional)	Inferred from voice
cache_directory	Writable directory path used to persist unpacked phonetic dictionary assets and LiteRT XNNPACK weight caches across runs for faster cold starts. Used by Kokoro-82M.	Directory Path (Optional)	""

Per-utterance options (SynthesisOptions)

Option Name	Description	Default Value
voice_id	Overrides the voice profile for a single synthesize call (for example, switching from "af_heart" to "ef_dora" in under 1 ms without reloading model weights).	null / None (uses task default)
locale	Optional BCP 47 locale override for a single utterance. Leave unset (null / None) when switching voices so the engine automatically resolves the voice's target language from the voice ID.	null / None (inferred from voiceId)

In Python, you pass the per-utterance options as the voice_id and locale keyword arguments of synthesize(), synthesize_stream(), and start_stream().

Inspect available voices

Call list_voices() on the initialized task to enumerate all voices packaged in the loaded .litertlm bundle:

for voice in tts.list_voices():
  print(f'{voice.voice_id} ({voice.display_name}) - '
        f'locale={voice.locale}, {voice.sample_rate_hz} Hz')

Run the task

Synthesize speech

Call synthesize() to convert a piece of text into speech in a single call. Multi-sentence text is split into clauses automatically.

result = tts.synthesize(
    'Hello from MediaPipe Text to Speech! Multi-sentence inputs are chunked '
    'automatically.')

Switch voices

To switch voices or languages on a single utterance, pass the target voice_id. Leave locale unset so the task resolves the language from the voice:

# Speak English with "af_heart", then switch to Spanish with "ef_dora":
en_result = tts.synthesize("Good morning! Let's practice Spanish.",
                           voice_id='af_heart')
es_result = tts.synthesize('¡Buenos días! Practiquemos español.',
                           voice_id='ef_dora')

Streaming speech synthesis

For a lower time to first audio on multi-sentence text, call synthesize_stream(). It returns an iterator that yields a TtsAudioResult for each clause as soon as it's ready, so you can start playing the first clause while the following clauses are synthesized:

for chunk in tts.synthesize_stream(
    'Streaming synthesis plays the first clause immediately. '
    'Subsequent clauses synthesize in the background.'):
  # Play chunk.samples as each clause arrives.
  play(chunk.samples, chunk.sample_rate_hz)

When streaming tokens incrementally from an LLM, open a stream with start_stream(), push text as tokens arrive with push_text(), and call finish_input() at the end of the response. Iterate over the stream to get the audio chunks. Iterating waits for the next chunk, so push text and read audio on different threads:

import threading

def push_llm_response(stream):
  for token in llm_tokens:
    stream.push_text(token)
  stream.finish_input()

with tts.start_stream(voice_id='af_heart') as stream:
  threading.Thread(target=push_llm_response, args=(stream,)).start()
  for chunk in stream:
    play(chunk.samples, chunk.sample_rate_hz)

Barge-in cancellation

If the user interrupts speech playback ("barge-in"), call tts.cancel() (or stream.cancel()) from any thread to abort the active synthesis. The interrupted synthesize() call or stream iteration raises a RuntimeError. You can keep using the task afterwards.

try:
  for chunk in tts.synthesize_stream(long_text):
    play(chunk.samples, chunk.sample_rate_hz)
except RuntimeError:
  pass  # Synthesis was cancelled.

# On user barge-in, from another thread:
tts.cancel()

Handle and play results

synthesize() returns a TtsAudioResult, and the streaming functions yield one TtsAudioResult per clause. Each result contains:

• samples: 1-D NumPy array of float32 mono PCM samples in the range [-1.0, 1.0]
• sample_rate_hz: int (24000 Hz for all supported models)
• duration_seconds: duration of the audio in seconds
• chunk_index and is_final: the position of the chunk in a stream, and whether it's the last chunk
• word_timestamps: tuple of WordTimestamp objects with the word, its position in the input text (start_char, end_char), and when it's spoken (start_sec, end_sec)

You can play the samples in a notebook with IPython.display.Audio(result.samples, rate=result.sample_rate_hz), or save them as a 16-bit WAV file with Python's built-in wave module:

import wave

import numpy as np

pcm = (np.clip(result.samples, -1.0, 1.0) * 32767).astype(np.int16)
with wave.open('speech.wav', 'wb') as wav_file:
  wav_file.setnchannels(1)
  wav_file.setsampwidth(2)  # 16 bits per sample.
  wav_file.setframerate(result.sample_rate_hz)
  wav_file.writeframes(pcm.tobytes())

Use the word timestamps to highlight words as they are read aloud. In streaming synthesis, the timestamps and character positions are relative to the start of the stream and the full input text.

for word in result.word_timestamps:
  print(f'{word.start_sec:.2f}s - {word.end_sec:.2f}s: '
        f'{text[word.start_char:word.end_char]}')

Release resources

When you no longer need the task, call close() to release the model and its memory. You can also use the task as a context manager in a with statement, as shown in Create the task, which closes it automatically.