<br />

MediaPipe Tasks automatically handles model-specific input
preprocessing---including image resizing, rotation, and value normalization for
vision models, and audio resampling, channel mixing, buffering, and spectrogram
framing for audio models. Before running inference, wrap your platform's image,
video, camera, or microphone buffers in the input container expected by your
target SDK.
Vision Tasks

[Prepare vision input data](https://developers.google.com/edge/mediapipe/solutions/prepare_data.md.txt#vision)

Wrap still images, decoded video frames, and live camera streams in `MPImage` (Android and iOS), `mp.Image` (Python), `VisionImage` (Flutter), or standard DOM media elements (Web), and configure frame rotation or regions of interest.
Audio Tasks

[Prepare audio input data](https://developers.google.com/edge/mediapipe/solutions/prepare_data.md.txt#audio)

Convert prerecorded audio files and live microphone PCM buffers into `AudioData` ring buffers or normalized `Float32` sample arrays across Web, Android, iOS, Python, and Flutter.

## Prepare vision input data

Vision tasks accept still images, decoded video frames, and live camera feeds
(on Android and iOS,
[Universal Embedder](https://developers.google.com/edge/mediapipe/solutions/retrieval/universal_embedder.md.txt)
also accepts `MPImage` for image embeddings, while Web, Python, and Flutter pass
encoded image bytes). Each task automatically resizes, normalizes, and converts
color spaces to match its underlying model bundle.

| Platform | Input type | Supported media sources \& orientation handling |
|---|---|---|
| **Web** | `HTMLImageElement`, `HTMLVideoElement`, `HTMLCanvasElement`, `ImageBitmap`, `OffscreenCanvas`, `ImageData` | Pass browser media elements directly to `detect()` or `detectForVideo()` without an intermediate wrapper. |
| **Android** | `MPImage` | Build from `ARGB_8888` `Bitmap` (`BitmapImageBuilder`, including CameraX `ImageProxy.toBitmap()`), `RGBA_8888` `android.media.Image` (`MediaImageBuilder`), or `ByteBuffer` (`ByteBufferImageBuilder`). Apply clockwise sensor rotation in 90° multiples with `ImageProcessingOptions`. |
| **iOS** | `MPImage` | Initialize from `UIImage`, `CVPixelBuffer`, or camera `CMSampleBuffer` with a non-mirrored `UIImage.Orientation` (`.up`, `.down`, `.left`, or `.right`). |
| **Python** | `mp.Image` | Load files with `mp.Image.create_from_file()` or wrap RGB/RGBA NumPy arrays with `mp.Image(image_format=mp.ImageFormat.SRGB, data=...)`. Convert OpenCV `BGR` frames to `RGB` first. |
| **Flutter** | `VisionImage` | Decode a file with `VisionImage.fromFile()` (a path on native platforms, a URL on the web), wrap 8-bit pixels with `VisionImage.fromPixels()` (`VisionPixelFormat.rgb`, `.rgba`, or `.bgra`, with optional `bytesPerRow` for padded camera rows), or wrap a browser `ImageBitmap` or video frame with `VisionImage.fromBrowserFrame()` on the web. Convert YUV camera buffers to RGB or RGBA first; in livestream mode, wrap the conversion in `VisionImage.deferred()`. Pass clockwise rotation in 90° multiples with the `rotationDegrees` argument of each call. |

### Vision input code examples

Select your platform and running mode to prepare still images, video frames
with monotonic millisecond timestamps, or live camera streams (shown here with
`HandLandmarker`; all Vision tasks accept the same media inputs):

### Web

npm`npm install @mediapipe/tasks-vision`[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_web.md.txt)·[API reference](https://developers.google.com/edge/api/mediapipe/js/tasks-vision.md.txt)

#### Image

`IMAGE`Pass any decoded `HTMLImageElement`, `HTMLCanvasElement`, `ImageBitmap`, or `ImageData` directly to the task.

```typescript
import type { HandLandmarker, HandLandmarkerResult } from "@mediapipe/tasks-vision";

export async function detectFromImageElement(
  handLandmarker: HandLandmarker,
  imgElement: HTMLImageElement,
): Promise<HandLandmarkerResult> {
  await imgElement.decode();
  return handLandmarker.detect(imgElement);
}

export async function detectFromBlob(
  handLandmarker: HandLandmarker,
  fileOrBlob: Blob,
): Promise<HandLandmarkerResult> {
  const bitmap = await createImageBitmap(fileOrBlob);
  try {
    return handLandmarker.detect(bitmap);
  } finally {
    bitmap.close();
  }
}
```

#### Video

`VIDEO`Pass an `HTMLVideoElement` (or decoded frame canvas) together with strictly increasing frame timestamps in milliseconds.

```typescript
import type { HandLandmarker, HandLandmarkerResult } from "@mediapipe/tasks-vision";

export async function analyzeVideoClip(
  handLandmarker: HandLandmarker,
  video: HTMLVideoElement,
  onResult: (result: HandLandmarkerResult, timestampMs: number) => void,
): Promise<void> {
  let lastTimestampMs = -1;

  const processNextFrame = () => {
    if (video.paused || video.ended) return;
    const timestampMs = Math.round(video.currentTime * 1000);
    if (timestampMs > lastTimestampMs) {
      lastTimestampMs = timestampMs;
      onResult(handLandmarker.detectForVideo(video, timestampMs), timestampMs);
    }
    video.requestVideoFrameCallback(processNextFrame);
  };

  await video.play();
  video.requestVideoFrameCallback(processNextFrame);
}
```

#### Livestream

`VIDEO + rAF`Attach a `getUserMedia` camera stream to an `HTMLVideoElement` and inspect only new frames inside `requestAnimationFrame`.

```typescript
import type { HandLandmarker, HandLandmarkerResult } from "@mediapipe/tasks-vision";

export async function startWebcamStream(
  handLandmarker: HandLandmarker,
  webcam: HTMLVideoElement,
  onResult: (result: HandLandmarkerResult) => void,
): Promise<void> {
  webcam.srcObject = await navigator.mediaDevices.getUserMedia({
    video: { facingMode: "user", width: { ideal: 1280 }, height: { ideal: 720 } },
  });
  await webcam.play();

  let lastVideoTime = -1;
  const renderLoop = () => {
    if (webcam.readyState >= 2 && webcam.currentTime !== lastVideoTime) {
      lastVideoTime = webcam.currentTime;
      onResult(handLandmarker.detectForVideo(webcam, performance.now()));
    }
    requestAnimationFrame(renderLoop);
  };
  renderLoop();
}
```

### Android (Kotlin)

Gradle`implementation("com.google.mediapipe:tasks-vision:latest.release")`[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_android.md.txt)·[MPImage API](https://developers.google.com/edge/api/mediapipe/java/com/google/mediapipe/framework/image/package-summary.md.txt)

#### Image

`IMAGE`Wrap an `ARGB_8888` `Bitmap` in `MPImage` using `BitmapImageBuilder`.

```kotlin
import android.graphics.Bitmap
import com.google.mediapipe.framework.image.BitmapImageBuilder
import com.google.mediapipe.tasks.vision.handlandmarker.HandLandmarker
import com.google.mediapipe.tasks.vision.handlandmarker.HandLandmarkerResult

fun detectBitmap(handLandmarker: HandLandmarker, bitmap: Bitmap): HandLandmarkerResult {
    // Hardware-backed bitmaps must be copied to ARGB_8888 before wrapping
    val argbBitmap = if (bitmap.config == Bitmap.Config.ARGB_8888) {
        bitmap
    } else {
        bitmap.copy(Bitmap.Config.ARGB_8888, /* isMutable = */ false)
    }
    val mpImage = BitmapImageBuilder(argbBitmap).build()
    return handLandmarker.detect(mpImage)
}
```

#### Video

`VIDEO`Extract frames with `MediaMetadataRetriever`, convert to `ARGB_8888`, and pass monotonic timestamps in milliseconds.

```kotlin
import android.content.Context
import android.graphics.Bitmap
import android.media.MediaMetadataRetriever
import android.net.Uri
import com.google.mediapipe.framework.image.BitmapImageBuilder
import com.google.mediapipe.tasks.vision.handlandmarker.HandLandmarker
import com.google.mediapipe.tasks.vision.handlandmarker.HandLandmarkerResult

fun detectVideoFrame(
    context: Context,
    videoUri: Uri,
    timestampUs: Long,
    handLandmarker: HandLandmarker
): HandLandmarkerResult? {
    val retriever = MediaMetadataRetriever()
    try {
        retriever.setDataSource(context, videoUri)
        val frame = retriever.getFrameAtTime(timestampUs, MediaMetadataRetriever.OPTION_CLOSEST) ?: return null

        val argbFrame = if (frame.config == Bitmap.Config.ARGB_8888) {
            frame
        } else {
            frame.copy(Bitmap.Config.ARGB_8888, false)
        }
        val mpImage = BitmapImageBuilder(argbFrame).build()
        return handLandmarker.detectForVideo(mpImage, timestampUs / 1000L)
    } finally {
        retriever.release()
    }
}
```

#### Livestream

`LIVE_STREAM`Convert each CameraX `ImageProxy` to a `Bitmap` inside `use` and pass clockwise sensor rotation with `ImageProcessingOptions`.

```kotlin
import android.os.SystemClock
import androidx.camera.core.ImageProxy
import com.google.mediapipe.framework.image.BitmapImageBuilder
import com.google.mediapipe.tasks.vision.core.ImageProcessingOptions
import com.google.mediapipe.tasks.vision.handlandmarker.HandLandmarker

fun analyzeCameraFrame(imageProxy: ImageProxy, handLandmarker: HandLandmarker) {
    imageProxy.use { proxy ->
        val mpImage = BitmapImageBuilder(proxy.toBitmap()).build()
        val imageProcessingOptions = ImageProcessingOptions.builder()
            .setRotationDegrees(proxy.imageInfo.rotationDegrees)
            .build()

        handLandmarker.detectAsync(mpImage, imageProcessingOptions, SystemClock.uptimeMillis())
    }
}
```

### Android (Java)

Gradle`implementation 'com.google.mediapipe:tasks-vision:latest.release'`[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_android.md.txt)·[MPImage API](https://developers.google.com/edge/api/mediapipe/java/com/google/mediapipe/framework/image/package-summary.md.txt)

#### Image

`IMAGE`Wrap an `ARGB_8888` `Bitmap` in `MPImage` using `BitmapImageBuilder`.

```java
import android.graphics.Bitmap;
import com.google.mediapipe.framework.image.BitmapImageBuilder;
import com.google.mediapipe.framework.image.MPImage;
import com.google.mediapipe.tasks.vision.handlandmarker.HandLandmarker;
import com.google.mediapipe.tasks.vision.handlandmarker.HandLandmarkerResult;

HandLandmarkerResult detectBitmap(HandLandmarker handLandmarker, Bitmap bitmap) {
  Bitmap argbBitmap = (bitmap.getConfig() == Bitmap.Config.ARGB_8888)
      ? bitmap
      : bitmap.copy(Bitmap.Config.ARGB_8888, /* isMutable= */ false);
  MPImage mpImage = new BitmapImageBuilder(argbBitmap).build();
  return handLandmarker.detect(mpImage);
}
```

#### Video

`VIDEO`Extract frames with `MediaMetadataRetriever`, convert to `ARGB_8888`, and pass monotonic timestamps in milliseconds.

```java
import android.content.Context;
import android.graphics.Bitmap;
import android.media.MediaMetadataRetriever;
import android.net.Uri;
import com.google.mediapipe.framework.image.BitmapImageBuilder;
import com.google.mediapipe.framework.image.MPImage;
import com.google.mediapipe.tasks.vision.handlandmarker.HandLandmarker;
import com.google.mediapipe.tasks.vision.handlandmarker.HandLandmarkerResult;
import java.io.IOException;

HandLandmarkerResult detectVideoFrame(
    Context context,
    Uri videoUri,
    long timestampUs,
    HandLandmarker handLandmarker) throws IOException {
  MediaMetadataRetriever retriever = new MediaMetadataRetriever();
  try {
    retriever.setDataSource(context, videoUri);
    Bitmap frame = retriever.getFrameAtTime(timestampUs, MediaMetadataRetriever.OPTION_CLOSEST);
    if (frame == null) {
      return null;
    }

    Bitmap argbFrame = (frame.getConfig() == Bitmap.Config.ARGB_8888)
        ? frame
        : frame.copy(Bitmap.Config.ARGB_8888, false);
    MPImage mpImage = new BitmapImageBuilder(argbFrame).build();
    return handLandmarker.detectForVideo(mpImage, timestampUs / 1000L);
  } finally {
    retriever.release();
  }
}
```

#### Livestream

`LIVE_STREAM`Convert each CameraX `ImageProxy` to a `Bitmap` in a `try / finally` block and pass clockwise sensor rotation with `ImageProcessingOptions`.

```java
import android.graphics.Bitmap;
import android.os.SystemClock;
import androidx.camera.core.ImageProxy;
import com.google.mediapipe.framework.image.BitmapImageBuilder;
import com.google.mediapipe.framework.image.MPImage;
import com.google.mediapipe.tasks.vision.core.ImageProcessingOptions;
import com.google.mediapipe.tasks.vision.handlandmarker.HandLandmarker;

void analyzeCameraFrame(ImageProxy imageProxy, HandLandmarker handLandmarker) {
  try {
    Bitmap bitmap = imageProxy.toBitmap();
    MPImage mpImage = new BitmapImageBuilder(bitmap).build();
    ImageProcessingOptions imageProcessingOptions = ImageProcessingOptions.builder()
        .setRotationDegrees(imageProxy.getImageInfo().getRotationDegrees())
        .build();

    handLandmarker.detectAsync(mpImage, imageProcessingOptions, SystemClock.uptimeMillis());
  } finally {
    imageProxy.close();
  }
}
```

### iOS

SwiftPM`google-ai-edge/mediapipe (MediaPipeTasksVision)`[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_ios.md.txt)·[MPImage API](https://developers.google.com/edge/api/mediapipe/swift/vision/Classes/MPImage.md.txt)

#### Image

`IMAGE`Initialize `MPImage` from a `UIImage` (or processed `CVPixelBuffer`). Mirrored `UIImage.Orientation` values are not supported.

```swift
import MediaPipeTasksVision
import UIKit

func detectImage(_ uiImage: UIImage, handLandmarker: HandLandmarker) throws -> HandLandmarkerResult {
    // Uses uiImage.imageOrientation by default (or pass orientation: .up/.down/.left/.right)
    let mpImage = try MPImage(uiImage: uiImage)
    return try handLandmarker.detect(image: mpImage)
}
```

#### Video

`VIDEO`Read uncompressed `32BGRA` `CMSampleBuffer` frames with `AVAssetReader` and compute millisecond timestamps.

```swift
import AVFoundation
import MediaPipeTasksVision

func analyzeVideo(
    asset: AVAsset,
    videoTrack: AVAssetTrack,
    handLandmarker: HandLandmarker,
    onResult: (HandLandmarkerResult, Int) -> Void
) throws {
    let reader = try AVAssetReader(asset: asset)
    let trackOutput = AVAssetReaderTrackOutput(
        track: videoTrack,
        outputSettings: [kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_32BGRA]
    )
    reader.add(trackOutput)
    reader.startReading()

    while let sampleBuffer = trackOutput.copyNextSampleBuffer() {
        let mpImage = try MPImage(sampleBuffer: sampleBuffer, orientation: .up)
        let pts = CMSampleBufferGetPresentationTimeStamp(sampleBuffer)
        let timestampMs = Int(CMTimeGetSeconds(pts) * 1000)
        let result = try handLandmarker.detect(videoFrame: mpImage, timestampInMilliseconds: timestampMs)
        onResult(result, timestampMs)
    }
}
```

#### Livestream

`LIVE_STREAM`Wrap camera `CMSampleBuffer` frames from `AVCaptureVideoDataOutputSampleBufferDelegate` in `32BGRA` pixel format.

```swift
import AVFoundation
import MediaPipeTasksVision
import UIKit

final class CameraFeedService: NSObject, AVCaptureVideoDataOutputSampleBufferDelegate {
    private let handLandmarker: HandLandmarker

    init(handLandmarker: HandLandmarker) {
        self.handLandmarker = handLandmarker
    }

    // Configure AVCaptureVideoDataOutput with kCVPixelFormatType_32BGRA
    func captureOutput(
        _ output: AVCaptureOutput,
        didOutput sampleBuffer: CMSampleBuffer,
        from connection: AVCaptureConnection
    ) {
        guard let mpImage = try? MPImage(sampleBuffer: sampleBuffer, orientation: .up) else { return }

        let pts = CMSampleBufferGetPresentationTimeStamp(sampleBuffer)
        let timestampMs = Int(CMTimeGetSeconds(pts) * 1000)
        try? handLandmarker.detectAsync(image: mpImage, timestampInMilliseconds: timestampMs)
    }
}
```

### Python

pip`pip install mediapipe opencv-python`[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_python.md.txt)·[mp.Image API](https://developers.google.com/edge/api/mediapipe/python/mp/Image.md.txt)

#### Image

`IMAGE`Load an image file directly with `mp.Image.create_from_file()` or wrap a contiguous `uint8` RGB NumPy array.

```python
import cv2
import mediapipe as mp

def detect_from_file(hand_landmarker: mp.tasks.vision.HandLandmarker, path: str):
    mp_image = mp.Image.create_from_file(path)
    return hand_landmarker.detect(mp_image)

def detect_from_opencv_frame(hand_landmarker: mp.tasks.vision.HandLandmarker, path: str):
    bgr_frame = cv2.imread(path)
    if bgr_frame is None:
        raise ValueError(f"Could not decode image: {path}")
    rgb_frame = cv2.cvtColor(bgr_frame, cv2.COLOR_BGR2RGB)
    mp_image = mp.Image(image_format=mp.ImageFormat.SRGB, data=rgb_frame)
    return hand_landmarker.detect(mp_image)
```

#### Video

`VIDEO`Decode frames with `cv2.VideoCapture`, convert `BGR` to `RGB`, and compute monotonic frame timestamps from the stream FPS.

```python
import cv2
import mediapipe as mp

def analyze_video_file(hand_landmarker: mp.tasks.vision.HandLandmarker, video_path: str):
    cap = cv2.VideoCapture(video_path)
    try:
        fps = cap.get(cv2.CAP_PROP_FPS) or 30.0
        frame_index = 0
        while cap.isOpened():
            ok, bgr_frame = cap.read()
            if not ok:
                break
            rgb_frame = cv2.cvtColor(bgr_frame, cv2.COLOR_BGR2RGB)
            mp_image = mp.Image(image_format=mp.ImageFormat.SRGB, data=rgb_frame)
            timestamp_ms = round((frame_index * 1000.0) / fps)
            frame_index += 1
            yield hand_landmarker.detect_for_video(mp_image, timestamp_ms)
    finally:
        cap.release()
```

#### Livestream

`LIVE_STREAM`Read webcam frames with `cv2.VideoCapture(0)`, convert `BGR` to `RGB`, and pass monotonic wall-clock timestamps in milliseconds.

```python
import time
import cv2
import mediapipe as mp

def stream_webcam(hand_landmarker: mp.tasks.vision.HandLandmarker, camera_index: int = 0):
    cap = cv2.VideoCapture(camera_index)
    try:
        while cap.isOpened():
            ok, bgr_frame = cap.read()
            if not ok:
                break
            rgb_frame = cv2.cvtColor(bgr_frame, cv2.COLOR_BGR2RGB)
            mp_image = mp.Image(image_format=mp.ImageFormat.SRGB, data=rgb_frame)
            timestamp_ms = time.monotonic_ns() // 1_000_000
            hand_landmarker.detect_async(mp_image, timestamp_ms)
    finally:
        cap.release()
```

### Flutter

pub`flutter pub add mediapipe_vision`[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_flutter.md.txt)·[Sample code](https://github.com/google/flutter-mediapipe/tree/main/samples/vision/hand_landmarker)

#### Image

`RunningMode.image`Decode a file with `VisionImage.fromFile()` or wrap 8-bit RGB, RGBA, or BGRA pixels with `VisionImage.fromPixels()`.

```dart
import 'package:mediapipe_vision/mediapipe_vision.dart';

final handLandmarker = await HandLandmarker.create(
  HandLandmarkerOptions(
    model: VisionModels.handLandmarker,
    runningMode: RunningMode.image,
  ),
);

// Let MediaPipe decode a file (a path on native platforms, a URL on the
// web), or wrap 8-bit RGB, RGBA, or BGRA pixels that you already have.
final fromFile = VisionImage.fromFile('/path/to/photo.jpg');
final fromPixels = VisionImage.fromPixels(
  pixels: rgba, // exactly height * bytesPerRow bytes
  width: width,
  height: height,
  format: VisionPixelFormat.rgba,
);
final result = await handLandmarker.detect(fromFile);
```

#### Video

`RunningMode.video`Decode frames with your video decoder, wrap each with `VisionImage.fromPixels()`, and pass strictly increasing millisecond timestamps and the video rotation.

```dart
import 'package:mediapipe_vision/mediapipe_vision.dart';

final handLandmarker = await HandLandmarker.create(
  HandLandmarkerOptions(
    model: VisionModels.handLandmarker,
    runningMode: RunningMode.video,
  ),
);

// decodedVideoFrames comes from your video decoder. Timestamps must strictly
// increase; pass the video's rotation so results match the display.
for (final frame in decodedVideoFrames) {
  final result = await handLandmarker.detectForVideo(
    VisionImage.fromPixels(
      pixels: frame.rgba,
      width: frame.width,
      height: frame.height,
      format: VisionPixelFormat.rgba,
    ),
    timestampMilliseconds: frame.timestampMs,
    rotationDegrees: frame.rotationDegrees, // clockwise, a multiple of 90
  );
}
```

#### Livestream

`RunningMode.liveStream`Wrap each camera frame in `VisionImage.deferred()` so conversion (for example, YUV to RGBA) runs only for frames the task processes, and pass the sensor rotation.

```dart
import 'package:mediapipe_vision/mediapipe_vision.dart';

final handLandmarker = await HandLandmarker.create(
  HandLandmarkerOptions(
    model: VisionModels.handLandmarker,
    runningMode: RunningMode.liveStream,
  ),
);
final subscription = handLandmarker.results.listen((result) {
  final hands = result.handLandmarks;
});

// cameraFrames comes from your camera plugin. VisionImage.deferred converts
// a frame (for example, YUV to RGBA) only if the task runs it.
cameraFrames.listen((cameraImage) {
  handLandmarker.detectAsync(
    VisionImage.deferred(() => convertToRgba(cameraImage)),
    timestampMilliseconds: stopwatch.elapsedMilliseconds,
    rotationDegrees: sensorRotationDegrees,
  );
});
```

> [!NOTE]
> **Rotation and region of interest:** On Web, Android, and Python, Vision tasks accept an optional `ImageProcessingOptions` argument per inference call to rotate the frame **clockwise** in 90° increments (`0`, `90`, `180`, or `270`) and---for `ImageClassifier` and `ImageEmbedder` only---crop a normalized `regionOfInterest` (`[0.0, 1.0]`; `ObjectDetector` does not support a region of interest). On iOS, pass a non-mirrored `UIImage.Orientation` (`.up`, `.down`, `.left`, or `.right`) when constructing `MPImage`; mirrored orientations (`.upMirrored`, `.downMirrored`, `.leftMirrored`, and `.rightMirrored`) are not supported. In Flutter, pass `rotationDegrees` (clockwise, a multiple of 90) and, for `ImageClassifier` and `ImageEmbedder`, a `VisionRegionOfInterest` as named arguments of each call.

## Prepare audio input data

Audio tasks automatically resample, buffer, and frame PCM audio to match the
model's sample rate and window size. Input container types depend on the task
and platform:

- **[Audio Classifier](https://developers.google.com/edge/mediapipe/solutions/audio/audio_classifier.md.txt)** uses `AudioData` (a typed PCM float ring buffer paired with `AudioDataFormat`) on Android, iOS, and Python, a `Float32Array` + optional sample rate on Web, or `AudioData` (a `Float32List` with its sample rate) in Flutter. (**[Universal
  Embedder](https://developers.google.com/edge/mediapipe/solutions/retrieval/universal_embedder)** also accepts `AudioData` on Android and iOS, `Float32Array` on Web, `list[float]` in Python, and `Float32List` in Flutter.)
- **[Audio Transcriber](https://developers.google.com/edge/mediapipe/solutions/audio/audio_transcriber.md.txt)** accepts mono `Float32` PCM sample arrays normalized to `[-1.0, 1.0]` directly (`Float32Array` on Web, `float[]` on Android, `UnsafePointer<Float>` on iOS, `np.ndarray` in Python, and `AudioData` in Flutter), resampling automatically when `sampleRateHz` / `sample_rate_hz` is set in the task options. (**[Text-to-Speech](https://developers.google.com/edge/mediapipe/solutions/audio/text_to_speech.md.txt)** is a text-in, audio-out task that accepts UTF-8 text strings and returns normalized 24 kHz mono `Float32` PCM samples.)

| Platform | Input type | Supported media sources \& format notes |
|---|---|---|
| **Web** | `Float32Array` (+ sample rate for `AudioClassifier` / `UniversalEmbedder`, or mono `Float32Array` for `AudioTranscriber`) | Decode audio files with `AudioContext.decodeAudioData()` (`audioBuffer.getChannelData(0)`) or stream live microphone chunks with `AudioWorkletNode`. |
| **Android** | `AudioData` (or `float[]` for `AudioTranscriber`) | Create with `AudioData.create(format, sampleCounts)` and populate from `float[]`, 16-bit `short[]`, or live `android.media.AudioRecord` buffers. |
| **iOS** | `AudioData` \& `MPPFloatBuffer` (or `UnsafePointer<Float>` for `AudioTranscriber`) | Read files with `AVAudioFile` (`.pcmFormatFloat32`) or tap the microphone with `AudioRecord` / `AVAudioEngine`. |
| **Python** | `AudioData` (or `np.ndarray` for `AudioTranscriber`) | Normalize PCM WAV or microphone NumPy arrays to `[-1.0, 1.0]` and call `AudioData.create_from_array(samples, sample_rate)`. |
| **Flutter** | `AudioData` | Create with `AudioData(samples: ..., sampleRate: ..., channels: ...)` from an interleaved `Float32List` in `[-1.0, 1.0]`, or read 16-bit PCM and 32-bit float WAV files with `decodeWav(bytes)`. For live audio, pass microphone blocks from your audio plugin to `classifyAsync` or `transcribe(..., endOfStream: false)`. |

### Audio input code examples

Select your platform and audio mode to prepare complete prerecorded clips or
contiguous live microphone streams (shown here with `AudioClassifier`):

### Web

npm`npm install @mediapipe/tasks-audio`[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_web.md.txt)·[API reference](https://developers.google.com/edge/api/mediapipe/js/tasks-audio.md.txt)

#### Audio clips

`AUDIO_CLIPS`Decode any browser-supported audio file (WAV, MP3, OGG, AAC) into a mono `Float32Array` with `AudioContext.decodeAudioData()`.

```typescript
import type { AudioClassifier, AudioClassifierResult } from "@mediapipe/tasks-audio";

export async function classifyAudioUrl(
  audioClassifier: AudioClassifier,
  audioUrl: string,
): Promise<AudioClassifierResult[]> {
  const audioCtx = new AudioContext({ sampleRate: 16000 });
  try {
    const response = await fetch(audioUrl);
    const audioBuffer = await audioCtx.decodeAudioData(await response.arrayBuffer());
    const pcmSamples: Float32Array = audioBuffer.getChannelData(0);
    return audioClassifier.classify(pcmSamples, audioBuffer.sampleRate);
  } finally {
    await audioCtx.close();
  }
}
```

#### Audio stream

`Web Audio stream`Stream microphone PCM windows through an `AudioWorklet`.

Define an `AudioWorkletProcessor` in its own module that collects fixed-size mono PCM windows (15600 samples = 0.975 s at 16 kHz, the window YAMNet expects) and posts each one to the main thread:

```javascript
class PcmChunkProcessor extends AudioWorkletProcessor {
  constructor({ processorOptions }) {
    super();
    this.buffer = new Float32Array(processorOptions?.windowSize ?? 15600);
    this.offset = 0;
  }
  process(inputs) {
    const input = inputs[0]?.[0];
    if (!input) return true;
    for (const sample of input) {
      this.buffer[this.offset++] = sample;
      if (this.offset === this.buffer.length) {
        this.port.postMessage(this.buffer.slice());
        this.offset = 0;
      }
    }
    return true;
  }
}
registerProcessor("pcm-chunk-processor", PcmChunkProcessor);
```

Register that module with `audioWorklet.addModule()`, route the microphone through it, and classify every window it posts:

```typescript
import type { AudioClassifier, AudioClassifierResult } from "@mediapipe/tasks-audio";

export async function startMicrophoneClassification(
  audioClassifier: AudioClassifier,
  onResult: (results: AudioClassifierResult[]) => void,
): Promise<void> {
  const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
  const audioCtx = new AudioContext({ sampleRate: 16000 });
  await audioCtx.audioWorklet.addModule("pcm-chunk-processor.js");

  const source = audioCtx.createMediaStreamSource(stream);
  const worklet = new AudioWorkletNode(audioCtx, "pcm-chunk-processor");
  worklet.port.onmessage = ({ data: pcmWindow }: MessageEvent<Float32Array>) => {
    onResult(audioClassifier.classify(pcmWindow, audioCtx.sampleRate));
  };
  source.connect(worklet);
}
```

### Android (Kotlin)

Gradle`implementation("com.google.mediapipe:tasks-audio:latest.release")`[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_android.md.txt)·[AudioData API](https://developers.google.com/edge/api/mediapipe/java/com/google/mediapipe/tasks/components/containers/AudioData.md.txt)

#### Audio clips

`AUDIO_CLIPS`Wrap decoded `FloatArray` (`[-1.0, 1.0]`) or 16-bit `ShortArray` PCM samples in `AudioData` with `AudioDataFormat`.

```kotlin
import com.google.mediapipe.tasks.audio.audioclassifier.AudioClassifier
import com.google.mediapipe.tasks.audio.audioclassifier.AudioClassifierResult
import com.google.mediapipe.tasks.components.containers.AudioData
import com.google.mediapipe.tasks.components.containers.AudioData.AudioDataFormat

fun classifyPcmClip(
    audioClassifier: AudioClassifier,
    pcmFloatSamples: FloatArray,
    sampleRateHz: Float = 16000f
): AudioClassifierResult {
    val format = AudioDataFormat.builder()
        .setNumOfChannels(1)
        .setSampleRate(sampleRateHz)
        .build()

    val audioData = AudioData.create(format, pcmFloatSamples.size)
    audioData.load(pcmFloatSamples)
    return audioClassifier.classify(audioData)
}
```

#### Audio stream

`AUDIO_STREAM`Create a 16 kHz mono `AudioRecord` instance and load live microphone frames into `AudioData` on a background thread.

```kotlin
import android.media.AudioRecord
import android.os.SystemClock
import com.google.mediapipe.tasks.audio.audioclassifier.AudioClassifier
import com.google.mediapipe.tasks.components.containers.AudioData
import com.google.mediapipe.tasks.components.containers.AudioData.AudioDataFormat

fun startAudioRecord(
    audioClassifier: AudioClassifier,
    sampleRate: Int = 16000
): Pair<AudioRecord, AudioData> {
    val audioRecord = audioClassifier.createAudioRecord(
        /* numChannels = */ 1,
        /* sampleRate = */ sampleRate,
        /* requiredInputBufferSize = */ sampleRate
    )
    val audioData = AudioData.create(AudioDataFormat.create(audioRecord.format), sampleRate)
    audioRecord.startRecording()
    return audioRecord to audioData
}

// Invoke periodically on a background ScheduledExecutorService
fun pollMicrophoneFrame(
    audioRecord: AudioRecord,
    audioData: AudioData,
    audioClassifier: AudioClassifier
) {
    if (audioData.load(audioRecord) > 0) {
        audioClassifier.classifyAsync(audioData, SystemClock.uptimeMillis())
    }
}
```

### Android (Java)

Gradle`implementation 'com.google.mediapipe:tasks-audio:latest.release'`[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_android.md.txt)·[AudioData API](https://developers.google.com/edge/api/mediapipe/java/com/google/mediapipe/tasks/components/containers/AudioData.md.txt)

#### Audio clips

`AUDIO_CLIPS`Wrap decoded `float[]` (`[-1.0, 1.0]`) or 16-bit `short[]` PCM samples in `AudioData` with `AudioDataFormat`.

```java
import com.google.mediapipe.tasks.audio.audioclassifier.AudioClassifier;
import com.google.mediapipe.tasks.audio.audioclassifier.AudioClassifierResult;
import com.google.mediapipe.tasks.components.containers.AudioData;
import com.google.mediapipe.tasks.components.containers.AudioData.AudioDataFormat;

AudioClassifierResult classifyPcmClip(
    AudioClassifier audioClassifier, float[] pcmFloatSamples, float sampleRateHz) {
  AudioDataFormat format = AudioDataFormat.builder()
      .setNumOfChannels(1)
      .setSampleRate(sampleRateHz)
      .build();

  AudioData audioData = AudioData.create(format, pcmFloatSamples.length);
  audioData.load(pcmFloatSamples);
  return audioClassifier.classify(audioData);
}
```

#### Audio stream

`AUDIO_STREAM`Create a 16 kHz mono `AudioRecord` instance and load live microphone frames into `AudioData` on a background thread.

```java
import android.media.AudioRecord;
import android.os.SystemClock;
import com.google.mediapipe.tasks.audio.audioclassifier.AudioClassifier;
import com.google.mediapipe.tasks.components.containers.AudioData;
import com.google.mediapipe.tasks.components.containers.AudioData.AudioDataFormat;

void streamMicrophoneChunk(AudioClassifier audioClassifier) {
  int sampleRate = 16000;
  AudioRecord audioRecord = audioClassifier.createAudioRecord(
      /* numChannels= */ 1,
      /* sampleRate= */ sampleRate,
      /* requiredInputBufferSize= */ sampleRate
  );
  AudioData audioData = AudioData.create(AudioDataFormat.create(audioRecord.getFormat()), sampleRate);
  audioRecord.startRecording();

  // Poll on a background executor and forward non-empty reads
  if (audioData.load(audioRecord) > 0) {
    audioClassifier.classifyAsync(audioData, SystemClock.uptimeMillis());
  }
}
```

### iOS

SwiftPM`google-ai-edge/mediapipe (MediaPipeTasksAudio)`[Sample code](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/audio_classifier/ios)·[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_ios.md.txt)

#### Audio clips

`AUDIO_CLIPS`Read `.pcmFormatFloat32` samples with `AVAudioFile` and load them into `AudioData` using `MPPFloatBuffer`.

```swift
import AVFoundation
import MediaPipeTasksAudio

func classifyAudioFile(
    at audioURL: URL,
    audioClassifier: AudioClassifier
) throws -> AudioClassifierResult? {
    let audioFile = try AVAudioFile(forReading: audioURL, commonFormat: .pcmFormatFloat32, interleaved: false)
    let frameCount = AVAudioFrameCount(audioFile.length)
    guard let pcmBuffer = AVAudioPCMBuffer(
        pcmFormat: audioFile.processingFormat,
        frameCapacity: frameCount
    ) else { return nil }
    try audioFile.read(into: pcmBuffer)

    let sampleCount = UInt(pcmBuffer.frameLength)
    let format = AudioDataFormat(channelCount: 1, sampleRate: audioFile.processingFormat.sampleRate)
    let audioData = AudioData(format: format, sampleCount: sampleCount)
    let floatBuffer = MPPFloatBuffer(data: pcmBuffer.floatChannelData?[0], length: sampleCount)
    try audioData.load(buffer: floatBuffer, offset: 0, length: sampleCount)
    return try audioClassifier.classify(audioClip: audioData)
}
```

#### Audio stream

`AUDIO_STREAM`Record live microphone audio with MediaPipe's `AudioRecord` wrapper (after requesting record permission) and load into `AudioData`.

```swift
import AVFoundation
import Foundation
import MediaPipeTasksAudio

// Acquire AVAudioSession record permission before calling startRecording()
func startMicrophoneStream(audioClassifier: AudioClassifier) throws -> (AudioRecord, AudioData) {
    let sampleRate = 16000.0
    let bufferLength: UInt = 16000
    let format = AudioDataFormat(channelCount: 1, sampleRate: sampleRate)
    let audioRecord = try AudioRecord(audioDataFormat: format, bufferLength: bufferLength)
    let audioData = AudioData(format: format, sampleCount: bufferLength)
    try audioRecord.startRecording()
    return (audioRecord, audioData)
}

// Invoke periodically on a background dispatch queue
func pollMicrophoneBuffer(
    audioRecord: AudioRecord,
    audioData: AudioData,
    audioClassifier: AudioClassifier
) throws {
    try audioData.load(audioRecord: audioRecord)
    let timestampMs = Int(ProcessInfo.processInfo.systemUptime * 1000)
    try audioClassifier.classifyAsync(audioBlock: audioData, timestampInMilliseconds: timestampMs)
}
```

### Python

pip`pip install mediapipe numpy scipy`[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_python.md.txt)·[AudioData API](https://developers.google.com/edge/api/mediapipe/python/mp/tasks/components/containers/AudioData.md.txt)

#### Audio clips

`AUDIO_CLIPS`Read a WAV file into a normalized `[-1.0, 1.0]` float NumPy array and wrap it with `AudioData.create_from_array()`.

```python
import mediapipe as mp
import numpy as np
from scipy.io import wavfile

AudioData = mp.tasks.components.containers.AudioData

def normalize_wav_to_mono_float32(raw_pcm: np.ndarray) -> np.ndarray:
    if np.issubdtype(raw_pcm.dtype, np.floating):
        samples = raw_pcm.astype(np.float32)
    elif raw_pcm.dtype == np.uint8:
        samples = (raw_pcm.astype(np.float32) - 128.0) / 128.0
    else:
        scale = float(1 << (8 * raw_pcm.dtype.itemsize - 1))
        samples = raw_pcm.astype(np.float32) / scale
    if samples.ndim > 1:
        samples = samples.mean(axis=1)
    return np.clip(samples, -1.0, 1.0)

def classify_wav_file(audio_classifier: mp.tasks.audio.AudioClassifier, wav_path: str):
    sample_rate, raw_pcm = wavfile.read(wav_path)
    samples = normalize_wav_to_mono_float32(raw_pcm)
    audio_data = AudioData.create_from_array(samples, sample_rate)
    return audio_classifier.classify(audio_data)
```

#### Audio stream

`AUDIO_STREAM`Convert each live microphone PCM chunk to normalized `[-1.0, 1.0]` float samples and pass monotonic millisecond timestamps.

```python
import time
import mediapipe as mp
import numpy as np

AudioData = mp.tasks.components.containers.AudioData

def on_microphone_int16_chunk(
    audio_classifier: mp.tasks.audio.AudioClassifier,
    int16_buffer: np.ndarray,
    sample_rate: int = 16000,
) -> None:
    float_chunk = np.clip(int16_buffer.astype(np.float32) / 32768.0, -1.0, 1.0)
    audio_data = AudioData.create_from_array(float_chunk, sample_rate)
    timestamp_ms = time.monotonic_ns() // 1_000_000
    audio_classifier.classify_async(audio_data, timestamp_ms)
```

### Flutter

pub`flutter pub add mediapipe_audio`[Live gallery](https://google.github.io/flutter-mediapipe/)·[Sample code](https://github.com/google/flutter-mediapipe/tree/main/samples/audio/audio_classifier)·[Setup guide](https://developers.google.com/edge/mediapipe/solutions/setup_flutter.md.txt)

#### Audio clips

`AudioRunningMode.audioClips`One call · complete clipClassifies every window of a complete prerecorded audio clip in one call, on a background isolate.

```dart
import 'dart:typed_data';

import 'package:mediapipe_audio/mediapipe_audio.dart';

Future<void> classifyClip(Uint8List wavBytes) async {
  final classifier = await AudioClassifier.create(
    AudioClassifierOptions(model: AudioModels.yamnet, maxResults: 3),
  );
  try {
    final clip = decodeWav(wavBytes);
    for (final chunk in await classifier.classify(clip)) {
      final top = chunk.classifications.first.categories.first;
      print('${chunk.timestampMilliseconds} ms: ${top.categoryName}');
    }
  } finally {
    await classifier.dispose();
  }
}
```

#### Audio stream

`AudioRunningMode.audioStream`Async · results streamEach call returns at once and results arrive on the task's `results` stream. Frames contiguous PCM blocks into the model's windows; every block is classified, in order.

```dart
import 'dart:typed_data';

import 'package:mediapipe_audio/mediapipe_audio.dart';

/// Prints the top sound of each window of [blocks], 16 kHz mono audio.
Future<void> listen(Stream<Float32List> blocks) async {
  final classifier = await AudioClassifier.create(
    AudioClassifierOptions(
      model: AudioModels.yamnet,
      maxResults: 1,
      runningMode: AudioRunningMode.audioStream,
    ),
  );
  classifier.results.listen((window) {
    final top = window.classifications.first.categories.first;
    print('${window.timestampMilliseconds} ms: ${top.categoryName}');
  });
  var samples = 0;
  try {
    await for (final block in blocks) {
      classifier.classifyAsync(
        AudioData(samples: block, sampleRate: 16000),
        // Where the block starts, counted from the samples before it.
        timestampMilliseconds: samples * 1000 ~/ 16000,
      );
      samples += block.length;
    }
  } finally {
    // Classifies the audio short of a window, then closes `results`.
    await classifier.dispose();
  }
}
```

> [!NOTE]
> **Audio permissions and threading:** Always request microphone permission (`android.permission.RECORD_AUDIO` on Android, `NSMicrophoneUsageDescription` + `AVAudioSession.requestRecordPermission` on iOS, or a user gesture before `getUserMedia` on Web) before starting live audio capture. Keep audio capture callbacks non-blocking by forwarding PCM buffers onto a dedicated worker queue for inference.