SpeechRecognizer

Kotlin |Java

public final class SpeechRecognizer implements Closeable

Provides an interface for performing on-device speech recognition.

It supports audio input from microphone, files, URIs, and file descriptors via AudioSource.

Typical usage:

Example 1: Recognizing from Microphone

// 1. Create a SpeechRecognizer with desired options.
val options: SpeechRecognizerOptions =
    speechRecognizerOptions {
        locale = Locale.US
        preferredMode = SpeechRecognizerOptions.Mode.MODE_ADVANCED
    }
val speechRecognizer: SpeechRecognizer = SpeechRecognition.getClient(options)

// 2. Check if the recognition model is available or needs downloading.
launch {
    val status: Int = speechRecognizer.checkStatus()
    if (status == FeatureStatus.DOWNLOADABLE) {
        // 3. If needed, download the model.
        speechRecognizer.download().collect { downloadStatus ->
            when (downloadStatus) {
                is DownloadStatus.DownloadCompleted -> {
                    // Model is ready, start recognition.
                    startMyRecognition(speechRecognizer)
                }
                is DownloadStatus.DownloadFailed -> {
                    // Handle download failure.
                }
                is DownloadStatus.DownloadProgress -> {
                    // Handle download progress.
                }
                else -> {}
            }
        }
    } else if (status == FeatureStatus.AVAILABLE) {
        // Model is ready, start recognition.
        startMyRecognition(speechRecognizer)
    } else {
        // Handle other statuses like DOWNLOADING or UNAVAILABLE.
    }
}

// 4. Define your recognition logic.
suspend fun startMyRecognition(recognizer: SpeechRecognizer) {
    // Create a request and collect the responses.
    val request = speechRecognizerRequest {
        audioSource = AudioSource.fromMic()
    }
    recognizer.startRecognition(request).collect { response ->
        // Process the responses.
        when (response) {
            is SpeechRecognizerResponse.PartialTextResponse -> { /* ... */}
            is SpeechRecognizerResponse.FinalTextResponse -> { /* ... */}
            else -> {}
        }
    }
}

// 5. Stop recognition when needed.
launch {
    speechRecognizer.stopRecognition()
    speechRecognizer.close()
}

Example 2: Recognizing from a File Descriptor in Streaming Mode (using a Pipe)

When using a ParcelFileDescriptor as the audio source in streaming mode, you must pace the writes to the file descriptor to match real-time playback (e.g., 3200 bytes every 100ms for 16kHz, 16-bit, mono PCM). Writing too fast may overwhelm the buffer.

suspend fun recognizeFromFile(recognizer: SpeechRecognizer, audioData: ByteArray) {
    val (readFd, writeFd) = ParcelFileDescriptor.createPipe()

    // Launch a coroutine to write to the pipe in a background thread (Dispatchers.IO)
    val writeJob = CoroutineScope(Dispatchers.IO).launch {
        ParcelFileDescriptor.AutoCloseOutputStream(writeFd).use { outputStream ->
            // Example for STREAMING mode: pace the writes to simulate real-time.
            // (3200 bytes is 100ms of 16kHz 16-bit mono PCM: 16000 samples/sec * 2 bytes/sample * 0.1 sec)
            val chunkSize = 3200
            var offset = 0
            while (offset < audioData.size && isActive) {
                val len = minOf(chunkSize, audioData.size - offset)
                outputStream.write(audioData, offset, len)
                offset += len
                delay(100) // Pace the stream
            }
        }
    }

    val audioSource = AudioSource.fromPfd(readFd, AudioSource.Mode.STREAMING)
    val request = speechRecognizerRequest {
        this.audioSource = audioSource
    }

    try {
        recognizer.startRecognition(request).collect { response ->
            when (response) {
                is SpeechRecognizerResponse.PartialTextResponse -> {
                    println("Partial: ${response.text}")
                }
                is SpeechRecognizerResponse.FinalTextResponse -> {
                    println("Final: ${response.text}")
                }
                else -> {}
            }
        }
    } finally {
        // Ensure the write job is cancelled and the audioSource is closed
        writeJob.cancel()
        audioSource.close()
    }
}

Example 3: Recognizing Pre-Recorded Audio in One-Shot Mode

For pre-recorded audio files or batch audio buffers, provide an AudioSource in one-shot mode (e.g. via AudioSource.fromFile). In one-shot mode, the recognizer processes the entire audio input at once before producing the final result.

// 1. Create a SpeechRecognizer.
val options = speechRecognizerOptions {
    locale = Locale.US
    preferredMode = SpeechRecognizerOptions.Mode.MODE_BASIC
}
val speechRecognizer = SpeechRecognition.getClient(options)

// 2. Recognize from a File in ONE_SHOT mode.
suspend fun recognizeFromFile(recognizer: SpeechRecognizer, audioFile: File) {
    val audioSource = AudioSource.fromFile(audioFile)
    val request = speechRecognizerRequest {
        this.audioSource = audioSource
    }

    try {
        recognizer.startRecognition(request).collect { response ->
            when (response) {
                is SpeechRecognizerResponse.FinalTextResponse -> {
                    println("Final: ${response.text}")
                }
                else -> {}
            }
        }
    } finally {
        audioSource.close()
    }
}

// 3. Close the recognizer when no longer needed.
speechRecognizer.close()

Summary

Public constructors
`@VisibleForTesting SpeechRecognizer( @NonNull SpeechRecognizerOptions options, @NonNull MLKitStatsLogger mlKitStatsLogger, GoogleAsrServiceGrpcKt.GoogleAsrServiceCoroutineStub coroutineStub )`

Public methods
`final @FeatureStatus int`	`checkStatus()` Checks the current availability status of the speech recognition feature.
`void`	`close()` Releases resources associated with the speech recognition engine.
`final @NonNull Flow<@NonNull DownloadStatus>`	`@OptIn(markerClass = [ExperimentalAtomicApi]) download()` Downloads the required model assets for the speech recognition feature if they are not already available.
`final @FeatureStatus int`	`@VisibleForTesting getFeatureStatus(@NonNull ModelAvailabilityResponse response)`
`final @NonNull Flow<@NonNull SpeechRecognizerResponse>`	`@OptIn(markerClass = [ExperimentalAtomicApi]) startRecognition(@NonNull SpeechRecognizerRequest request)` Starts a speech recognition session when the returned `Flow` is collected.
`final void`	`stopRecognition()` Stops the current speech recognition session.
`final void`	`warmup()` Prepares the speech recognition engine by loading required resources in advance to reduce latency for subsequent recognition requests.

Public constructors

SpeechRecognizer

@VisibleForTesting
public SpeechRecognizer(
    @NonNull SpeechRecognizerOptions options,
    @NonNull MLKitStatsLogger mlKitStatsLogger,
    GoogleAsrServiceGrpcKt.GoogleAsrServiceCoroutineStub coroutineStub
)

Public methods

checkStatus

public final @FeatureStatus int checkStatus()

Checks the current availability status of the speech recognition feature.

Returns
`@FeatureStatus int`	`FeatureStatus` indicating feature readiness.

close

public void close()

Releases resources associated with the speech recognition engine.

This should be called when the speech recognizer is no longer needed.

download

@OptIn(markerClass = [ExperimentalAtomicApi])
public final @NonNull Flow<@NonNull DownloadStatus> download()

Downloads the required model assets for the speech recognition feature if they are not already available.

Use this method to proactively download models before inference. A flow of DownloadStatus reports progress and completion status.

Returns
`@NonNull Flow<@NonNull DownloadStatus>`	`Flow` for collecting to start the download and receive `DownloadStatus`.

getFeatureStatus

@VisibleForTesting
public final @FeatureStatus int getFeatureStatus(@NonNull ModelAvailabilityResponse response)

startRecognition

@OptIn(markerClass = [ExperimentalAtomicApi])
public final @NonNull Flow<@NonNull SpeechRecognizerResponse> startRecognition(@NonNull SpeechRecognizerRequest request)

Starts a speech recognition session when the returned Flow is collected.

Downstream collectors receive streaming updates such as SpeechRecognizerResponse.PartialTextResponse, SpeechRecognizerResponse.FinalTextResponse, and SpeechRecognizerResponse.AudioLevelResponse. Runtime recognition errors are delivered as SpeechRecognizerResponse.ErrorResponse items within the flow rather than thrown as exceptions during collection, and SpeechRecognizerResponse.CompletedResponse is emitted upon normal completion.

Parameters
`@NonNull SpeechRecognizerRequest request`	the `SpeechRecognizerRequest` containing the audio source and other configuration.

Returns
`@NonNull Flow<@NonNull SpeechRecognizerResponse>`	A `Flow` of `SpeechRecognizerResponse`.

Throws
`IllegalArgumentException`	if word timing is enabled with `AudioSource.Mode.ONE_SHOT`.

stopRecognition

public final void stopRecognition()

Stops the current speech recognition session.

This method gracefully terminates the recognition job. The SpeechRecognizerResponse.CompletedResponse will be sent after the job is fully closed.

warmup

public final void warmup()

Prepares the speech recognition engine by loading required resources in advance to reduce latency for subsequent recognition requests.