BaseAudioTaskApi

public class BaseAudioTaskApi implements AutoCloseable

Known direct subclasses
AudioClassifier

Performs audio classification on audio clips or audio stream.


The base class of MediaPipe audio tasks.

Summary

Public constructors

BaseAudioTaskApi(
    TaskRunner runner,
    RunningMode runningMode,
    String audioStreamName,
    String sampleRateStreamName
)

Constructor to initialize a BaseAudioTaskApi.

Public methods

void

Closes and cleans up the MediaPipe audio task.

AudioRecord

Creates an android.media.AudioRecord instance to record audio stream that has mono channel at sample rate at sample rate 16kHz, the sample rate required for models like Yamnet.

AudioRecord
createAudioRecord(
    int numChannels,
    int sampleRate,
    int requiredInputBufferSize
)

Creates an android.media.AudioRecord instance to record audio stream.

Protected methods

void
checkOrSetSampleRate(double sampleRate)

Checks or sets the audio sample rate in the audio stream mode.

TaskResult

A synchronous method to process audio clips.

void
sendAudioStreamData(AudioData audioClip, long timestampMs)

An asynchronous method to send audio stream data to the TaskRunner.

Public constructors

BaseAudioTaskApi

public BaseAudioTaskApi(
    TaskRunner runner,
    RunningMode runningMode,
    String audioStreamName,
    String sampleRateStreamName
)

Constructor to initialize a BaseAudioTaskApi.

Parameters
TaskRunner runner

a TaskRunner.

RunningMode runningMode

a mediapipe audio task RunningMode.

String audioStreamName

the name of the input audio stream.

String sampleRateStreamName

the name of the audio sample rate stream.

Public methods

close

public void close()

Closes and cleans up the MediaPipe audio task.

createAudioRecord

public AudioRecord createAudioRecord()

Creates an android.media.AudioRecord instance to record audio stream that has mono channel at sample rate at sample rate 16kHz, the sample rate required for models like Yamnet. The returned AudioRecord instance is initialized and client needs to call startRecording method to start recording.

Throws
java.lang.IllegalArgumentException

if the model required channel count is unsupported

java.lang.IllegalStateException

if AudioRecord instance failed to initialize

createAudioRecord

public AudioRecord createAudioRecord(
    int numChannels,
    int sampleRate,
    int requiredInputBufferSize
)

Creates an android.media.AudioRecord instance to record audio stream. The returned AudioRecord instance is initialized and client needs to call startRecording method to start recording.

Note that MediaPipe Audio tasks will up/down sample automatically to fit the sample rate required by the model. The default sample rate of the MediaPipe pretrained audio model, Yamnet, is 16kHz.

Parameters
int numChannels

the number of audio channels.

int sampleRate

the audio sample rate.

int requiredInputBufferSize

the required input buffer size in number of float elements.

Throws
java.lang.IllegalArgumentException

if the model required channel count is unsupported

java.lang.IllegalStateException

if AudioRecord instance failed to initialize

Protected methods

checkOrSetSampleRate

protected void checkOrSetSampleRate(double sampleRate)

Checks or sets the audio sample rate in the audio stream mode.

Parameters
double sampleRate

the audio sample rate.

Throws
com.google.mediapipe.framework.MediaPipeException

if the task is not in the audio stream mode or the provided sample rate is inconsistent with the previously received.

processAudioClip

protected TaskResult processAudioClip(AudioData audioClip)

A synchronous method to process audio clips. The call blocks the current thread until a failure status or a successful result is returned.

Parameters
AudioData audioClip

a MediaPipe AudioDatra object for processing.

Throws
com.google.mediapipe.framework.MediaPipeException

if the task is not in the audio clips mode.

sendAudioStreamData

protected void sendAudioStreamData(AudioData audioClip, long timestampMs)

An asynchronous method to send audio stream data to the TaskRunner. The results will be available in the user-defined result listener.

Parameters
AudioData audioClip

a MediaPipe AudioDatra object for processing.

long timestampMs

the corresponding timestamp of the input image in milliseconds.

Throws
com.google.mediapipe.framework.MediaPipeException

if the task is not in the stream mode.