# Classes The following declarations are available globally. - ` ### [AudioClassifier](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/AudioClassifier.md.txt) ` Class that performs classification on audio data. This API expects a TFLite model with mandatory TFLite Model Metadata that contains the mandatory AudioProperties of the solo input audio tensor and the optional (but recommended) category labels as AssociatedFiles with type TENSOR_AXIS_LABELS per output classification tensor. Input tensor (kTfLiteFloat32) - input audio buffer of size `[batch * samples]`. - batch inference is not supported (`batch` is required to be 1). - for multi-channel models, the channels must be interleaved. At least one output tensor with: (kTfLiteFloat32) - `[1 x N]` array with `N` represents the number of categories. - optional (but recommended) category labels as AssociatedFiles with type TENSOR_AXIS_LABELS, containing one label per line. The first such AssociatedFile (if any) is used to fill the `category_name` field of the results. The `display_name` field is filled from the AssociatedFile (if any) whose locale matches the `display_names_locale` field of the `AudioClassifierOptions` used at creation time ("en" by default, i.e. English). If none of these are available, only the `index` field of the results will be filled. #### Declaration Swift class AudioClassifier: NSObject - ` ### [AudioClassifierOptions](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/AudioClassifierOptions.md.txt) ` Options for setting up a `AudioClassifier`. #### Declaration Swift class AudioClassifierOptions: TaskOptions, NSCopying - ` ### [AudioClassifierResult](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/AudioClassifierResult.md.txt) ` Represents the classification results generated by `AudioClassifier`. \* #### Declaration Swift class AudioClassifierResult: TaskResult, NSCopying - ` ### [AudioData](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/AudioData.md.txt) ` A wrapper class for input audio samples used in on-device machine learning using MediaPipe Task library. Internally this class uses a ring buffer to hold input audio data. Clients could feed input audio data via `load` method and access the aggregated audio samples via `buffer` property. Note that this class can only handle input audio in Float. TODO: Investigate if `AVAudioFormatInt32` should be supported. Only necessary if you can change the input format of `AVAudioEngine` or load audio files using `AVAudioFormatInt32`. #### Declaration Swift class AudioData: NSObject - ` ### [AudioDataFormat](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/AudioDataFormat.md.txt) ` Wraps properties describing the format of the incoming audio samples, namely number of channels and the sample rate. #### Declaration Swift class AudioDataFormat: NSObject - ` ### [AudioPart](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/AudioPart.md.txt) ` Audio part representing the audio content block of a multi-modal block. #### Declaration Swift class AudioPart: TaskPart - ` ### [AudioRecord](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/AudioRecord.md.txt) ` A wrapper class to record the device's microphone continuously. Currently this class only supports recording up to 2 channels. If the number of channels is 2, then the mono microphone input is duplicated to provide dual channel data. Internally the class manages an instance of `AVAudioEngine` for tapping the microphone samples. Recording sets the following parameters on the `AVAudioSession.sharedInstance()`: 1. `category` = .playAndRecord 2. `mode` = .default 3. `categoryOptions` = \[.overrideMutedMicrophoneInterruption\] If you need more control, use the native `AVAudioEngine` directly for tapping the microphone. `AudioRecord` does not actively monitor audio interruptions, route changes, resetting of media services etc. They can be handled in the app using the notifications provided by `AVFAudio`. Refer to the official docs for the following: 1. Handling audio interruptions https://developer.apple.com/documentation/avfaudio/handling_audio_interruptions 2. Responding to audio route changes https://developer.apple.com/documentation/avfaudio/responding_to_audio_route_change 3. Responding to resetting of media services https://developer.apple.com/documentation/avfaudio/avaudiosessionmediaserviceswereresetnotification iOS may automatically stop the audio engine in some cases. A few examples of such events are a system interrupt, route change, etc. In such cases, `read(offset:length:)` returns `TasksErrorCode.audioRecordNotTappingMicError`. You can restart recording using `startRecording()`. Note: Avoid setting the `AVAudioSession.sharedInstance()` category, mode, options, channel configuration (enabling stereo) or other parameters while audio is being actively recorded using the audio record. Setting these parameters or changing the configuration of `AVAudioSession.sharedInstance()` while a recording is in progress will result in undefined behaviour. #### Declaration Swift class AudioRecord: NSObject - ` ### [AudioTranscriber](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/AudioTranscriber.md.txt) ` Performs automatic speech recognition (ASR) on streaming mono PCM float audio samples. #### Declaration Swift class AudioTranscriber: NSObject - ` ### [AudioTranscriberOptions](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/AudioTranscriberOptions.md.txt) ` Options for configuring a MediaPipe `MPPAudioTranscriber` task. #### Declaration Swift class AudioTranscriberOptions: TaskOptions, NSCopying - ` ### [AudioTranscriberResult](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/AudioTranscriberResult.md.txt) ` Represents the transcription results generated by `MPPAudioTranscriber`. #### Declaration Swift class AudioTranscriberResult: TaskResult, NSCopying - ` ### [AudioTranscriberSegment](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/AudioTranscriberSegment.md.txt) ` A single transcribed segment emitted by `MPPAudioTranscriber`. #### Declaration Swift class AudioTranscriberSegment: NSObject - ` ### [BaseOptions](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/BaseOptions.md.txt) ` Holds the base options that is used for creation of any type of task. It has fields with important information acceleration configuration, TFLite model source etc. #### Declaration Swift class BaseOptions: NSObject, NSCopying - ` ### [ClassificationResult](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/ClassificationResult.md.txt) ` Represents the classification results of a model. Typically used as a result for classification tasks. #### Declaration Swift class ClassificationResult: NSObject - ` ### [Classifications](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/Classifications.md.txt) ` Represents the list of classification for a given classifier head. Typically used as a result for classification tasks. #### Declaration Swift class Classifications: NSObject - ` ### [Embedding](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/Embedding.md.txt) ` Represents the embedding for a given embedder head. Typically used in embedding tasks. One and only one of the two 'floatEmbedding' and 'quantizedEmbedding' will contain data, based on whether or not the embedder was configured to perform scala quantization. #### Declaration Swift class Embedding: NSObject - ` ### [EmbeddingResult](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/EmbeddingResult.md.txt) ` Represents the embedding results of a model. Typically used as a result for embedding tasks. #### Declaration Swift class EmbeddingResult: NSObject - ` ### [ImagePart](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/ImagePart.md.txt) ` Image part representing the image content block of a multi-modal block. #### Declaration Swift class ImagePart: TaskPart - ` ### [MPPFloatBuffer](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/MPPFloatBuffer.md.txt) ` A wrapper class to store pointer to a float array and its size. #### Declaration Swift class MPPFloatBuffer: NSObject, NSCopying - ` ### [ResultCategory](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/ResultCategory.md.txt) ` Category is a util class that contains a label, its display name, a float value as score, and the index of the label in the corresponding label file. Typically it's used as the result of classification tasks. #### Declaration Swift class ResultCategory: NSObject - ` ### [TTSStream](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/TTSStream.md.txt) ` Bidirectional / pull-based streaming synthesis handle supporting Level 2 incremental LLM text pushing (`pushText` + `finishInput`) alongside `readNext()` and `cancel()`. #### Declaration Swift final class TTSStream: Sendable - ` ### [TaskOptions](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/TaskOptions.md.txt) ` MediaPipe Tasks options base class. Any MediaPipe task-specific options class should extend this class. #### Declaration Swift class TaskOptions: NSObject, NSCopying - ` ### [TaskPart](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/TaskPart.md.txt) ` Represents a part of a multi-modal content block. #### Declaration Swift class TaskPart: NSObject - ` ### [TaskResult](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/TaskResult.md.txt) ` MediaPipe Tasks result base class. Any MediaPipe task result class should extend this class. #### Declaration Swift class TaskResult: NSObject, NSCopying - ` ### [TextPart](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/TextPart.md.txt) ` Text part representing the text content block of a multi-modal block. #### Declaration Swift class TextPart: TaskPart - ` ### [TextToSpeech](https://developers.google.com/edge/api/mediapipe/swift/audio/Classes/TextToSpeech.md.txt) ` MediaPipe TextToSpeech task interface for Swift (iOS / macOS). #### Declaration Swift final class TextToSpeech: Sendable