# ImageClassifier

    class ImageClassifier: NSObject

Performs classification on images.

The API expects a TFLite model with optional, but strongly recommended, \[TFLite Model Metadata.\](https://www.tensorflow.org/lite/convert/metadata").

The API supports models with one image input tensor and one or more output tensors. To be more specific, here are the requirements.

Input tensor (kTfLiteUInt8/kTfLiteFloat32) - image input of size `[batch x height x width x channels]`. - batch inference is not supported (`batch` is required to be 1). - only RGB inputs are supported (`channels` is required to be 3). - if type is kTfLiteFloat32, NormalizationOptions are required to be attached to the metadata for input normalization.

At least one output tensor with: (kTfLiteUInt8/kTfLiteFloat32) - `N `classes and either 2 or 4 dimensions, i.e. `[1 x N]` or `[1 x 1 x 1 x N]` - optional (but recommended) label map(s) as AssociatedFiles with type TENSOR_AXIS_LABELS, containing one label per line. The first such AssociatedFile (if any) is used to fill the `class_name` field of the results. The `display_name` field is filled from the AssociatedFile (if any) whose locale matches the `display_names_locale` field of the `ImageClassifierOptions` used at creation time ("en" by default, i.e. English). If none of these are available, only the `index` field of the results will be filled. - optional score calibration can be attached using ScoreCalibrationOptions and an AssociatedFile with type TENSOR_AXIS_SCORE_CALIBRATION. See metadata_schema.fbs \[1\] for more details.
- `
  ### [init(modelPath:)](https://developers.google.com/edge/api/mediapipe/swift/vision/Classes/ImageClassifier.md.txt#/c:objc(cs)MPPImageClassifier(im)initWithModelPath:error:)

  ` Creates a new instance of `ImageClassifier` from an absolute path to a TensorFlow Lite model file stored locally on the device and the default `ImageClassifierOptions`.

  #### Declaration

  Swift

      convenience init(modelPath: String) throws

  #### Parameters

  |---|---|
  | `modelPath` | An absolute path to a TensorFlow Lite model file stored locally on the device. |

  #### Return Value

  A new instance of `ImageClassifier` with the given model path. `nil` if there is an error in initializing the image classifier.
- `
  ### [init(options:)](https://developers.google.com/edge/api/mediapipe/swift/vision/Classes/ImageClassifier.md.txt#/c:objc(cs)MPPImageClassifier(im)initWithOptions:error:)

  ` Creates a new instance of `ImageClassifier` from the given `ImageClassifierOptions`.

  #### Declaration

  Swift

      init(options: ImageClassifierOptions) throws

  #### Parameters

  |---|---|
  | `options` | The options of type `ImageClassifierOptions` to use for configuring the `ImageClassifier`. |

  #### Return Value

  A new instance of `ImageClassifier` with the given options. `nil` if there is an error in initializing the image classifier.
- `
  ### [classify(image:)](https://developers.google.com/edge/api/mediapipe/swift/vision/Classes/ImageClassifier.md.txt#/c:objc(cs)MPPImageClassifier(im)classifyImage:error:)

  ` Performs image classification on the provided `MPImage` using the whole image as region of interest. Rotation will be applied according to the `orientation` property of the provided `MPImage`. Only use this method when the `ImageClassifier` is created with running mode, `.image`.

  This method supports classification of RGBA images. If your `MPImage` has a source type of `.pixelBuffer` or `.sampleBuffer`, the underlying pixel buffer must use `kCVPixelFormatType_32BGRA` as its pixel format.

  If your `MPImage` has a source type of `.image` ensure that the color space is RGB with an Alpha channel.

  #### Declaration

  Swift

      func classify(image: MPImage) throws -> ImageClassifierResult

  #### Parameters

  |---|---|
  | `image` | The `MPImage` on which image classification is to be performed. |

  #### Return Value

  An `ImageClassifierResult` object that contains a list of image classifications.
- `
  ### [classify(image:regionOfInterest:)](https://developers.google.com/edge/api/mediapipe/swift/vision/Classes/ImageClassifier.md.txt#/c:objc(cs)MPPImageClassifier(im)classifyImage:regionOfInterest:error:)

  ` Performs image classification on the provided `MPImage` cropped to the specified region of interest. Rotation will be applied on the cropped image according to the `orientation` property of the provided `MPImage`. Only use this method when the `ImageClassifier` is created with running mode, `.image`.

  This method supports classification of RGBA images. If your `MPImage` has a source type of `.pixelBuffer` or `.sampleBuffer`, the underlying pixel buffer must use `kCVPixelFormatType_32BGRA` as its pixel format.

  If your `MPImage` has a source type of `.image` ensure that the color space is RGB with an Alpha channel.

  #### Declaration

  Swift

      func classify(image: MPImage, regionOfInterest roi: CGRect) throws -> ImageClassifierResult

  #### Parameters

  |---|---|
  | `image` | The `MPImage` on which image classification is to be performed. |
  | `roi` | A `CGRect` specifying the region of interest within the given `MPImage`, on which image classification should be performed. |

  #### Return Value

  An `ImageClassifierResult` object that contains a list of image classifications.
- `
  ### [classify(videoFrame:timestampInMilliseconds:)](https://developers.google.com/edge/api/mediapipe/swift/vision/Classes/ImageClassifier.md.txt#/c:objc(cs)MPPImageClassifier(im)classifyVideoFrame:timestampInMilliseconds:error:)

  ` Performs image classification on the provided video frame of type `MPImage` using the whole image as region of interest. Rotation will be applied according to the `orientation` property of the provided `MPImage`. Only use this method when the `ImageClassifier` is created with running mode `.video`.

  It's required to provide the video frame's timestamp (in milliseconds). The input timestamps must be monotonically increasing.

  This method supports classification of RGBA images. If your `MPImage` has a source type of `.pixelBuffer` or `.sampleBuffer`, the underlying pixel buffer must use `kCVPixelFormatType_32BGRA` as its pixel format.

  If your `MPImage` has a source type of `.image` ensure that the color space is RGB with an Alpha channel.

  #### Declaration

  Swift

      func classify(videoFrame image: MPImage, timestampInMilliseconds: Int) throws -> ImageClassifierResult

  #### Parameters

  |---|---|
  | `image` | The `MPImage` on which image classification is to be performed. |
  | `timestampInMilliseconds` | The video frame's timestamp (in milliseconds). The input timestamps must be monotonically increasing. |

  #### Return Value

  An `ImageClassifierResult` object that contains a list of image classifications.
- `
  ### [classify(videoFrame:timestampInMilliseconds:regionOfInterest:)](https://developers.google.com/edge/api/mediapipe/swift/vision/Classes/ImageClassifier.md.txt#/c:objc(cs)MPPImageClassifier(im)classifyVideoFrame:timestampInMilliseconds:regionOfInterest:error:)

  ` Performs image classification on the provided video frame of type `MPImage` cropped to the specified region of interest. Rotation will be applied according to the `orientation` property of the provided `MPImage`. Only use this method when the `ImageClassifier` is created with `.video`.

  It's required to provide the video frame's timestamp (in milliseconds). The input timestamps must be monotonically increasing.

  This method supports classification of RGBA images. If your `MPImage` has a source type of `.pixelBuffer` or `.sampleBuffer`, the underlying pixel buffer must use `kCVPixelFormatType_32BGRA` as its pixel format.

  If your `MPImage` has a source type of `.image` ensure that the color space is RGB with an Alpha channel.

  #### Declaration

  Swift

      func classify(videoFrame image: MPImage, timestampInMilliseconds: Int, regionOfInterest roi: CGRect) throws -> ImageClassifierResult

  #### Parameters

  |---|---|
  | `image` | A live stream image data of type `MPImage` on which image classification is to be performed. |
  | `timestampInMilliseconds` | The video frame's timestamp (in milliseconds). The input timestamps must be monotonically increasing. |
  | `roi` | A `CGRect` specifying the region of interest within the video frame of type `MPImage`, on which image classification should be performed. |

  #### Return Value

  An `ImageClassifierResult` object that contains a list of image classifications.
- `
  ### [classifyAsync(image:timestampInMilliseconds:)](https://developers.google.com/edge/api/mediapipe/swift/vision/Classes/ImageClassifier.md.txt#/c:objc(cs)MPPImageClassifier(im)classifyAsyncImage:timestampInMilliseconds:error:)

  ` Sends live stream image data of type `MPImage` to perform image classification using the whole image as region of interest. Rotation will be applied according to the `orientation` property of the provided `MPImage`. Only use this method when the `ImageClassifier` is created with running mode `.liveStream`.

  The object which needs to be continuously notified of the available results of image classification must confirm to `ImageClassifierLiveStreamDelegate` protocol and implement the `imageClassifier(_:didFinishClassificationWithResult:timestampInMilliseconds:error:)` delegate method.

  It's required to provide a timestamp (in milliseconds) to indicate when the input image is sent to the image classifier. The input timestamps must be monotonically increasing.

  This method supports classification of RGBA images. If your `MPImage` has a source type of `.pixelBuffer` or `.sampleBuffer`, the underlying pixel buffer must use `kCVPixelFormatType_32BGRA` as its pixel format.

  If the input `MPImage` has a source type of `.image` ensure that the color space is RGB with an Alpha channel.

  If this method is used for classifying live camera frames using `AVFoundation`, ensure that you request `AVCaptureVideoDataOutput` to output frames in `kCMPixelFormat_32BGRA` using its `videoSettings` property.

  #### Declaration

  Swift

      func classifyAsync(image: MPImage, timestampInMilliseconds: Int) throws

  #### Parameters

  |---|---|
  | `image` | A live stream image data of type `MPImage` on which image classification is to be performed. |
  | `timestampInMilliseconds` | The timestamp (in milliseconds) which indicates when the input image is sent to the image classifier. The input timestamps must be monotonically increasing. |

  #### Return Value

  `true` if the image was sent to the task successfully, otherwise `false`.
- `
  ### [classifyAsync(image:timestampInMilliseconds:regionOfInterest:)](https://developers.google.com/edge/api/mediapipe/swift/vision/Classes/ImageClassifier.md.txt#/c:objc(cs)MPPImageClassifier(im)classifyAsyncImage:timestampInMilliseconds:regionOfInterest:error:)

  ` Sends live stream image data of type `MPImage` to perform image classification, cropped to the specified region of interest.. Rotation will be applied according to the `orientation` property of the provided `MPImage`. Only use this method when the `ImageClassifier` is created with `.liveStream`.

  The object which needs to be continuously notified of the available results of image classification must confirm to `ImageClassifierLiveStreamDelegate` protocol and implement the `imageClassifier(_:didFinishClassificationWithResult:timestampInMilliseconds:error:)` delegate method.

  It's required to provide a timestamp (in milliseconds) to indicate when the input image is sent to the image classifier. The input timestamps must be monotonically increasing.

  This method supports classification of RGBA images. If your `MPImage` has a source type of `.pixelBuffer` or `.sampleBuffer`, the underlying pixel buffer must use `kCVPixelFormatType_32BGRA` as its pixel format.

  If the input `MPImage` has a source type of `.image` ensure that the color space is RGB with an Alpha channel.

  If this method is used for classifying live camera frames using `AVFoundation`, ensure that you request `AVCaptureVideoDataOutput` to output frames in `kCMPixelFormat_32BGRA` using its `videoSettings` property.

  #### Declaration

  Swift

      func classifyAsync(image: MPImage, timestampInMilliseconds: Int, regionOfInterest roi: CGRect) throws

  #### Parameters

  |---|---|
  | `image` | A live stream image data of type `MPImage` on which image classification is to be performed. |
  | `timestampInMilliseconds` | The timestamp (in milliseconds) which indicates when the input image is sent to the image classifier. The input timestamps must be monotonically increasing. |
  | `roi` | A `CGRect` specifying the region of interest within the given live stream image data of type `MPImage`, on which image classification should be performed. |

  #### Return Value

  `true` if the image was sent to the task successfully, otherwise `false`.