MLKitVision Framework Reference

AI-generated Key Takeaways

• MLKVisionImage is an object used for vision detection tasks, accepting either images or image buffers as input.

• It's crucial to set the imageOrientation property of input UIImage to .up and ensure it has a non-NULL CGImage.

• When using CMSampleBufferRef, the buffer must contain uncompressed pixel data in specific supported formats.

• For performance, minimize the creation and lifespan of MLKVisionImage instances, especially when initialized with CMSampleBufferRef.

• MLKVisionImage offers a property orientation to specify the display orientation of the image, defaulting to .up.

MLKVisionImage

@interface MLKVisionImage : NSObject <MLKCompatibleImage>

An image or image buffer used for vision detection.

• orientation

  The display orientation of the image. The default is .up.

  Declaration

  Objective-C

  @property (nonatomic) int orientation;

• -initWithImage:

  Initializes a VisionImage object with the given image.

  Declaration

  Objective-C

  - (nonnull instancetype)initWithImage:(id)image;

  Parameters

  image

  Image to use in vision detection. The given image should be rotated, so its imageOrientation property is set to .up. The UIImage must have non-NULL CGImage property.

  Return Value

  A VisionImage instance with the given image.

• -initWithBuffer:

  Initializes a VisionImage object with the given image buffer. To improve performance, it is recommended to minimize the lifespan and number of instances of this class when initializing with a CMSampleBufferRef.

  Declaration

  Objective-C

  - (nonnull instancetype)initWithBuffer:(nonnull CMSampleBufferRef)sampleBuffer;

  Parameters

  sampleBuffer

  Image buffer to use in vision detection. The buffer must be based on a pixel buffer (not compressed data), and the pixel format must be one of: - kCVPixelFormatType_32BGRA - kCVPixelFormatType_420YpCbCr8BiPlanarFullRange - kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange In practice: this works with the video output of the phone’s camera, but not other arbitrary sources of CMSampleBufferRefs.

  Return Value

  A VisionImage instance with the given image buffer.

• -init

  Unavailable.

  Declaration

  Objective-C

  - (nonnull instancetype)init;