MLKitVision Framework Reference

  • VisionImage is an object used for vision detection tasks, accepting images or image buffers as input.

  • It's crucial to set the imageOrientation property of input UIImage to .up before initialization to ensure proper orientation.

  • When using CMSampleBufferRef for initialization, ensure it's a pixel buffer with a supported pixel format like kCVPixelFormatType_32BGRA for optimal performance.

  • For CMSampleBufferRef inputs, compatibility is primarily with phone camera output, not arbitrary buffer sources.

  • Direct initialization using init() is unavailable; utilize init(image:) or init(buffer:) instead.

VisionImage

class VisionImage : NSObject, MLKitCompatibleImage

An image or image buffer used for vision detection.

  • The display orientation of the image. The default is .up.

    Declaration

    Swift

    var orientation: Int32 { get set }

  • Initializes a VisionImage object with the given image.

    Declaration

    Swift

    init(image: Any!)

    Parameters

    image

    Image to use in vision detection. The given image should be rotated, so its imageOrientation property is set to .up. The UIImage must have non-NULL CGImage property.

    Return Value

    A VisionImage instance with the given image.

  • Initializes a VisionImage object with the given image buffer. To improve performance, it is recommended to minimize the lifespan and number of instances of this class when initializing with a CMSampleBufferRef.

    Declaration

    Swift

    init(buffer sampleBuffer: CMSampleBuffer)

    Parameters

    sampleBuffer

    Image buffer to use in vision detection. The buffer must be based on a pixel buffer (not compressed data), and the pixel format must be one of: - kCVPixelFormatType_32BGRA - kCVPixelFormatType_420YpCbCr8BiPlanarFullRange - kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange In practice: this works with the video output of the phone’s camera, but not other arbitrary sources of CMSampleBufferRefs.

    Return Value

    A VisionImage instance with the given image buffer.

  • Unavailable.