The wrapper class for MediaPipe Image objects.
Images are stored as ImageData, ImageBitmap or WebGLTexture objects. You can convert the underlying type to any other type by passing the desired type to getAs...(). As type conversions can be expensive, it is recommended to limit these conversions. You can verify what underlying types are already available by invoking has...().
Images that are returned from a MediaPipe Tasks are owned by by the underlying C++ Task. If you need to extend the lifetime of these objects, you can invoke the clone() method. To free up the resources obtained during any clone or type conversion operation, it is important to invoke close() on the MPImage instance.
Converting to and from ImageBitmap requires that the MediaPipe task is initialized with an OffscreenCanvas. As we require WebGL2 support, this places some limitations on Browser support as outlined here: https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas/getContext
Signature:
export declare class MPImage
Properties
| Property | Modifiers | Type | Description |
|---|---|---|---|
| canvas | HTMLCanvasElement | OffscreenCanvas | undefined | Returns the canvas element that the image is bound to. | |
| height | number | Returns the height of the image. | |
| width | number | Returns the width of the image. |
Methods
| Method | Modifiers | Description |
|---|---|---|
| clone() | Creates a copy of the resources stored in this MPImage. You can invoke this method to extend the lifetime of an image returned by a MediaPipe Task. Note that performance critical applications should aim to only use the MPImage within the MediaPipe Task callback so that copies can be avoided. |
|
| close() | Frees up any resources owned by this MPImage instance.Note that this method does not free images that are owned by the C++ Task, as these are freed automatically once you leave the MediaPipe callback. Additionally, some shared state is freed only once you invoke the Task's close() method. |
|
| getAsImageBitmap() | Returns the underlying image as an ImageBitmap. Note that conversions to ImageBitmap are expensive, especially if the data currently resides on CPU.Processing with ImageBitmaps requires that the MediaPipe Task was initialized with an OffscreenCanvas with WebGL2 support. See https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas/getContext for a list of supported platforms. The current image as an ImageBitmap object. |
|
| getAsImageData() | Returns the underlying image as an ImageData object. Note that this involves an expensive GPU to CPU transfer if the current image is only available as an ImageBitmap or WebGLTexture. The current image as an ImageData object. |
|
| getAsWebGLTexture() | Returns the underlying image as a WebGLTexture object. Note that this involves a CPU to GPU transfer if the current image is only available as an ImageData object. The returned texture is bound to the current canvas (see .canvas). The current image as a WebGLTexture. |
|
| hasImageBitmap() | Returns whether this MPImage contains a mask of type ImageBitmap. |
|
| hasImageData() | Returns whether this MPImage contains a mask of type ImageData. |
|
| hasWebGLTexture() | Returns whether this MPImage contains a mask of type WebGLTexture. |
MPImage.canvas
Returns the canvas element that the image is bound to.
Signature:
readonly canvas: HTMLCanvasElement | OffscreenCanvas | undefined;
MPImage.height
Returns the height of the image.
Signature:
readonly height: number;
MPImage.width
Returns the width of the image.
Signature:
readonly width: number;
MPImage.clone()
Creates a copy of the resources stored in this MPImage. You can invoke this method to extend the lifetime of an image returned by a MediaPipe Task. Note that performance critical applications should aim to only use the MPImage within the MediaPipe Task callback so that copies can be avoided.
Signature:
clone(): MPImage;
Returns:
MPImage.close()
Frees up any resources owned by this MPImage instance.
Note that this method does not free images that are owned by the C++ Task, as these are freed automatically once you leave the MediaPipe callback. Additionally, some shared state is freed only once you invoke the Task's close() method.
Signature:
close(): void;
Returns:
void
MPImage.getAsImageBitmap()
Returns the underlying image as an ImageBitmap. Note that conversions to ImageBitmap are expensive, especially if the data currently resides on CPU.
Processing with ImageBitmaps requires that the MediaPipe Task was initialized with an OffscreenCanvas with WebGL2 support. See https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas/getContext for a list of supported platforms.
The current image as an ImageBitmap object.
Signature:
getAsImageBitmap(): ImageBitmap;
Returns:
ImageBitmap
MPImage.getAsImageData()
Returns the underlying image as an ImageData object. Note that this involves an expensive GPU to CPU transfer if the current image is only available as an ImageBitmap or WebGLTexture.
The current image as an ImageData object.
Signature:
getAsImageData(): ImageData;
Returns:
ImageData
MPImage.getAsWebGLTexture()
Returns the underlying image as a WebGLTexture object. Note that this involves a CPU to GPU transfer if the current image is only available as an ImageData object. The returned texture is bound to the current canvas (see .canvas).
The current image as a WebGLTexture.
Signature:
getAsWebGLTexture(): WebGLTexture;
Returns:
WebGLTexture
MPImage.hasImageBitmap()
Returns whether this MPImage contains a mask of type ImageBitmap.
Signature:
hasImageBitmap(): boolean;
Returns:
boolean
MPImage.hasImageData()
Returns whether this MPImage contains a mask of type ImageData.
Signature:
hasImageData(): boolean;
Returns:
boolean
MPImage.hasWebGLTexture()
Returns whether this MPImage contains a mask of type WebGLTexture.
Signature:
hasWebGLTexture(): boolean;
Returns:
boolean