Performs object detection on images.
The API expects a TFLite model with TFLite Model Metadata..
The API supports models with one image input tensor and four output tensors. To be more specific, here are the requirements.
- Input image tensor (
kTfLiteUInt8
/kTfLiteFloat32
)- image input of size
[batch x height x width x channels]
. - batch inference is not supported (
batch
is required to be 1). - only RGB inputs are supported (
channels
is required to be 3). - if type is
kTfLiteFloat32
, NormalizationOptions are required to be attached to the metadata for input normalization.
- image input of size
DetectionPostProcess
op, i.e:
- Location tensor (
kTfLiteFloat32
):- tensor of size
[1 x num_results x 4]
, the inner array representing bounding boxes in the form [top, left, right, bottom]. BoundingBoxProperties
are required to be attached to the metadata and must specifytype=BOUNDARIES
andcoordinate_type=RATIO
.
- tensor of size
kTfLiteFloat32
):
- tensor of size
[1 x num_results]
, each value representing the integer index of a class. - if label maps are attached to the metadata as
TENSOR_VALUE_LABELS
associated files, they are used to convert the tensor values into labels.
kTfLiteFloat32
):
- tensor of size
[1 x num_results]
, each value representing the score of the detected object.
kTfLiteFloat32
):
- integer num_results as a tensor of size
[1]
.
An example of such model can be found on TensorFlow Hub..
Nested Classes
class | ObjectDetector.ObjectDetectorOptions | Options for setting up an ObjectDetector . |
Public Methods
static ObjectDetector |
createFromBuffer(Context context, ByteBuffer modelBuffer)
Creates an
ObjectDetector instance from a model buffer and the default ObjectDetector.ObjectDetectorOptions . |
static ObjectDetector |
createFromFile(Context context, String modelPath)
Creates an
ObjectDetector instance from a model file and the default ObjectDetector.ObjectDetectorOptions . |
static ObjectDetector |
createFromFile(Context context, File modelFile)
Creates an
ObjectDetector instance from a model file and the default ObjectDetector.ObjectDetectorOptions . |
static ObjectDetector |
createFromOptions(Context context, ObjectDetector.ObjectDetectorOptions detectorOptions)
Creates an
ObjectDetector instance from an ObjectDetector.ObjectDetectorOptions . |
ObjectDetectorResult | |
ObjectDetectorResult |
detect(MPImage image, ImageProcessingOptions imageProcessingOptions)
Performs object detection on the provided single image.
|
void |
detectAsync(MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs)
Sends live image data to perform object detection, and the results will be available via the
OutputHandler.ResultListener provided in the ObjectDetector.ObjectDetectorOptions . |
void |
detectAsync(MPImage image, long timestampMs)
Sends live image data to perform object detection with default image processing options, i.e.
|
ObjectDetectorResult |
detectForVideo(MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs)
Performs object detection on the provided video frame.
|
ObjectDetectorResult |
detectForVideo(MPImage image, long timestampMs)
Performs object detection on the provided video frame with default image processing options,
i.e.
|
Inherited Methods
Public Methods
public static ObjectDetector createFromBuffer (Context context, ByteBuffer modelBuffer)
Creates an ObjectDetector
instance from a model buffer and the default ObjectDetector.ObjectDetectorOptions
.
Parameters
context | an Android ERROR(/Context) . |
---|---|
modelBuffer | a direct ByteBuffer or a ERROR(/MappedByteBuffer) of the detection
model. |
Throws
if there is an error during ObjectDetector creation.
|
public static ObjectDetector createFromFile (Context context, String modelPath)
Creates an ObjectDetector
instance from a model file and the default ObjectDetector.ObjectDetectorOptions
.
Parameters
context | an Android ERROR(/Context) . |
---|---|
modelPath | path to the detection model with metadata in the assets. |
Throws
if there is an error during ObjectDetector creation.
|
public static ObjectDetector createFromFile (Context context, File modelFile)
Creates an ObjectDetector
instance from a model file and the default ObjectDetector.ObjectDetectorOptions
.
Parameters
context | an Android ERROR(/Context) . |
---|---|
modelFile | the detection model File instance. |
Throws
IOException | if an I/O error occurs when opening the tflite model file. |
---|---|
if there is an error during ObjectDetector creation.
|
public static ObjectDetector createFromOptions (Context context, ObjectDetector.ObjectDetectorOptions detectorOptions)
Creates an ObjectDetector
instance from an ObjectDetector.ObjectDetectorOptions
.
Parameters
context | an Android ERROR(/Context) . |
---|---|
detectorOptions | an ObjectDetector.ObjectDetectorOptions instance. |
Throws
if there is an error during ObjectDetector creation.
|
public ObjectDetectorResult detect (MPImage image)
Performs object detection on the provided single image with default image processing options,
i.e. without any rotation applied. Only use this method when the ObjectDetector
is
created with ERROR(/RunningMode.IMAGE)
.
ObjectDetector
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|
Throws
if there is an internal error. |
public ObjectDetectorResult detect (MPImage image, ImageProcessingOptions imageProcessingOptions)
Performs object detection on the provided single image. Only use this method when the ObjectDetector
is created with ERROR(/RunningMode.IMAGE)
.
ObjectDetector
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|---|
imageProcessingOptions | the ImageProcessingOptions specifying how to process the
input image before running inference. Note that region-of-interest is not supported
by this task: specifying ImageProcessingOptions.regionOfInterest() will result in
this method throwing an IllegalArgumentException. |
Throws
IllegalArgumentException | if the ImageProcessingOptions specify a
region-of-interest. |
---|---|
if there is an internal error. |
public void detectAsync (MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs)
Sends live image data to perform object detection, and the results will be available via the
OutputHandler.ResultListener
provided in the ObjectDetector.ObjectDetectorOptions
. Only use this method when
the ObjectDetector
is created with ERROR(/RunningMode.LIVE_STREAM)
.
It's required to provide a timestamp (in milliseconds) to indicate when the input image is sent to the object detector. The input timestamps must be monotonically increasing.
ObjectDetector
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|---|
imageProcessingOptions | the ImageProcessingOptions specifying how to process the
input image before running inference. Note that region-of-interest is not supported
by this task: specifying ImageProcessingOptions.regionOfInterest() will result in
this method throwing an IllegalArgumentException. |
timestampMs | the input timestamp (in milliseconds). |
Throws
IllegalArgumentException | if the ImageProcessingOptions specify a
region-of-interest. |
---|---|
if there is an internal error. |
public void detectAsync (MPImage image, long timestampMs)
Sends live image data to perform object detection with default image processing options, i.e.
without any rotation applied, and the results will be available via the OutputHandler.ResultListener
provided in the ObjectDetector.ObjectDetectorOptions
. Only use this method when the ObjectDetector
is created with ERROR(/RunningMode.LIVE_STREAM)
.
It's required to provide a timestamp (in milliseconds) to indicate when the input image is sent to the object detector. The input timestamps must be monotonically increasing.
ObjectDetector
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|---|
timestampMs | the input timestamp (in milliseconds). |
Throws
if there is an internal error. |
public ObjectDetectorResult detectForVideo (MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs)
Performs object detection on the provided video frame. Only use this method when the ObjectDetector
is created with ERROR(/RunningMode.VIDEO)
.
It's required to provide the video frame's timestamp (in milliseconds). The input timestamps must be monotonically increasing.
ObjectDetector
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|---|
imageProcessingOptions | the ImageProcessingOptions specifying how to process the
input image before running inference. Note that region-of-interest is not supported
by this task: specifying ImageProcessingOptions.regionOfInterest() will result in
this method throwing an IllegalArgumentException. |
timestampMs | the input timestamp (in milliseconds). |
Throws
IllegalArgumentException | if the ImageProcessingOptions specify a
region-of-interest. |
---|---|
if there is an internal error. |
public ObjectDetectorResult detectForVideo (MPImage image, long timestampMs)
Performs object detection on the provided video frame with default image processing options,
i.e. without any rotation applied. Only use this method when the ObjectDetector
is
created with ERROR(/RunningMode.VIDEO)
.
It's required to provide the video frame's timestamp (in milliseconds). The input timestamps must be monotonically increasing.
ObjectDetector
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|---|
timestampMs | the input timestamp (in milliseconds). |
Throws
if there is an internal error. |