With ML Kit's image labeling APIs you can detect and extract information about entities in an image across a broad group of categories. The default image labeling model can identify general objects, places, activities, animal species, products, and more.
You can also use a custom image classification model to tailor detection to a specific use case. See Using a custom TensorFlow Lite model and or Using a custom AutoML Vision Edge model for more information.
- A powerful general-purpose base classifier Recognizes more than 400 categories that describe the most commonly found objects in photos.
- Tailor to your use case with custom models Use other pre-trained models from TensorFlow Hub or your own custom model trained with TensorFlow, AutoML Vision Edge or TensorFlow Lite Model maker.
- Easy-to-use high-level APIs No need to deal with low-level model input/output, image pre- and post-processing, or building a processing pipeline. ML Kit extracts the labels from the TensorFlow Lite model and provides them as a text description.
Note that this API is intended for image classification models that describe the full image. For classifying one or more objects in an image, such as shoes or pieces of furniture, the Object Detection & Tracking API may be a better fit.
Supported image classification models
The Image Labeling APIs support different image classification models:
|Supported image classification models|
|Base model||By default the API uses a powerful general-purpose image labeling model that recognizes more than 400 entities that cover the most commonly-found concepts in photos.|
|Custom TensorFlow Lite models||To target application-specific concepts, the API accepts custom image classification models from a wide range of sources. These can be pre-trained models downloaded from TensorFlow Hub or your own models trained with TensorFlow Lite Model Maker or TensorFlow itself. Custom models must be bundled with your app.|
|Custom AutoML Vision Edge models||The API supports image classification models trained with AutoML Vision Edge. Models can be bundled with your app or hosted with Firebase Machine Learning and downloaded at run-time.|
Using the base model
ML Kit’s base model returns a list of entities that identify people, things, places, activities, and so on. Each entity comes with a score that indicates the confidence the ML model has in its relevance. With this information, you can perform tasks such as automatic metadata generation and content moderation. The default model provided with ML Kit recognizes more than 400 different entities.
The base model in the image labeling API supports 400+ labels, such as the following examples:
Here is an example of the entities that were recognized in the accompanying photo.
Using a custom TensorFlow Lite model
ML Kit's base image labeling model is built for general-purpose use. It's trained to recognize 400 categories that describe the most commonly-found objects in photos. Your app might need a specialized image classification model that recognizes a narrower number of categories in more detail, such as a model that distinguishes between species of flowers or types of food.
ML Kit's Image Labeling feature supports custom image classification models from a wide range of sources. You can use pre-trained models from TensorFlow Hub, or train your own model with TensorFlow Lite Model maker or TensorFlow. Please refer to Custom models with ML Kit to learn more. Custom models are bundled with your app.
Using a custom AutoML Vision Edge model
ML Kit's Image Labeling feature supports models trained using AutoML Vision Edge. It supports bundling the model with your app or dymically downloading it from the cloud using Firebase Machine Learning's Model deployment service. Please refer to train your own model using AutoML Vision Edge for details.