AI-generated Key Takeaways
-
CXR Foundation is an ML model creating embeddings from chest X-rays to build AI models efficiently with less data and compute.
-
The model, optimized for chest X-rays, has shown success with other X-ray types including those of other body parts and veterinary X-rays.
-
Key use cases include data-efficient classification for clinical findings and image quality, zero-shot classification using textual prompts without training data, and semantic image retrieval based on embedding distances.
CXR Foundation is a machine learning (ML) model that produces embeddings based on images of chest X-rays. The embeddings can be used to efficiently build AI models for chest X-ray related tasks, requiring less data and less compute than having to fully train a model without the embeddings or the pretrained model.
The model has been optimized for chest X-rays, but researchers have reported success using it for other types of X-rays, including X-rays of other body parts and even veterinary X-rays.
Trained on large scale datasets, CXR Foundation helps organizations in healthcare and life sciences do more with their chest X-ray data with less data, accelerating their ability to build AI models for chest X-ray image analysis.
For details about how to use the model and how it was trained, see the CXR Foundation model card.
Common Use Cases
The following sections present common use cases for the model. You're free to pursue any use case, as long as it adheres to the Health AI Developer Foundations terms of use.
Data-efficient classification
CXR Foundation can be used for data-efficient classification tasks, including:
- Clinical findings like fracture or pneumothorax
- Determining X-ray image quality
- Determining the X-ray view or body part
- Determining the presence of devices
- Discovering misplaced tubes
With a small amount of labelled data, you can train a classifier model on top of CXR Foundation embeddings. Furthermore, the embedding from each X-ray only needs to be generated once and can be used as an input for a variety of different classifiers, with very little additional compute.
For an example of how to use the model to train classifiers using the CXR-14 public dataset, see the CXR Foundation linear classifier notebook in Colab.
Zero-shot classification
By using the contrastive mode (ELIXR-contrastive / v2.0 text), users can get a classification score without any additional training data through textual prompts. Zero-shot works by measuring the relative distance of the image embeddings from a positive e.g., "pleural effusion present", and negative text prompt e.g., "normal X-ray". The use cases are the same as data-efficient classification but don't require data to train. The zero-shot method will outperform data-efficient classifications at low levels of training data, while the data-efficient classification will tend to exceed zero-shot performance with larger amounts of data. See ELIXR paper for more details.
See our zero-shot classification Colab notebook in Colab for an example of how to implement zero-shot.
Semantic image retrieval
By using the contrastive mode (ELIXR-contrastive / v2.0 text), users can rank a set of X-rays across a search query. Similar to zero-shot classification, language-based image retrieval relies on the distance between the embeddings of the set of images and the text embeddings from the search query.
For an example of how to use the contrastive embeddings for image retrieval or zero-shot classification, see the following notebooks:
See our semantic image retrieval Colab notebook in Colab for an example of how to implement semantic image retrieval.