The Inception Score (IS) measures diversity and quality of generated images
Image: MMuzammils, CC BY-SA 3.0, via Wikimedia Commons
The Inception Score (IS) measures diversity and quality of generated images
The Inception Score (IS) is an algorithm used to assess the quality of images created by a generative image model such as a generative adversarial network (GAN). The score is calculated based on the output of a separate, pretrained Inception v3 image classification model applied to a sample of (typically around 30,000) images generated by the generative model. The Inception Score is maximized when the following conditions are true:
The entropy of the distribution of labels predicted by the Inceptionv3 model for the generated images is minimized. This corresponds to the desideratum of generated images being "sharp" or "distinct".
The predictions of the classification model are evenly distributed across all possible labels. This corresponds to the desideratum that the output of the generative model is "diverse".
It has been somewhat superseded by the related Fréchet inception distance (FID). While the Inception Score only evaluates the distribution of generated images, the FID compares the distribution of generated images with the distribution of a set of real images ("ground truth").
Example
A generative model produces 30,000 images, and the Inception v3 model classifies them. If the entropy of the label distribution is low and labels are evenly distributed, the Inception Score would be high, indicating high quality and diversity.
Remember this
Understanding the Inception Score helps in evaluating and improving the performance of generative models in creating high-quality and diverse images.
Text adapted from Wikipedia, licensed under CC BY-SA 4.0.
Fréchet inception distance
Fréchet inception distance (FID) compares image distributions
NDCG measures: ranking quality with graded relevance scores
NDCG measures ranking quality with graded relevance scores
to normalize features: when features have different scales and you use distance-based methods
Why do some things need to be adjusted to compare fairly?
Effect size
Cohen's D benchmarks: 0.2 = small, 0.5 = medium, 0.8 = large effect
Her Alibi
Bruce Beresford directed Her Alibi
Brier score
Brier score measures mean squared error of probability predictions
Swipe through 100 ML concepts daily
Open Pocket Polymath