CLIPScore<T>

Models & Types · AiDotNet.Metrics

CLIPScore metric for evaluating text-image alignment and image quality.

How It Works

CLIPScore measures how well an image matches a text description (or reference image) using CLIP embeddings. It’s widely used for evaluating text-to-image generation models.

Two main variants:

  • CLIPScore (text-image): Measures alignment between generated images and text prompts
  • RefCLIPScore (image-image): Measures similarity between generated and reference images

Typical CLIPScore values (0-100 scale):

  • 30: Excellent alignment (image matches text well)

  • 25-30: Good alignment
  • 20-25: Moderate alignment
  • <20: Poor alignment

Based on “CLIPScore: A Reference-free Evaluation Metric for Image Captioning” by Hessel et al. (2021)

Constructors

Constructor Summary
CLIPScore(IMultimodalEmbedding<T> clipModel) Initializes a new instance of CLIPScore calculator.

Properties

Property Summary
IMultimodalEmbedding<T> ClipModel { get; } Gets the CLIP model used for computing embeddings.

Methods

Method Summary
T ComputeCaptionScore(double[] imageData, string candidateCaption, IEnumerable<string> referenceCaptions) Computes CLIPScore for image captioning evaluation.
T ComputeCombinedScore(double[] generatedImageData, string text, double[] referenceImageData, double textWeight) Computes combined CLIPScore using both text-image and image-image similarity.
T ComputeDirectionalSimilarity(double[] sourceImageData, double[] editedImageData, string sourceText, string editText) Computes directional similarity for image editing evaluation.
T ComputeImageImageScore(double[] generatedImageData, double[] referenceImageData) Computes RefCLIPScore between a generated image and a reference image.
T ComputeScoreImprovement(double[] beforeImageData, double[] afterImageData, string text) Computes CLIPScore improvement between before and after images.
T ComputeTextImageScore(double[] imageData, string text) Computes CLIPScore between an image and a text description.
T ComputeTextImageScoreBatch(IList<double[]> images, IList<string> texts) Computes CLIPScore for a batch of images and their corresponding text descriptions.