IBlipModel<T>

Interfaces · AiDotNet.Interfaces

Defines the contract for BLIP (Bootstrapped Language-Image Pre-training) models.

Implements: IMultimodalEmbedding<T>

Known implementations: BlipNeuralNetwork<T>

For Beginners

BLIP is like CLIP but with extra superpowers!

What CLIP can do:

  • Compare images and text (are they related?)
  • Zero-shot classification (classify without training)

What BLIP adds:

  • Generate captions for images (describe what you see)
  • Answer questions about images (VQA)
  • Better image-text matching with cross-attention

BLIP was trained on a larger, cleaner dataset using a special “bootstrapping” technique that improves the quality of training data automatically.

How It Works

BLIP extends CLIP’s capabilities with additional vision-language tasks: image captioning, image-text matching, and visual question answering. This interface extends IMultimodalEmbedding with these features.

Methods

Method Summary
IEnumerable<ValueTuple<int, T>> RetrieveImages(string query, IEnumerable<Vector<T>> imageEmbeddings, int topK) Retrieves the most relevant images for a text query from a collection.
IEnumerable<ValueTuple<int, T>> RetrieveTexts(Tensor<T> image, IEnumerable<Vector<T>> textEmbeddings, int topK) Retrieves the most relevant texts for an image from a collection.
IEnumerable<ValueTuple<string, T>> RankCaptions(Tensor<T> image, IEnumerable<string> candidates) Ranks a set of candidate captions by how well they match an image.
IEnumerable<string> GenerateCaptions(Tensor<T> image, int numCaptions, int maxLength) Generates multiple candidate captions for an image.
T ComputeImageTextMatch(Tensor<T> image, string text) Determines whether a given text accurately describes an image.
string AnswerQuestion(Tensor<T> image, string question, int maxLength) Answers a question about an image’s content.
string GenerateCaption(Tensor<T> image, int maxLength, int numBeams) Generates a caption describing the content of an image.