DeCLIPOptions

Options & Configuration · AiDotNet.VisionLanguage.Encoders

Configuration options for the DeCLIP (Data-efficient CLIP) model.

Inheritance: ModelOptions → ContrastiveEncoderOptions → DeCLIPOptions

For Beginners

DeCLIP learns more from less data by using multiple learning strategies at once. While regular CLIP only learns from image-text pairs, DeCLIP also learns from images alone (self-supervision) and from similar images nearby in the dataset. This makes it much more data-efficient - achieving the same performance with fewer training examples.

How It Works

DeCLIP (Li et al., ICLR 2022) improves CLIP’s data efficiency by adding self-supervised learning objectives alongside the contrastive image-text loss. It uses image self-supervision (SimSiam), text self-supervision (masked language modeling), and nearest-neighbor supervision to extract more learning signal from each image-text pair.

Constructors

Constructor Summary
DeCLIPOptions() Initializes default DeCLIP options.
DeCLIPOptions(DeCLIPOptions other) Initializes a new instance by copying from another instance.

Properties

Property Summary
ContrastiveLossType LossType { get; set; } Gets or sets the contrastive loss type.
double ImageSelfSupervisedWeight { get; set; } Gets or sets the weight for the image self-supervised (SimSiam) loss.
double NearestNeighborWeight { get; set; } Gets or sets the weight for the nearest-neighbor supervision loss.
double TextMLMWeight { get; set; } Gets or sets the weight for the text masked language modeling loss.
double TextMaskingRatio { get; set; } Gets or sets the masking ratio for text masked language modeling.
int NumNearestNeighbors { get; set; } Gets or sets the number of nearest neighbors for supervision.