SIBAlgorithm<T, TInput, TOutput>

Models & Types · AiDotNet.MetaLearning.Algorithms

Implementation of SIB (Sequential Information Bottleneck) for transductive few-shot learning.

Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → SIBAlgorithm<T, TInput, TOutput>

For Beginners

SIB is like an intelligent clustering algorithm:

How it works:

  1. Start with class prototypes from support examples
  2. Assign all query examples to the closest prototype (initial clusters)
  3. Iteratively refine:

a. For each example, compute KL divergence to each cluster b. Reassign to the best cluster (minimum KL divergence) c. Update cluster statistics d. Repeat until convergence

  1. Multiple restarts ensure we don’t get stuck in bad solutions

Information Bottleneck trade-off:

  • Want to KEEP: Information about which class each example belongs to
  • Want to REMOVE: Noise, irrelevant variations, outlier effects
  • Beta parameter controls this balance

Why multiple restarts? The SIB optimization landscape can have local optima. Running from different starting points increases the chance of finding the globally best clustering.

How It Works

SIB uses the information bottleneck principle to iteratively refine cluster assignments for transductive few-shot classification. It processes all examples (support + query) jointly, optimizing a trade-off between information retention and compression.

Algorithm - SIB:

Reference: Hu, Y., Gripon, V., & Pateux, S. (2020). Leveraging the Feature Distribution in Transfer-based Few-Shot Learning.

Constructors

Constructor Summary
SIBAlgorithm(SIBOptions<T, TInput, TOutput> options) Initializes a new SIB meta-learner.

Properties

Property Summary
MetaLearningAlgorithmType AlgorithmType { get; }

Methods

Method Summary
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task) Adapts to a new task using the Sequential Information Bottleneck.
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch) Performs one meta-training step for SIB.