SIBOptions<T, TInput, TOutput>

Options & Configuration · AiDotNet.MetaLearning.Options

Configuration options for SIB (Sequential Information Bottleneck) (Hu et al., 2020) few-shot learning.

Inheritance: ModelOptions → SIBOptions<T, TInput, TOutput>

Implements: IMetaLearnerOptions<T>

For Beginners

SIB balances two competing goals:

The Information Bottleneck principle:

  1. Keep USEFUL information: Cluster assignments should predict labels well
  2. Remove USELESS information: Don’t memorize irrelevant details

How SIB works for few-shot learning:

  1. Initialize cluster centroids from support class prototypes
  2. Assign ALL examples (support + query) to clusters
  3. Iteratively refine assignments by:
  • Reassigning each example to the most informative cluster
  • Updating cluster centroids based on new assignments
  • Balancing information retention vs. compression

Analogy: Efficient note-taking Imagine taking notes from a lecture:

  • Too much detail = you copy everything (no compression, hard to review)
  • Too little detail = you miss key points (too much compression)
  • SIB finds the sweet spot: keep important info, discard noise

Key property: Transductive - uses query set structure for better predictions

How It Works

SIB uses the information bottleneck principle for transductive few-shot learning. It iteratively optimizes cluster assignments by maximizing mutual information between data representations and cluster labels while compressing nuisance information.

Reference: Hu, Y., Gripon, V., & Pateux, S. (2020). Leveraging the Feature Distribution in Transfer-based Few-Shot Learning.

Constructors

Constructor Summary
SIBOptions(IFullModel<T, TInput, TOutput> metaModel) Initializes a new instance of SIBOptions.

Properties

Property Summary
IEpisodicDataLoader<T, TInput, TOutput> DataLoader { get; set; } Gets or sets the episodic data loader.
IFullModel<T, TInput, TOutput> MetaModel { get; set; } Gets or sets the feature extractor model.
IGradientBasedOptimizer<T, TInput, TOutput> InnerOptimizer { get; set; } Must stay null: this algorithm takes no plain gradient inner step (it adapts without one, or by its own update rule), so a supplied optimizer is rejected when the learner is built.
IGradientBasedOptimizer<T, TInput, TOutput> MetaOptimizer { get; set; } Optimizer for the meta-update.
ILossFunction<T> LossFunction { get; set; } Loss function for meta-training; null uses the default multi-class classification loss.
bool EnableCheckpointing { get; set; }
bool UseFirstOrder { get; set; }
double Beta { get; set; } Gets or sets the compression parameter (beta) for the information bottleneck.
double InnerLearningRate { get; set; }
double OuterLearningRate { get; set; }
double Temperature { get; set; } Gets or sets the temperature for soft cluster assignments.
double? GradientClipThreshold { get; set; }
int AdaptationSteps { get; set; } Number of gradient steps taken on each task’s support set.
int CheckpointFrequency { get; set; }
int EvaluationFrequency { get; set; }
int EvaluationTasks { get; set; }
int MetaBatchSize { get; set; }
int NumMetaIterations { get; set; }
int NumRestarts { get; set; } Gets or sets the number of random restarts for avoiding local optima.
int NumSIBIterations { get; set; } Gets or sets the number of SIB optimization iterations.
int? RandomSeed { get; set; } Gets or sets the random seed.

Methods

Method Summary
IMetaLearnerOptions<T> Clone()
bool IsValid()