SIBOptions<T, TInput, TOutput>
Options & Configuration · AiDotNet.MetaLearning.Options
Configuration options for SIB (Sequential Information Bottleneck) (Hu et al., 2020) few-shot learning.
Inheritance: ModelOptions → SIBOptions<T, TInput, TOutput>
Implements: IMetaLearnerOptions<T>
For Beginners
SIB balances two competing goals:
The Information Bottleneck principle:
- Keep USEFUL information: Cluster assignments should predict labels well
- Remove USELESS information: Don’t memorize irrelevant details
How SIB works for few-shot learning:
- Initialize cluster centroids from support class prototypes
- Assign ALL examples (support + query) to clusters
- Iteratively refine assignments by:
- Reassigning each example to the most informative cluster
- Updating cluster centroids based on new assignments
- Balancing information retention vs. compression
Analogy: Efficient note-taking Imagine taking notes from a lecture:
- Too much detail = you copy everything (no compression, hard to review)
- Too little detail = you miss key points (too much compression)
- SIB finds the sweet spot: keep important info, discard noise
Key property: Transductive - uses query set structure for better predictions
How It Works
SIB uses the information bottleneck principle for transductive few-shot learning. It iteratively optimizes cluster assignments by maximizing mutual information between data representations and cluster labels while compressing nuisance information.
Reference: Hu, Y., Gripon, V., & Pateux, S. (2020). Leveraging the Feature Distribution in Transfer-based Few-Shot Learning.
Constructors
| Constructor | Summary |
|---|---|
SIBOptions(IFullModel<T, TInput, TOutput> metaModel) |
Initializes a new instance of SIBOptions. |
Properties
| Property | Summary |
|---|---|
IEpisodicDataLoader<T, TInput, TOutput> DataLoader { get; set; } |
Gets or sets the episodic data loader. |
IFullModel<T, TInput, TOutput> MetaModel { get; set; } |
Gets or sets the feature extractor model. |
IGradientBasedOptimizer<T, TInput, TOutput> InnerOptimizer { get; set; } |
Must stay null: this algorithm takes no plain gradient inner step (it adapts without one, or by its own update rule), so a supplied optimizer is rejected when the learner is built. |
IGradientBasedOptimizer<T, TInput, TOutput> MetaOptimizer { get; set; } |
Optimizer for the meta-update. |
ILossFunction<T> LossFunction { get; set; } |
Loss function for meta-training; null uses the default multi-class classification loss. |
bool EnableCheckpointing { get; set; } |
|
bool UseFirstOrder { get; set; } |
|
double Beta { get; set; } |
Gets or sets the compression parameter (beta) for the information bottleneck. |
double InnerLearningRate { get; set; } |
|
double OuterLearningRate { get; set; } |
|
double Temperature { get; set; } |
Gets or sets the temperature for soft cluster assignments. |
double? GradientClipThreshold { get; set; } |
|
int AdaptationSteps { get; set; } |
Number of gradient steps taken on each task’s support set. |
int CheckpointFrequency { get; set; } |
|
int EvaluationFrequency { get; set; } |
|
int EvaluationTasks { get; set; } |
|
int MetaBatchSize { get; set; } |
|
int NumMetaIterations { get; set; } |
|
int NumRestarts { get; set; } |
Gets or sets the number of random restarts for avoiding local optima. |
int NumSIBIterations { get; set; } |
Gets or sets the number of SIB optimization iterations. |
int? RandomSeed { get; set; } |
Gets or sets the random seed. |
Methods
| Method | Summary |
|---|---|
IMetaLearnerOptions<T> Clone() |
|
bool IsValid() |