SImPaOptions<T, TInput, TOutput>

Options & Configuration · AiDotNet.MetaLearning.Options

Configuration for SImPaAlgorithm — SImPa, statistical implicit PAC-Bayes meta-learning.

Inheritance: ModelOptions → SImPaOptions<T, TInput, TOutput>

Implements: IMetaLearnerOptions<T>

For Beginners

These settings control how the model represents its uncertainty about each task (a small generator network turning random numbers into parameters), how the divergence from the prior is estimated when there is no formula for it, and how confident the generalization guarantee is.

How It Works

Defaults are the paper’s where it states them (Nguyen, Do and Carneiro, arXiv:2003.02455): z ~ U[0,1]^128, generator hidden widths 256 and 512, epsilon = 0.1, one posterior sample per task while training and 32 at test time, 512 Monte Carlo samples for the phi-network, and a learning rate of 1e-4.

Constructors

Constructor Summary
SImPaOptions(IFullModel<T, TInput, TOutput> metaModel) Creates the options for a meta-model.

Properties

Property Summary
IEpisodicDataLoader<T, TInput, TOutput> DataLoader { get; set; } Episodic data loader that supplies the meta-training tasks; null when tasks are passed to training directly.
IFullModel<T, TInput, TOutput> MetaModel { get; set; }
IGradientBasedOptimizer<T, TInput, TOutput> InnerOptimizer { get; set; } Must stay null: this algorithm takes no plain gradient inner step (it adapts without one, or by its own update rule), so a supplied optimizer is rejected when the learner is built.
IGradientBasedOptimizer<T, TInput, TOutput> MetaOptimizer { get; set; } Must stay null: this algorithm’s meta-update is part of the method rather than a plain gradient step, so a supplied optimizer is rejected when the learner is built.
ILossFunction<T> LossFunction { get; set; } Loss function for meta-training; null uses the default multi-class classification loss.
bool EnableCheckpointing { get; set; }
bool UseFirstOrder { get; set; }
double Epsilon { get; set; } Gets or sets epsilon, the bound’s confidence parameter.
double InnerLearningRate { get; set; } Gets or sets the inner-loop learning rate.
double KLEstimatorLearningRate { get; set; } Gets or sets the ascent step size for the phi-network.
double MetaPosteriorStdDev { get; set; } Gets or sets the meta-parameter posterior standard deviation, sigma_0.
double OuterLearningRate { get; set; } Gets or sets the outer-loop learning rate.
double PhiMetaLearningRate { get; set; } Gets or sets the ascent step size for phi_0, the meta-level initialisation of the phi-network (Algorithm 1 line 13).
double PriorStdDev { get; set; } Gets or sets the prior standard deviation over task parameters, sigma_w.
double? GradientClipThreshold { get; set; }
int AdaptationPosteriorSamples { get; set; } Gets or sets the posterior samples drawn at ADAPTATION time.
int AdaptationSteps { get; set; } Gets or sets the inner adaptation steps applied to the generator per task.
int CheckpointFrequency { get; set; }
int EvaluationFrequency { get; set; }
int EvaluationTasks { get; set; }
int GeneratorFirstHiddenWidth { get; set; } Gets or sets the generator’s first hidden width.
int GeneratorSecondHiddenWidth { get; set; } Gets or sets the generator’s second hidden width.
int KLEstimatorHiddenWidth { get; set; } Gets or sets the phi-network’s hidden width.
int KLEstimatorSteps { get; set; } Gets or sets the ascent steps taken on the compression-lemma objective.
int KLMonteCarloSamples { get; set; } Gets or sets the Monte Carlo samples used to train the phi-network.
int LatentDimension { get; set; } Gets or sets the latent noise width for the implicit posterior generator.
int MetaBatchSize { get; set; } Gets or sets T, tasks per meta-iteration.
int NumMetaIterations { get; set; }
int TrainingPosteriorSamples { get; set; } Gets or sets the posterior samples drawn per task while META-TRAINING.
int? RandomSeed { get; set; }

Methods

Method Summary
IMetaLearnerOptions<T> Clone()
bool IsValid()