LEOOptions<T, TInput, TOutput>
Options & Configuration · AiDotNet.MetaLearning.Options
Configuration options for Latent Embedding Optimization (LEO) algorithm.
Inheritance: ModelOptions → LEOOptions<T, TInput, TOutput>
Implements: IMetaLearnerOptions<T>
For Beginners
Imagine you have a very large model with millions of parameters. Updating all of them during few-shot learning is slow and can lead to overfitting. LEO learns to “compress” these parameters into a much smaller space (like 64 numbers instead of millions). Adaptation happens in this compressed space, which is faster and more robust to overfitting.
How It Works
LEO performs meta-learning by learning a low-dimensional latent space for model parameters. Instead of adapting the full model parameters directly (like MAML), LEO:
Key Insight: Not all parameter configurations make sense for neural networks. By learning a latent space, LEO restricts adaptation to the “manifold” of sensible parameter settings, preventing bad updates.
Constructors
| Constructor | Summary |
|---|---|
LEOOptions(IFullModel<T, TInput, TOutput> metaModel) |
Initializes a new instance of the LEOOptions class with the required meta-model. |
Properties
| Property | Summary |
|---|---|
IEpisodicDataLoader<T, TInput, TOutput> DataLoader { get; set; } |
Gets or sets the episodic data loader for sampling tasks. |
IFullModel<T, TInput, TOutput> MetaModel { get; set; } |
Gets or sets the meta-model (feature encoder) to be trained. |
IGradientBasedOptimizer<T, TInput, TOutput> InnerOptimizer { get; set; } |
Must stay null: this algorithm takes no plain gradient inner step (it adapts without one, or by its own update rule), so a supplied optimizer is rejected when the learner is built. |
IGradientBasedOptimizer<T, TInput, TOutput> MetaOptimizer { get; set; } |
Optimizer for the meta-update. |
ILossFunction<T> LossFunction { get; set; } |
Loss function for meta-training; null uses cross-entropy over logits. |
bool EnableCheckpointing { get; set; } |
Gets or sets whether to save model checkpoints. |
bool ShareEncoder { get; set; } |
Gets or sets whether to share the encoder across all classes. |
bool UseFirstOrder { get; set; } |
Gets or sets whether to drop the second-order terms of the meta-gradient. |
bool UseOrthogonalInit { get; set; } |
Gets or sets whether to initialise the decoder with orthonormal latent rows. |
bool UseRelationEncoder { get; set; } |
Gets or sets whether to use a relation network for encoding. |
double DropoutRate { get; set; } |
Gets or sets the dropout rate on the feature embeddings while meta-training. |
double EncoderPenaltyWeight { get; set; } |
Gets or sets the weight of the penalty pulling the encoder’s initial codes toward the adapted codes. |
double EntropyWeight { get; set; } |
Gets or sets the weight of an entropy bonus on the decoder’s weight distribution. |
double FineTuningLearningRate { get; set; } |
Gets or sets the initial fine-tuning step size; LEO then learns one step size per embedding dimension. |
double InnerLearningRate { get; set; } |
Gets or sets the initial latent step size; LEO then learns one step size per latent dimension. |
double KLWeight { get; set; } |
Gets or sets the KL divergence weight for the latent space regularization. |
double L2Regularization { get; set; } |
Gets or sets the L2 weight on the encoder, relation network and decoder. |
double OrthogonalityWeight { get; set; } |
Gets or sets the weight of the decoder orthogonality penalty. |
double OuterLearningRate { get; set; } |
Gets or sets the learning rate for the outer loop (meta-update). |
double? GradientClipThreshold { get; set; } |
Gets or sets the maximum norm of each meta-gradient block. |
int AdaptationSteps { get; set; } |
Gets or sets the number of adaptation steps in latent space. |
int CheckpointFrequency { get; set; } |
Gets or sets how often to save checkpoints. |
int EmbeddingDimension { get; set; } |
Gets or sets the width of each example’s feature embedding: the feature encoder’s per-example output width. |
int EvaluationFrequency { get; set; } |
Gets or sets how often to evaluate the meta-learner. |
int EvaluationTasks { get; set; } |
Gets or sets the number of tasks to use for evaluation. |
int FineTuningSteps { get; set; } |
Gets or sets the number of fine-tuning steps taken directly on the generated classifier weights after the latent steps. |
int HiddenDimension { get; set; } |
Gets or sets the width of the relation network’s hidden layers. |
int LatentDimension { get; set; } |
Gets or sets the dimensionality of the latent space. |
int MetaBatchSize { get; set; } |
Gets or sets the number of tasks to sample per meta-training iteration. |
int NumClasses { get; set; } |
Gets or sets the number of output classes. |
int NumMetaIterations { get; set; } |
Gets or sets the total number of meta-training iterations. |
int? RandomSeed { get; set; } |
Gets or sets the random seed for reproducibility. |
Methods
| Method | Summary |
|---|---|
IMetaLearnerOptions<T> Clone() |
Creates a deep copy of the LEO options. |
bool IsValid() |
Validates that all LEO configuration options are properly set. |