MbPAOptions<T, TInput, TOutput>
Options & Configuration · AiDotNet.MetaLearning.Options
Configuration options for Memory-based Parameter Adaptation (MbPA), Sprechmann et al., ICLR 2018 (arXiv:1802.10542).
Inheritance: ModelOptions → MbPAOptions<T, TInput, TOutput>
Implements: IMetaLearnerOptions<T>
For Beginners
A normal network has to see a lot of examples, many times, before it changes its mind about anything. MbPA gives it a notebook. When a new input arrives, it flips to the most similar things it has written down and briefly tunes its last layer to fit those, answers, and then tears the page out. Nothing is permanently relearned, so old knowledge is not overwritten — but the answer reflects the relevant memories.
How It Works
MbPA stores (embedding, target) pairs in an episodic memory. At prediction time it looks
up the K nearest neighbours of the query’s embedding and takes a few gradient steps that fit the
OUTPUT network to those neighbours, regularized toward the trained parameters. The adapted
parameters produce that one output and are then discarded.
The defaults sit inside the ranges the paper reports: K and the step count both in [1, 20], and a local learning rate in [0, 1].
Constructors
| Constructor | Summary |
|---|---|
MbPAOptions(IFullModel<T, TInput, TOutput> metaModel) |
Creates options for meta-learning metaModel, with the default settings. |
Properties
| Property | Summary |
|---|---|
IEpisodicDataLoader<T, TInput, TOutput> DataLoader { get; set; } |
Episodic data loader that supplies the meta-training tasks; null when tasks are passed to training directly. |
IFullModel<T, TInput, TOutput> MetaModel { get; set; } |
Gets or sets the embedding network f_gamma. |
IGradientBasedOptimizer<T, TInput, TOutput> InnerOptimizer { get; set; } |
Must stay null: this algorithm takes no plain gradient inner step (it adapts without one, or by its own update rule), so a supplied optimizer is rejected when the learner is built. |
IGradientBasedOptimizer<T, TInput, TOutput> MetaOptimizer { get; set; } |
Optimizer for the meta-update. |
ILossFunction<T> LossFunction { get; set; } |
Loss function for meta-training; null uses the default regression loss. |
MbPAOutputDistribution OutputDistribution { get; set; } |
Gets or sets the output distribution the local adaptation fits, i.e. |
bool EnableCheckpointing { get; set; } |
|
bool UseFirstOrder { get; set; } |
|
bool WriteMemoryDuringTraining { get; set; } |
Gets or sets whether observed examples are written into the episodic memory during training. |
double InnerLearningRate { get; set; } |
|
double KernelEpsilon { get; set; } |
Gets or sets epsilon in the retrieval kernel kern(h, q) = 1 / (eps + ||h - q||^2), which keeps the weight finite when a stored key coincides with the query. |
double LocalLearningRate { get; set; } |
Gets or sets the local adaptation learning rate alpha_M. |
double OuterLearningRate { get; set; } |
|
double RegularizationBeta { get; set; } |
Gets or sets beta, the strength of the pull back toward the trained parameters. |
double? GradientClipThreshold { get; set; } |
|
int AdaptationSteps { get; set; } |
Gets or sets the number of adaptation steps used by the generic meta-learning contract. |
int CheckpointFrequency { get; set; } |
|
int EvaluationFrequency { get; set; } |
|
int EvaluationTasks { get; set; } |
|
int FeatureDimension { get; set; } |
Gets or sets the width of the embedding produced by MetaModel, which is both the memory key width and the output network’s input width. |
int LocalAdaptationSteps { get; set; } |
Gets or sets T, the number of local gradient steps taken per prediction. |
int MemorySize { get; set; } |
Gets or sets the maximum number of (embedding, target) pairs held in the episodic memory. |
int MetaBatchSize { get; set; } |
|
int NumMetaIterations { get; set; } |
|
int NumNeighbors { get; set; } |
Gets or sets K, the number of neighbours retrieved from the episodic memory. |
int OutputDimension { get; set; } |
Gets or sets the width of the output network’s output — the number of classes for a categorical head, or the target width for a Gaussian one. |
int? RandomSeed { get; set; } |
Methods
| Method | Summary |
|---|---|
IMetaLearnerOptions<T> Clone() |
|
bool IsValid() |