LFTOptions<T, TInput, TOutput>

Options & Configuration · AiDotNet.MetaLearning.Options

Configuration options for Learned Feature-Wise Transformation (LFT), Tseng, Lee, Huang and Yang, “Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation” (ICLR 2020, arXiv:2001.08735).

Inheritance: ModelOptions → LFTOptions<T, TInput, TOutput>

Implements: IMetaLearnerOptions<T>

For Beginners

A model trained only on photographs often does badly on sketches, because the numbers coming out of its feature extractor look different. Rather than trying to make those numbers identical for photos and sketches, this deliberately jiggles them during training — a little differently every time — so the comparison step learns not to depend on their exact scale. How much to jiggle is itself learned, by checking which amount helps most on a domain held out from that training step.

How It Works

Metric-based few-shot learners “often fail to generalize to unseen domains due to large discrepancy of the feature distribution across domains”. LFT’s answer is to stop trying to make the encoder domain-invariant and instead show the metric function many DIFFERENT feature distributions during training, by perturbing intermediate features with affine transforms whose scale and bias are sampled from learned hyper-parameters.

The transformations exist only during training — they are removed before the model is used — so they cost nothing at inference. What survives is a metric function that has been trained against distribution shift rather than against one fixed distribution.

Constructors

Constructor Summary
LFTOptions(IFullModel<T, TInput, TOutput> metaModel) Initializes a new instance configured to meta-train the supplied model.

Properties

Property Summary
IEpisodicDataLoader<T, TInput, TOutput> DataLoader { get; set; } Episodic data loader that supplies the meta-training tasks; null when tasks are passed to training directly.
IFullModel<T, TInput, TOutput> MetaModel { get; set; } Gets or sets the feature encoder E and metric model M.
IGradientBasedOptimizer<T, TInput, TOutput> InnerOptimizer { get; set; } Optimizer for task adaptation.
IGradientBasedOptimizer<T, TInput, TOutput> MetaOptimizer { get; set; } Optimizer for the meta-update.
ILossFunction<T> LossFunction { get; set; } Loss function for meta-training; null uses the default multi-class classification loss.
bool EnableCheckpointing { get; set; }
bool LearnTransformationHyperparameters { get; set; } Gets or sets whether the transformation hyper-parameters are LEARNED by the paper’s learning-to-learn split, rather than left at their initial values.
bool UseFirstOrder { get; set; }
double HyperparameterLearningRate { get; set; } Gets or sets the step size used to update the transformation hyper-parameters in the second stage.
double InitialBiasHyperparameter { get; set; } Gets or sets the initial value of every entry of theta_beta, the hyper-parameter governing the BIAS term’s spread.
double InitialScaleHyperparameter { get; set; } Gets or sets the initial value of every entry of theta_gamma, the hyper-parameter governing the SCALE term’s spread.
double InnerLearningRate { get; set; }
double OuterLearningRate { get; set; }
double PseudoSeenFraction { get; set; } Gets or sets the fraction of each task batch assigned to the PSEUDO-SEEN domain, the remainder becoming the pseudo-unseen domain.
double? GradientClipThreshold { get; set; }
int AdaptationSteps { get; set; }
int CheckpointFrequency { get; set; }
int EvaluationFrequency { get; set; }
int EvaluationTasks { get; set; }
int FeatureDimension { get; set; } Gets or sets the width of the feature vector the transformation modulates, i.e.
int MetaBatchSize { get; set; }
int NumMetaIterations { get; set; }
int OutputDimension { get; set; } Gets or sets the metric head’s output width — the number of classes the few-shot task discriminates between.
int? RandomSeed { get; set; }

Methods

Method Summary
IMetaLearnerOptions<T> Clone()
bool IsValid() Returns whether every option holds a value the algorithm can run with.