LFTAlgorithm<T, TInput, TOutput>

Models & Types · AiDotNet.MetaLearning.Algorithms

Learned Feature-Wise Transformation (LFT): train a metric-based few-shot learner against SIMULATED domain shift by perturbing features with affine transforms whose spread is itself learned.

Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → LFTAlgorithm<T, TInput, TOutput>

Implements: IGeneratedParameterRegistrar<T>

For Beginners

A model trained on photographs usually stumbles on sketches, because the numbers its feature extractor produces look different. Instead of forcing those numbers to match, this deliberately jiggles them while training — differently each time — so the comparison step learns to ignore their exact scale. To decide how much jiggle helps, each round holds one group of tasks out, trains on the rest with jiggling, and then checks the held-out group with the jiggling switched off.

How It Works

Tseng, Lee, Huang and Yang, “Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation” (ICLR 2020, arXiv:2001.08735). The problem: metric-based few-shot methods “often fail to generalize to unseen domains due to large discrepancy of the feature distribution across domains”. The insight is to stop trying to make the encoder produce domain-invariant features and instead make the METRIC function insensitive to which distribution it is handed, by showing it many during training.

Four properties carry the method, and all four are implemented rather than described.

Structure. Following this library’s ANIL and MbPA convention, the configured meta-model is the feature ENCODER and the algorithm owns the metric head. The transformation sits BETWEEN them, which is what puts it in the gradient path: the head’s update is computed from transformed features, so theta_f genuinely changes what the model learns.

That placement is load-bearing, and getting it wrong is silent. An earlier revision here applied the transformation only to the reported loss while the model’s own ComputeGradients computed its loss internally and never saw it. Training was then completely independent of theta_f, the pseudo-unseen loss was identical for every hyper-parameter value, and the learning-to-learn gradient was exactly zero — a model that cites the paper and implements none of it. LearningToLearn_MovesTheHyperparameters exists to catch precisely that.

The paper inserts a transformation after each batch-normalization layer inside a residual encoder; one insertion point at the encoder output is the same mechanism where the generic IFullModel contract can express it, and is documented as such rather than presented as the full multi-layer placement.

Estimator. Stage 2’s gradient with respect to theta_f flows through stage 1’s update, so an exact computation needs second-order autodiff through an arbitrary model, which this generic contract cannot provide. The hyper-parameters are therefore updated by SPSA — the same estimator this library already uses for auxiliary parameters elsewhere. The OBJECTIVE is the paper’s exactly; only the way its gradient is estimated differs.

Constructors

Constructor Summary
LFTAlgorithm(LFTOptions<T, TInput, TOutput> options) Initializes LFT over a feature encoder.

Properties

Property Summary
FeatureWiseTransformation<T> Transformation { get; } Gets the transformation, so its learned hyper-parameters can be inspected.
MetaLearningAlgorithmType AlgorithmType { get; }
Vector<T> MetricHead { get; } Gets the metric head the transformation feeds.
bool TransformationActive { get; set; } Gets whether the transformation is currently being applied.

Methods

Method Summary
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task)
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch)