LFTAlgorithm<T, TInput, TOutput>
Models & Types · AiDotNet.MetaLearning.Algorithms
Learned Feature-Wise Transformation (LFT): train a metric-based few-shot learner against SIMULATED domain shift by perturbing features with affine transforms whose spread is itself learned.
Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → LFTAlgorithm<T, TInput, TOutput>
Implements: IGeneratedParameterRegistrar<T>
For Beginners
A model trained on photographs usually stumbles on sketches, because the numbers its feature extractor produces look different. Instead of forcing those numbers to match, this deliberately jiggles them while training — differently each time — so the comparison step learns to ignore their exact scale. To decide how much jiggle helps, each round holds one group of tasks out, trains on the rest with jiggling, and then checks the held-out group with the jiggling switched off.
How It Works
Tseng, Lee, Huang and Yang, “Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation” (ICLR 2020, arXiv:2001.08735). The problem: metric-based few-shot methods “often fail to generalize to unseen domains due to large discrepancy of the feature distribution across domains”. The insight is to stop trying to make the encoder produce domain-invariant features and instead make the METRIC function insensitive to which distribution it is handed, by showing it many during training.
Four properties carry the method, and all four are implemented rather than described.
Structure. Following this library’s ANIL and MbPA convention, the configured meta-model is the feature ENCODER and the algorithm owns the metric head. The transformation sits BETWEEN them, which is what puts it in the gradient path: the head’s update is computed from transformed features, so theta_f genuinely changes what the model learns.
That placement is load-bearing, and getting it wrong is silent. An earlier revision here applied
the transformation only to the reported loss while the model’s own ComputeGradients
computed its loss internally and never saw it. Training was then completely independent of
theta_f, the pseudo-unseen loss was identical for every hyper-parameter value, and the
learning-to-learn gradient was exactly zero — a model that cites the paper and implements none of
it. LearningToLearn_MovesTheHyperparameters exists to catch precisely that.
The paper inserts a transformation after each batch-normalization layer inside a residual
encoder; one insertion point at the encoder output is the same mechanism where the generic
IFullModel contract can express it, and is documented as such rather than presented as the
full multi-layer placement.
Estimator. Stage 2’s gradient with respect to theta_f flows through stage 1’s
update, so an exact computation needs second-order autodiff through an arbitrary model, which
this generic contract cannot provide. The hyper-parameters are therefore updated by SPSA — the
same estimator this library already uses for auxiliary parameters elsewhere. The OBJECTIVE is the
paper’s exactly; only the way its gradient is estimated differs.
Constructors
| Constructor | Summary |
|---|---|
LFTAlgorithm(LFTOptions<T, TInput, TOutput> options) |
Initializes LFT over a feature encoder. |
Properties
| Property | Summary |
|---|---|
FeatureWiseTransformation<T> Transformation { get; } |
Gets the transformation, so its learned hyper-parameters can be inspected. |
MetaLearningAlgorithmType AlgorithmType { get; } |
|
Vector<T> MetricHead { get; } |
Gets the metric head the transformation feeds. |
bool TransformationActive { get; set; } |
Gets whether the transformation is currently being applied. |
Methods
| Method | Summary |
|---|---|
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task) |
|
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch) |