MCLAlgorithm<T, TInput, TOutput>
Models & Types · AiDotNet.MetaLearning.Algorithms
Implementation of MCL: episodic meta-learning with a supervised contrastive auxiliary loss.
Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → MCLAlgorithm<T, TInput, TOutput>
Implements: IGeneratedParameterRegistrar<T>
For Beginners
MCL teaches the features two things at once: be good at the few-shot task (the episodic loss), and be well organised, with same-class examples close together and different-class examples far apart (the contrastive loss).
How It Works
Two objectives train one embedding. The episodic term is the prototypical loss (Snell et al. 2017): each class’s
support embeddings average into a prototype and each query is scored by a softmax over negative squared
distances. The auxiliary term is the supervised contrastive loss (Khosla et al. 2020, eq. 2): over the
unit-normalised outputs of a projection head, each example is pulled toward every other example of its class and
pushed away from the rest,
L_i = -mean over p in P(i) of log( exp(z_i . z_p / tau) / sum over a != i of exp(z_i . z_a / tau) ),
where P(i) is the set of other examples sharing example i’s label. The objective is
L_proto + ContrastiveWeight * L_supcon, and both terms reach the embedding through exact tape gradients.
What this replaced. The contrastive term compared single scalars of a flattened feature vector - the “cosine similarity” of two scalars is just the sign of their product - and inferred each “example’s” class from its position in that vector rather than from the labels. The projection head indexed one weight matrix through a wrapping modulus and was trained by simultaneous perturbation rather than by its gradient. No prototypes were computed at all, despite the documented algorithm, and the embedding’s own update differentiated its raw output against the class indices, so neither the episodic structure nor the contrastive term was in the meta-gradient. Adaptation multiplied the shared backbone’s parameters by a scalar derived from the projections, which mutated the meta-model every time a task was adapted.
References: Snell, J., Swersky, K., & Zemel, R. (2017). Prototypical Networks for Few-shot Learning. Khosla, P., Teterwak, P., Wang, C., et al. (2020). Supervised Contrastive Learning.
Constructors
| Constructor | Summary |
|---|---|
MCLAlgorithm(MCLOptions<T, TInput, TOutput> options) |
Initializes a new MCL meta-learner. |
Properties
| Property | Summary |
|---|---|
MetaLearningAlgorithmType AlgorithmType { get; } |
Methods
| Method | Summary |
|---|---|
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task) |
|
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch) |
Performs one meta-training step: each task’s prototypical loss plus its supervised contrastive term, differentiated exactly into the embedding network and the projection head. |