ProtoNetsAlgorithm<T, TInput, TOutput>

Models & Types · AiDotNet.MetaLearning.Algorithms

Implementation of Prototypical Networks (ProtoNets) algorithm for few-shot learning.

Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → ProtoNetsAlgorithm<T, TInput, TOutput>

Implements: IGeneratedParameterRegistrar<T>

For Beginners

ProtoNets learns to recognise new classes from a few examples: it averages each class’s examples into a “prototype” and labels a new example by the closest prototype. Training shapes the feature space so that examples of one class cluster around their prototype.

How It Works

Prototypical Networks (Snell et al. 2017) learn an embedding in which each class is represented by the mean of its embedded support examples, its prototype, and a query is classified by a softmax over negative distances to the prototypes: p(y = k | x) = softmax(-d(f(x), c_k)), with d the squared Euclidean distance. Learning minimises -log p(y = k | x) of the true class by SGD on the embedding.

The gradient reaches the embedding through the prototypes. Support and query examples go through the embedding together, and the prototypes, distances and softmax are built from those embeddings on the tape, so the loss is differentiated through both the query embeddings and the prototypes the support embeddings form.

Extensions beyond the paper, all off by default and learned on the same objective.UseAttentionMechanism weights each support example in its prototype by a softmax of a learned attention score (uniform - the paper’s mean - at initialisation). Mahalanobis learns a diagonal metric, a Bregman divergence as Snell et al. discuss. UseAdaptiveClassScaling learns a positive scale per class slot, in the spirit of TADAM’s learned metric scaling (Oreshkin et al. 2018).

Reference: Snell, J., Swersky, K., & Zemel, R. (2017). Prototypical Networks for Few-shot Learning.

Constructors

Constructor Summary
ProtoNetsAlgorithm(ProtoNetsOptions<T, TInput, TOutput> options) Initializes a new instance of the ProtoNetsAlgorithm class.

Properties

Property Summary
MetaLearningAlgorithmType AlgorithmType { get; } Gets the algorithm type identifier for this meta-learner.

Methods

Method Summary
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task) Adapts to a new task by computing class prototypes from the support set.
ModelOptions GetOptions()
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch) Performs one meta-training step: for each episode, the prototype loss of its query examples, differentiated through the prototypes into the embedding and into any learned metric.