TADAMAlgorithm<T, TInput, TOutput>
Models & Types · AiDotNet.MetaLearning.Algorithms
Implementation of Task-Dependent Adaptive Metric (TADAM) algorithm for few-shot learning.
Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → TADAMAlgorithm<T, TInput, TOutput>
Implements: IGeneratedParameterRegistrar<T>
For Beginners
TADAM improves on ProtoNets by making the feature extractor “aware” of the current task:
How it works:
- Extract features from support set examples
- Summarize the task as the average of its class prototypes
- Use that summary to rescale each feature dimension for this task
- Compute prototypes from the rescaled features
- Classify queries by their scaled distance to each prototype
How It Works
TADAM (Oreshkin, Rodriguez and Lacoste, NeurIPS 2018) extends Prototypical Networks with:
- Metric scaling (Sec. 2.1, Eq. 3):
p(y = k | x) = softmax(-alpha d(f(x), c_k))with ONE learned scalar
alpha - not a per-dimension weight vector and not a separate temperature, which alpha already is.
- Task conditioning (Sec. 2.2): a task-embedding network (TEN) maps the task representation
c_bar, the
mean of the class prototypes, to FiLM parameters, with a scalar post-multiplier that starts at zero and is L2-penalized, so an untrained TEN is the identity.
- Auxiliary co-training (Sec. 2.3), here an extra classification term on the support set.
The feature extractor is a black-box model, so FiLM is applied to its OUTPUT features rather than inside its
layers: h' = (1 + gamma) h. A shift beta would cancel exactly in every query-prototype difference
at that level, so only the scale is generated. The extractor, alpha and the TEN all train on the exact
gradient of the episode’s query cross-entropy: the logits are built with engine tensor ops, so the extractor’s
gradient flows through the prototypes and the metric (it was trained against the raw labels before).
Algorithm - TADAM:
Constructors
| Constructor | Summary |
|---|---|
TADAMAlgorithm(TADAMOptions<T, TInput, TOutput> options) |
Initializes a new instance of the TADAMAlgorithm class. |
Properties
| Property | Summary |
|---|---|
MetaLearningAlgorithmType AlgorithmType { get; } |
Methods
| Method | Summary |
|---|---|
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task) |
|
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch) |