TETNPAlgorithm<T, TInput, TOutput>

Models & Types · AiDotNet.MetaLearning.Algorithms

Translation-Equivariant Transformer Neural Process (TE-TNP, 2024). Combines TNP’s transformer attention with relative positional encoding so that predictions are equivariant to translations of the input space.

Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → NeuralProcessBase<T, TInput, TOutput> → TETNPAlgorithm<T, TInput, TOutput>

Implements: IGeneratedParameterRegistrar<T>

How It Works

Key Idea: Standard TNP uses absolute positions in attention. TE-TNP replaces position keys/queries with sinusoidal encodings of the relative displacement between context and target points: PE(x_i - x_j). This makes the mapping equivariant to input translations.

Algorithm:

  • Encode each context pair (x, y) into a representation r_i via the base encoder.
  • Compute pairwise relative position encodings between all context points.
  • Apply multi-head self-attention using relative positional keys to refine representations.
  • Aggregate refined representations and modulate model parameters.

Positions are the support INPUTS (the mean of each example’s input features), so attention depends on x_i - x_j and the refinement is translation equivariant by construction; the paper uses no regularizer for it (EquivarianceRegWeight is an optional extra, 0 by default).

Reference: Ashman et al., “Translation-Equivariant Transformer Neural Processes”, ICML 2024.

Constructors

Constructor Summary
TETNPAlgorithm(TETNPOptions<T, TInput, TOutput> options)

Properties

Property Summary
MetaLearningAlgorithmType AlgorithmType { get; }

Methods

Method Summary
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task)
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch)