TETNPAlgorithm<T, TInput, TOutput>
Models & Types · AiDotNet.MetaLearning.Algorithms
Translation-Equivariant Transformer Neural Process (TE-TNP, 2024). Combines TNP’s transformer attention with relative positional encoding so that predictions are equivariant to translations of the input space.
Inheritance: ModelBase<T, TInput, TOutput> → MetaLearnerBase<T, TInput, TOutput> → NeuralProcessBase<T, TInput, TOutput> → TETNPAlgorithm<T, TInput, TOutput>
Implements: IGeneratedParameterRegistrar<T>
How It Works
Key Idea: Standard TNP uses absolute positions in attention. TE-TNP replaces position keys/queries with sinusoidal encodings of the relative displacement between context and target points: PE(x_i - x_j). This makes the mapping equivariant to input translations.
Algorithm:
- Encode each context pair (x, y) into a representation r_i via the base encoder.
- Compute pairwise relative position encodings between all context points.
- Apply multi-head self-attention using relative positional keys to refine representations.
- Aggregate refined representations and modulate model parameters.
Positions are the support INPUTS (the mean of each example’s input features), so attention depends on
x_i - x_j and the refinement is translation equivariant by construction; the paper uses no regularizer for
it (EquivarianceRegWeight is an optional extra, 0 by default).
Reference: Ashman et al., “Translation-Equivariant Transformer Neural Processes”, ICML 2024.
Constructors
| Constructor | Summary |
|---|---|
TETNPAlgorithm(TETNPOptions<T, TInput, TOutput> options) |
Properties
| Property | Summary |
|---|---|
MetaLearningAlgorithmType AlgorithmType { get; } |
Methods
| Method | Summary |
|---|---|
IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task) |
|
T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch) |