ScaledPositionalEncodingLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Adds sinusoidal positional encodings scaled by a trainable weight: x_i + α · PE(i).
Inheritance: LayerBase<T> → ScaledPositionalEncodingLayer<T>
Implements: IShapeContract
For Beginners
Tells each position where it is in the sequence, with a learned volume knob for how loudly.
How It Works
Transformer TTS (Li et al. 2019, §3.2, Eq. 8): text embeddings and mel spectrograms live at different scales, so fixed
positional embeddings “may impose heavy constraints on both the encoder and decoder pre-nets”; the triangle positional
embeddings of Vaswani et al. (PE(pos, 2i) = sin(pos / 10000^(2i/d)), PE(pos, 2i+1) = cos(...)) are added
with a trainable weight α, one for the encoder and one for the decoder. α starts at 1.
Input and output are [time, d] or [batch, time, d].
Constructors
| Constructor | Summary |
|---|---|
ScaledPositionalEncodingLayer(int dimension) |
Creates the layer for features of width dimension. |
Properties
| Property | Summary |
|---|---|
T Alpha { get; } |
The current scale α. |
bool SupportsTraining { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, string> GetParameterRoles() |
Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases). |
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
Derived from this type’s [TensorLayout] declarations. |
IReadOnlyList<Tensor<T>> GetTrainableParameters() |
Returns all trainable parameter tensors marked with [TrainableParameter]. |
void ResetState() |
|
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) |
Replaces trainable parameter tensors (e.g., with ParameterBuffer views). |
void UpdateParameters(T learningRate) |
|
void ZeroGrad() |
Clears all gradient fields discovered by convention ({paramName}Gradient). |