PositionalEncodingLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Represents a layer that adds positional encodings to input sequences.
Inheritance: LayerBase<T> → PositionalEncodingLayer<T>
Implements: IShapeContract
For Beginners
This layer adds information about position to your sequence data.
Think of it like numbering the words in a sentence:
- Without position information, a model only knows which words are in the sentence
- With position information, it knows which word comes first, second, third, etc.
For example, the sentences “dog bites man” and “man bites dog” contain the same words but have completely different meanings because of word order. Positional encoding helps models understand this difference.
The layer uses a clever mathematical pattern of sine and cosine waves to encode positions. This approach has several advantages:
- It creates a unique pattern for each position
- Similar positions have similar encodings (helpful for generalization)
- It can potentially handle sequences longer than those seen during training
- The encodings have consistent patterns that models can learn from
How It Works
The PositionalEncodingLayer adds position-dependent signals to input embeddings, which helps sequence models like Transformers understand the order of elements in a sequence. Since attention-based models have no inherent notion of sequence order, positional encodings provide this critical information. The encodings use sine and cosine functions of different frequencies to create unique position-dependent patterns.
Constructors
| Constructor | Summary |
|---|---|
PositionalEncodingLayer(int maxSequenceLength, int embeddingSize) |
Initializes a new instance of the PositionalEncodingLayer class with the specified maximum sequence length and embedding size. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
The computation engine (CPU or GPU) for vectorized operations. |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
Derived from this type’s [TensorLayout] declarations. |
void ResetState() |
Resets the internal state of the positional encoding layer. |