DecoderLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Represents a Decoder Layer in a Transformer architecture.
Inheritance: LayerBase<T> → DecoderLayer<T>
Implements: IShapeContract
For Beginners
The Decoder Layer helps in generating output sequences (like translations) by considering both what it has generated so far and the information from the input sequence.
It’s like writing a story where you:
- Look at what you’ve written so far (self-attention)
- Refer back to your source material (cross-attention to encoder output)
- Think about how to continue the story (feed-forward network)
This process helps in creating coherent and context-aware outputs.
How It Works
The Decoder Layer is a key component in sequence-to-sequence models, particularly in Transformer architectures. It processes the target sequence and incorporates information from the encoder output.
Constructors
| Constructor | Summary |
|---|---|
DecoderLayer(int attentionSize, int feedForwardSize, IActivationFunction<T> activation) |
Initializes a new instance of the DecoderLayer class with vector activation. |
DecoderLayer(int attentionSize, int feedForwardSize, IVectorActivationFunction<T> activation) |
Initializes a new instance of the DecoderLayer class with vector activation. |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<LayerPort> InputPorts { get; } |
The three named inputs ForwardTracedPorts reads, so that anything routing by declared ports wires the encoder memory instead of silently dropping it. |
bool SupportsTraining { get; } |
Gets a value indicating whether this layer supports training. |
int InputSize { get; set; } |
Gets the size of the input features for this layer. |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
Derived from this type’s [TensorLayout] declarations. |
LayerInputDomain GetInputDomain(int[] inputShape) |
Generated from the primary [TensorPort] declaration. |
Tensor<T> Forward(NamedInputs<T> inputs) |
|
Tensor<T> ForwardDefault(Tensor<T> input) |
|
void ClearGradients() |
|
void ResetState() |
Resets the state of the decoder layer. |
void UpdateParameters(T learningRate) |
Updates the layer’s parameters with the provided values. |
void UpdateParameters(Vector<T> parameters) |
Updates the layer’s parameters with the provided values. |
void UpdateParametersGpu(IGpuOptimizerConfig config) |
Updates layer parameters using GPU-resident optimizer. |