ResidualConvReluNormLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

A residual stack of Conv1D → LayerNorm → ReLU → Dropout layers with a zero-initialized 1×1 output projection: y = x + P(f(x)), which starts as the identity.

Inheritance: LayerBase<T> → ResidualConvReluNormLayer<T>

Implements: IShapeContract

For Beginners

A few convolutions that learn a correction to their input, starting from “no change”.

How It Works

Glow-TTS’s encoder pre-net (Kim et al. 2020, §3.3: “the pre-net … three convolutional layers … with a residual connection”; reference implementation modules.ConvReluNorm: kernel 5, dropout 0.5, layer normalization over channels with ε = 1e-4, and a projection whose weight and bias start at zero).

Input and output are [time, channels] or [batch, time, channels].

Constructors

Constructor Summary
ResidualConvReluNormLayer(int channels, int kernelSize, int layers, double dropoutRate) Creates the stack.

Properties

Property Summary
bool SupportsTraining { get; }

Methods

Method Summary
Dictionary<string, string> GetParameterRoles() Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases).
IReadOnlyList<ILayer<T>> GetSubLayers()
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) Derived from this type’s [TensorLayout] declarations.
IReadOnlyList<Tensor<T>> GetTrainableParameters() Returns all trainable parameter tensors marked with [TrainableParameter].
void ResetState()
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) Replaces trainable parameter tensors (e.g., with ParameterBuffer views).
void ZeroGrad() Clears all gradient fields discovered by convention ({paramName}Gradient).