ResidualConvReluNormLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
A residual stack of Conv1D → LayerNorm → ReLU → Dropout layers with a zero-initialized 1×1 output
projection: y = x + P(f(x)), which starts as the identity.
Inheritance: LayerBase<T> → ResidualConvReluNormLayer<T>
Implements: IShapeContract
For Beginners
A few convolutions that learn a correction to their input, starting from “no change”.
How It Works
Glow-TTS’s encoder pre-net (Kim et al. 2020, §3.3: “the pre-net … three convolutional layers … with a residual
connection”; reference implementation modules.ConvReluNorm: kernel 5, dropout 0.5, layer normalization over
channels with ε = 1e-4, and a projection whose weight and bias start at zero).
Input and output are [time, channels] or [batch, time, channels].
Constructors
| Constructor | Summary |
|---|---|
ResidualConvReluNormLayer(int channels, int kernelSize, int layers, double dropoutRate) |
Creates the stack. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, string> GetParameterRoles() |
Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases). |
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
Derived from this type’s [TensorLayout] declarations. |
IReadOnlyList<Tensor<T>> GetTrainableParameters() |
Returns all trainable parameter tensors marked with [TrainableParameter]. |
void ResetState() |
|
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) |
Replaces trainable parameter tensors (e.g., with ParameterBuffer views). |
void ZeroGrad() |
Clears all gradient fields discovered by convention ({paramName}Gradient). |