ActNormFlowLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Activation normalization (Kingma & Dhariwal 2018): a per-channel affine map z = b + e^s ⊙ x whose scale
and bias are initialized from the first batch so that its output has zero mean and unit variance per channel.
Inheritance: LayerBase<T> → ActNormFlowLayer<T>
Implements: IInvertibleFlowStep<T>, IShapeContract
For Beginners
Rescales each channel to a standard range, learning the scale after setting it once from real data.
How It Works
Glow-TTS’s decoder starts every flow block with one (Kim et al. 2020, §3.3, Fig. 8; reference
implementation modules.ActNorm with data-dependent initialization). The log-determinant is
T · Σ s per sequence. Whether the data-dependent initialization has run is part of the layer’s persisted
state, so a restored model does not re-initialize its trained values.
Constructors
| Constructor | Summary |
|---|---|
ActNormFlowLayer(int channels, bool initialized) |
Creates the layer. |
Properties
| Property | Summary |
|---|---|
bool IsDataInitialized { get; } |
Whether the data-dependent initialization has run. |
bool SupportsTraining { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, string> GetParameterRoles() |
Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases). |
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
Derived from this type’s [TensorLayout] declarations. |
IReadOnlyList<Tensor<T>> GetTrainableParameters() |
Returns all trainable parameter tensors marked with [TrainableParameter]. |
ValueTuple<Tensor<T>, Tensor<T>> Transform(Tensor<T> input, bool reverse) |
|
void ResetState() |
|
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) |
Replaces trainable parameter tensors (e.g., with ParameterBuffer views). |
void UpdateParameters(T learningRate) |
|
void ZeroGrad() |
Clears all gradient fields discovered by convention ({paramName}Gradient). |