ActNormFlowLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

Activation normalization (Kingma & Dhariwal 2018): a per-channel affine map z = b + e^s ⊙ x whose scale and bias are initialized from the first batch so that its output has zero mean and unit variance per channel.

Inheritance: LayerBase<T> → ActNormFlowLayer<T>

Implements: IInvertibleFlowStep<T>, IShapeContract

For Beginners

Rescales each channel to a standard range, learning the scale after setting it once from real data.

How It Works

Glow-TTS’s decoder starts every flow block with one (Kim et al. 2020, §3.3, Fig. 8; reference implementation modules.ActNorm with data-dependent initialization). The log-determinant is T · Σ s per sequence. Whether the data-dependent initialization has run is part of the layer’s persisted state, so a restored model does not re-initialize its trained values.

Constructors

Constructor Summary
ActNormFlowLayer(int channels, bool initialized) Creates the layer.

Properties

Property Summary
bool IsDataInitialized { get; } Whether the data-dependent initialization has run.
bool SupportsTraining { get; }

Methods

Method Summary
Dictionary<string, string> GetParameterRoles() Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases).
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) Derived from this type’s [TensorLayout] declarations.
IReadOnlyList<Tensor<T>> GetTrainableParameters() Returns all trainable parameter tensors marked with [TrainableParameter].
ValueTuple<Tensor<T>, Tensor<T>> Transform(Tensor<T> input, bool reverse)
void ResetState()
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) Replaces trainable parameter tensors (e.g., with ParameterBuffer views).
void UpdateParameters(T learningRate)
void ZeroGrad() Clears all gradient fields discovered by convention ({paramName}Gradient).