InstanceNormalizationLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

Represents an Instance Normalization layer that normalizes each channel independently across spatial dimensions.

Inheritance: LayerBase<T> → InstanceNormalizationLayer<T>

Implements: IShapeContract

For Beginners

This layer helps stabilize training, especially for style transfer and image generation.

Think of Instance Normalization like adjusting the contrast of each color channel independently:

  • Each channel (e.g., red, green, blue) is normalized on its own
  • Each image in the batch is treated independently
  • This removes instance-specific contrast information

Key advantages:

  • Works well for style transfer and image generation
  • Independent of batch size (works with batch size of 1)
  • Removes instance-specific style information, making it ideal for style transfer

Common usage:

  • Style transfer networks (separates content from style)
  • GANs (Generative Adversarial Networks)
  • Image-to-image translation

How It Works

Instance Normalization normalizes each channel independently for each sample in the batch. Unlike Batch Normalization which computes statistics across the batch dimension, Instance Normalization computes statistics independently for each sample and each channel. This is essentially Group Normalization with numGroups = numChannels.

Constructors

Constructor Summary
InstanceNormalizationLayer(int numChannels, double epsilon, bool affine) Initializes a new instance of the InstanceNormalizationLayer.

Properties

Property Summary
bool Affine { get; } Gets whether affine transformation (learnable gamma and beta) is enabled.
bool MakesUpstreamBiasRedundant { get; } Gets a value indicating whether this layer supports training.
bool SupportsTraining { get; }
int NumChannels { get; } Gets the number of channels this layer normalizes.

Methods

Method Summary
Dictionary<string, string> GetParameterRoles() Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases).
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) Derived from this type’s [TensorLayout] declarations.
IReadOnlyList<Tensor<T>> GetTrainableParameters() Returns all trainable parameter tensors marked with [TrainableParameter].
T GetEpsilon() Gets the epsilon value used for numerical stability.
Tensor<T> GetBetaTensor() Gets the beta (shift) parameters as a tensor.
Tensor<T> GetGammaTensor() Gets the gamma (scale) parameters as a tensor.
Vector<T> GetBeta() Gets the beta (shift) parameters.
Vector<T> GetGamma() Gets the gamma (scale) parameters.
void ClearGradients()
void ResetState()
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) Replaces trainable parameter tensors (e.g., with ParameterBuffer views).
void UpdateParameters(T learningRate) Updates the layer’s parameters using the calculated gradients.
void ZeroGrad() Clears all gradient fields discovered by convention ({paramName}Gradient).