SpectralNormalizationLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Represents a spectral normalization layer that normalizes the weights of a layer by their spectral norm.
Inheritance: LayerBase<T> → SpectralNormalizationLayer<T>
Implements: IShapeContract
For Beginners
Spectral normalization keeps layer weights from getting too large.
Key benefits:
- Stabilizes GAN training by preventing extreme weight values
- Ensures the discriminator doesn’t become too powerful too quickly
- Helps prevent mode collapse in GANs
- Computationally efficient compared to other normalization methods
How it works:
- Computes the largest singular value of the weight matrix
- Divides all weights by this value
- Keeps weights normalized throughout training
Reference: Miyato et al., “Spectral Normalization for Generative Adversarial Networks” (2018)
How It Works
Spectral normalization is a weight normalization technique that constrains the Lipschitz constant of a neural network layer. It does this by dividing the weight matrix by its largest singular value (spectral norm). This technique is particularly effective for stabilizing GAN training.
Constructors
| Constructor | Summary |
|---|---|
SpectralNormalizationLayer(ILayer<T> innerLayer, int powerIterations) |
Initializes a new instance of the SpectralNormalizationLayer class. |
Properties
| Property | Summary |
|---|---|
bool SupportsGpuTraining { get; } |
Gets a value indicating whether this layer supports GPU-resident training. |
bool SupportsTraining { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, string> GetParameterRoles() |
Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases). |
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
IReadOnlyList<Tensor<T>> GetTrainableParameters() |
Returns all trainable parameter tensors marked with [TrainableParameter]. |
IReadOnlyList<ValueTuple<string, Tensor<T>>> GetRegisteredBuffers() |
|
void ClearGradients() |
|
void ResetState() |
Resets the internal state of the layer. |
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) |
Replaces trainable parameter tensors (e.g., with ParameterBuffer views). |
void UpdateParameters(T learningRate) |
Updates the parameters of the inner layer. |
void UpdateParametersGpu(IGpuOptimizerConfig config) |
GPU-resident parameter update using the provided optimizer configuration. |
void ZeroGrad() |
Clears all gradient fields discovered by convention ({paramName}Gradient). |