BatchNormalizationLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Implements batch normalization for neural networks, which normalizes the inputs across a mini-batch.
Inheritance: LayerBase<T> → BatchNormalizationLayer<T>
Implements: IShapeContract
For Beginners
Batch normalization is like standardizing test scores in a classroom.
Imagine a class where each student (input) has a raw test score. Batch normalization:
- Calculates the average score and how spread out the scores are
- Converts each score to show how many standard deviations it is from the average
- Applies adjustable scaling and shifting to the standardized scores
This helps neural networks learn more efficiently by:
- Keeping input values in a consistent range
- Reducing the “internal covariate shift” problem
- Making the network less sensitive to poor weight initialization
- Allowing higher learning rates without divergence
In practice, this means your network will typically train faster and perform better.
How It Works
Batch normalization helps stabilize and accelerate training by normalizing layer inputs. It works by normalizing each feature to have zero mean and unit variance across the batch, then applying learnable scale (gamma) and shift (beta) parameters.
Benefits include:
- Faster training convergence
- Reduced sensitivity to weight initialization
- Ability to use higher learning rates
- Acts as a form of regularization
Constructors
| Constructor | Summary |
|---|---|
BatchNormalizationLayer(double epsilon, double momentum) |
|
BatchNormalizationLayer(int numFeatures, double epsilon, double momentum) |
AiDotNet#1370 eager-init constructor. |
Properties
| Property | Summary |
|---|---|
BatchNormDataLayout Layout { get; set; } |
Initializes a new instance of the BatchNormalizationLayer class. |
bool MakesUpstreamBiasRedundant { get; } |
|
bool SupportsTraining { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, string> GetParameterRoles() |
Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases). |
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
Derived from this type’s [TensorLayout] declarations. |
IReadOnlyList<Tensor<T>> GetTrainableParameters() |
Returns all trainable parameter tensors marked with [TrainableParameter]. |
IReadOnlyList<ValueTuple<string, Tensor<T>>> GetRegisteredBuffers() |
|
OnnxLayerOutputs ConvertToOnnx(OnnxGraphBuilder builder, OnnxLayerInputs inputs) |
Emits an ONNX BatchNormalization op using the layer’s running statistics (inference mode). |
T GetEpsilon() |
Gets the epsilon value used for numerical stability. |
T GetMomentum() |
Gets the momentum value for running statistics. |
Tensor<T> GetBeta() |
Gets the beta (shift) parameters of the batch normalization layer. |
Tensor<T> GetGamma() |
Gets a value indicating whether this layer supports training mode. |
Tensor<T> GetRunningMean() |
Gets the running mean of the batch normalization layer. |
Tensor<T> GetRunningVariance() |
Gets the running variance of the batch normalization layer. |
void ClearGradients() |
|
void ResetState() |
|
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) |
Replaces trainable parameter tensors (e.g., with ParameterBuffer views). |
void SetTrainingMode(bool isTraining) |
Switches the layer between training and inference behavior. |
void UpdateParameters(T learningRate) |
Updates the layer’s parameters using the computed gradients. |
void ZeroGrad() |
Clears all gradient fields discovered by convention ({paramName}Gradient). |
void ZeroInitGamma() |