ResidualLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

Represents a residual layer that adds the identity mapping (input) to the output of an inner layer.

Inheritance: LayerBase<T> → ResidualLayer<T>

Implements: IShapeContract

For Beginners

This layer helps neural networks learn more effectively, especially when they’re very deep.

Think of it as a “correction mechanism”:

  • The inner layer tries to learn how to improve or adjust the input
  • The original input is preserved and added back in at the end
  • This allows the network to focus on learning the changes needed, rather than recreating the entire signal

Benefits include:

  • Solves the “vanishing gradient problem” that makes deep networks hard to train
  • Enables training of much deeper networks (hundreds of layers instead of just dozens)
  • Improves learning speed and accuracy

For example, in image recognition, a residual layer might learn to emphasize important features while preserving the original image information through the skip connection.

How It Works

A residual layer implements the core concept of residual networks (ResNets), where the layer learns the residual (difference) between the identity mapping and the desired underlying mapping rather than the complete transformation. This is achieved by adding a skip connection that passes the input directly to the output, where it’s added to the transformed output of an inner layer.

Constructors

Constructor Summary
ResidualLayer(ILayer<T> innerLayer, IActivationFunction<T> activationFunction) Initializes a new instance of the ResidualLayer class with the specified input shape, inner layer, and vector activation function.
ResidualLayer(ILayer<T> innerLayer, IVectorActivationFunction<T> vectorActivation) Initializes a new instance of the ResidualLayer class with the specified input shape, inner layer, and vector activation function.

Properties

Property Summary
bool IsShapeResolved { get; } A residual block is only fully shape-resolved once its WRAPPED inner layer is.
bool SupportsTraining { get; } Gets a value indicating whether this layer supports training through backpropagation.

Methods

Method Summary
IReadOnlyList<ILayer<T>> GetSubLayers()
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) Derived from this type’s [TensorLayout] declarations.
void ClearGradients()
void ResetState() Resets the internal state of the residual layer and its inner layer.
void SetInnerLayer(ILayer<T> innerLayer) Sets a new inner layer for the residual layer.
void SetTrainingMode(bool isTraining) Gets all trainable parameters from the inner layer as a single vector.
void UpdateParameters(T learningRate) Updates the parameters of the inner layer using the calculated gradients.