ResidualNeuralNetwork<T>

Models & Types · AiDotNet.NeuralNetworks

Represents a Residual Neural Network, which is a type of neural network that uses skip connections to address the vanishing gradient problem in deep networks.

Inheritance: NeuralNetworkBase<T> → DeclaredModelLayoutBase<T> → VectorModelLayoutBase<T> → ResidualNeuralNetwork<T>

Implements: IAuxiliaryLossLayer<T>

For Beginners

A Residual Neural Network is like a highway system for information in a neural network.

Think of it like this:

  • In a traditional neural network, information must pass through every layer sequentially
  • In a ResNet, there are “shortcut paths” or “highways” that let information skip ahead

For example, imagine trying to pass a message through a line of 100 people:

  • In a regular network, each person must whisper to the next person in line
  • In a ResNet, some people can also shout directly to someone 5 positions ahead

This design solves a major problem: in very deep networks (many layers), information and learning signals tend to fade away or “vanish” as they travel through many layers. The shortcuts in ResNets help information flow more easily through the network, allowing for much deeper networks (some with over 100 layers!) that can learn more complex patterns.

ResNets revolutionized image recognition and are now used in many AI systems that need to identify complex patterns in data.

How It Works

A Residual Neural Network (ResNet) is an advanced neural network architecture that introduces “skip connections” or “shortcuts” that allow information to bypass one or more layers. These residual connections help address the vanishing gradient problem that occurs in very deep networks, enabling the training of networks with many more layers than previously possible. ResNets were a breakthrough in deep learning that significantly improved performance on image recognition and other tasks.

Example

var architecture = new NeuralNetworkArchitecture<float>(inputFeatures: 8, outputSize: 4);
var input = Tensor<float>.CreateRandom(new[] { 1, 784 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
    .ConfigureModel(new ResidualNeuralNetwork<float>(architecture))
    .Build(trainX, trainY);
var output = result.Predict(input);

Constructors

Constructor Summary
ResidualNeuralNetwork() Initializes a new instance of the ResidualNeuralNetwork class with the specified architecture.
ResidualNeuralNetwork(NeuralNetworkArchitecture<T> architecture, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, ResidualNeuralNetworkOptions options)

Properties

Property Summary
T AuxiliaryLossWeight { get; set; } Gets or sets the weight for the deep supervision auxiliary loss.
bool SupportsTraining { get; } Indicates whether this network supports training (learning from data).
bool UseAuxiliaryLoss { get; set; } Gets or sets whether auxiliary loss (deep supervision) should be used during training.

Methods

Method Summary
Dictionary<string, string> GetAuxiliaryLossDiagnostics() Gets diagnostic information about the deep supervision auxiliary loss.
Dictionary<string, string> GetDiagnostics() Gets diagnostic information about this component’s state and behavior.
ModelMetadata<T> GetModelMetadata() Gets metadata about the Residual Neural Network model.
T ComputeAuxiliaryLoss() Computes the auxiliary loss for deep supervision from intermediate auxiliary classifiers.
void AddAuxiliaryClassifier(ILayer<T> classifier, int layerPosition) Adds an auxiliary classifier at the specified layer position for deep supervision.