SqueezeAndExcitationLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

Represents a Squeeze-and-Excitation layer that recalibrates channel-wise feature responses adaptively.

Inheritance: LayerBase<T> → SqueezeAndExcitationLayer<T>

Implements: IAuxiliaryLossLayer<T>, IShapeContract

For Beginners

This layer helps the neural network focus on the most important features.

Think of it like how your brain works when looking at a picture:

  • First, you get a rough idea of what’s in the image (the “squeeze” step)
  • Then, you decide which parts to pay more attention to (the “excitation” step)
  • Finally, you look at the image again with this focused attention

For example, if the network is processing an image of a cat, the Squeeze-and-Excitation layer might:

  • First compress all the information to understand “this is probably a cat”
  • Then decide to pay more attention to features that look like ears, whiskers, and fur
  • Finally enhance those important features in the original image data

This helps the network become more accurate and efficient by focusing on what matters most.

How It Works

A Squeeze-and-Excitation layer enhances the representational power of a network by explicitly modeling the interdependencies between channels. It does this by performing two operations:

  1. “Squeeze” - aggregating feature maps across spatial dimensions to produce a channel descriptor
  2. “Excitation” - using this descriptor to recalibrate the original feature maps channel-wise

Constructors

Constructor Summary
SqueezeAndExcitationLayer(int channels, int reductionRatio, IActivationFunction<T> firstActivation, IActivationFunction<T> secondActivation, IInitializationStrategy<T> initializationStrategy) Initializes a new instance of the SqueezeAndExcitationLayer class with vector activation functions.
SqueezeAndExcitationLayer(int channels, int reductionRatio, IVectorActivationFunction<T> firstVectorActivation, IVectorActivationFunction<T> secondVectorActivation, IInitializationStrategy<T> initializationStrategy) Initializes a new instance of the SqueezeAndExcitationLayer class with vector activation functions.

Properties

Property Summary
T AuxiliaryLossWeight { get; set; } Gets or sets the weight for the auxiliary loss contribution.
T SparsityWeight { get; set; } Gets or sets the weight for L1 sparsity regularization on attention weights.
bool SupportsTraining { get; } Gets a value indicating whether this layer supports training.
bool UseAuxiliaryLoss { get; set; } Gets or sets a value indicating whether auxiliary loss is enabled for this layer.

Methods

Method Summary
Dictionary<string, string> GetAuxiliaryLossDiagnostics() Gets diagnostic information about the auxiliary loss computation.
Dictionary<string, string> GetDiagnostics() Gets diagnostic information about this component’s state and behavior.
Dictionary<string, string> GetParameterRoles() Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases).
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) Derived from this type’s [TensorLayout] declarations.
IReadOnlyList<Tensor<T>> GetTrainableParameters() Returns all trainable parameter tensors marked with [TrainableParameter].
T ComputeAuxiliaryLoss() Computes the auxiliary loss for this layer based on channel attention regularization.
void ClearGradients()
void ResetState() Resets the internal state of the Squeeze-and-Excitation layer.
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) Replaces trainable parameter tensors (e.g., with ParameterBuffer views).
void UpdateParameters(T learningRate) Updates the layer’s parameters using the calculated gradients and the specified learning rate.
void ZeroGrad() Clears all gradient fields discovered by convention ({paramName}Gradient).