SqueezeAndExcitationLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Represents a Squeeze-and-Excitation layer that recalibrates channel-wise feature responses adaptively.
Inheritance: LayerBase<T> → SqueezeAndExcitationLayer<T>
Implements: IAuxiliaryLossLayer<T>, IShapeContract
For Beginners
This layer helps the neural network focus on the most important features.
Think of it like how your brain works when looking at a picture:
- First, you get a rough idea of what’s in the image (the “squeeze” step)
- Then, you decide which parts to pay more attention to (the “excitation” step)
- Finally, you look at the image again with this focused attention
For example, if the network is processing an image of a cat, the Squeeze-and-Excitation layer might:
- First compress all the information to understand “this is probably a cat”
- Then decide to pay more attention to features that look like ears, whiskers, and fur
- Finally enhance those important features in the original image data
This helps the network become more accurate and efficient by focusing on what matters most.
How It Works
A Squeeze-and-Excitation layer enhances the representational power of a network by explicitly modeling the interdependencies between channels. It does this by performing two operations:
- “Squeeze” - aggregating feature maps across spatial dimensions to produce a channel descriptor
- “Excitation” - using this descriptor to recalibrate the original feature maps channel-wise
Constructors
| Constructor | Summary |
|---|---|
SqueezeAndExcitationLayer(int channels, int reductionRatio, IActivationFunction<T> firstActivation, IActivationFunction<T> secondActivation, IInitializationStrategy<T> initializationStrategy) |
Initializes a new instance of the SqueezeAndExcitationLayer class with vector activation functions. |
SqueezeAndExcitationLayer(int channels, int reductionRatio, IVectorActivationFunction<T> firstVectorActivation, IVectorActivationFunction<T> secondVectorActivation, IInitializationStrategy<T> initializationStrategy) |
Initializes a new instance of the SqueezeAndExcitationLayer class with vector activation functions. |
Properties
| Property | Summary |
|---|---|
T AuxiliaryLossWeight { get; set; } |
Gets or sets the weight for the auxiliary loss contribution. |
T SparsityWeight { get; set; } |
Gets or sets the weight for L1 sparsity regularization on attention weights. |
bool SupportsTraining { get; } |
Gets a value indicating whether this layer supports training. |
bool UseAuxiliaryLoss { get; set; } |
Gets or sets a value indicating whether auxiliary loss is enabled for this layer. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, string> GetAuxiliaryLossDiagnostics() |
Gets diagnostic information about the auxiliary loss computation. |
Dictionary<string, string> GetDiagnostics() |
Gets diagnostic information about this component’s state and behavior. |
Dictionary<string, string> GetParameterRoles() |
Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases). |
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
Derived from this type’s [TensorLayout] declarations. |
IReadOnlyList<Tensor<T>> GetTrainableParameters() |
Returns all trainable parameter tensors marked with [TrainableParameter]. |
T ComputeAuxiliaryLoss() |
Computes the auxiliary loss for this layer based on channel attention regularization. |
void ClearGradients() |
|
void ResetState() |
Resets the internal state of the Squeeze-and-Excitation layer. |
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) |
Replaces trainable parameter tensors (e.g., with ParameterBuffer views). |
void UpdateParameters(T learningRate) |
Updates the layer’s parameters using the calculated gradients and the specified learning rate. |
void ZeroGrad() |
Clears all gradient fields discovered by convention ({paramName}Gradient). |