AttentionalFeatureFusionLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Attentional feature fusion, AFF (Dai et al., “Attentional Feature Fusion”, WACV 2021), in its 2-D form: merges two feature maps with a per-position, per-channel soft selection computed from both.
Inheritance: LayerBase<T> → AttentionalFeatureFusionLayer<T>
Implements: IShapeContract
For Beginners
Two pictures of the same thing (a summary and a close-up) are blended pixel by pixel, and the blend weights are learned from both pictures, so the network decides where to trust which.
How It Works
For maps x and y of shape [batch, C, H, W], with s = x + y:
- local context
L(s)= 1×1 conv (C → C/r), BatchNorm, ReLU, 1×1 conv (C/r → C), BatchNorm; - global context
G(s)= the same stack applied to the spatial average ofs; w = sigmoid(L(s) + G(s))and the output is2·x·w + 2·y·(1 − w).
This is the fusion module LAION-CLAP uses to merge the global and local views of a long clip (Wu et al. 2023
§3.3; reference feature_fusion.AFF with type='2D', r = 4). Like the reference, a batch of
one is duplicated while training, so BatchNorm sees two identical samples rather than refusing a single one.
Call Forward(x, y). A single input is fused with itself, which yields 2·x.
Constructors
| Constructor | Summary |
|---|---|
AttentionalFeatureFusionLayer(int channels, int reduction) |
Creates the fusion module. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
void ResetState() |