AttentionalFeatureFusionLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

Attentional feature fusion, AFF (Dai et al., “Attentional Feature Fusion”, WACV 2021), in its 2-D form: merges two feature maps with a per-position, per-channel soft selection computed from both.

Inheritance: LayerBase<T> → AttentionalFeatureFusionLayer<T>

Implements: IShapeContract

For Beginners

Two pictures of the same thing (a summary and a close-up) are blended pixel by pixel, and the blend weights are learned from both pictures, so the network decides where to trust which.

How It Works

For maps x and y of shape [batch, C, H, W], with s = x + y:

  • local context L(s) = 1×1 conv (C → C/r), BatchNorm, ReLU, 1×1 conv (C/r → C), BatchNorm;
  • global context G(s) = the same stack applied to the spatial average of s;
  • w = sigmoid(L(s) + G(s)) and the output is 2·x·w + 2·y·(1 − w).

This is the fusion module LAION-CLAP uses to merge the global and local views of a long clip (Wu et al. 2023 §3.3; reference feature_fusion.AFF with type='2D', r = 4). Like the reference, a batch of one is duplicated while training, so BatchNorm sees two identical samples rather than refusing a single one.

Call Forward(x, y). A single input is fused with itself, which yields 2·x.

Constructors

Constructor Summary
AttentionalFeatureFusionLayer(int channels, int reduction) Creates the fusion module.

Properties

Property Summary
bool SupportsTraining { get; }

Methods

Method Summary
IReadOnlyList<ILayer<T>> GetSubLayers()
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank)
void ResetState()