FactorizedSpatioTemporalAttention<T>

Models & Types · AiDotNet.Diffusion.Attention

Factorized spatio-temporal attention that applies spatial and temporal attention separately.

Inheritance: LayerBase<T> → FactorizedSpatioTemporalAttention<T>

Implements: IShapeContract

For Beginners

Factorized Spatio-Temporal Attention processes spatial (within-frame) and temporal (across-frame) relationships separately. This is much more efficient than joint attention while still capturing both spatial detail and temporal motion.

How It Works

References:

  • Paper: “Scalable Diffusion Models with Transformers” (Peebles and Xie, 2023)
  • Paper: “Video Diffusion Models” (Ho et al., 2022)

Factorized spatio-temporal attention decomposes full 3D attention into separate spatial and temporal components. This reduces computational complexity from O((THW)^2) to O(T*(HW)^2 + HW*T^2), making it feasible for high-resolution long videos.

Architecture:

  • Spatial attention: self-attention within each frame (across H*W positions)
  • Temporal attention: self-attention across frames (for each spatial position)
  • LayerNorm + residual connections around each attention block

Constructors

Constructor Summary
FactorizedSpatioTemporalAttention(int channels, int numHeads, int numFrames, int spatialSize) Initializes a new factorized spatio-temporal attention layer.

Properties

Property Summary
bool SupportsTraining { get; }
int Channels { get; } Gets the number of channels.
int NumFrames { get; } Gets the number of frames.

Methods

Method Summary
IReadOnlyList<ILayer<T>> GetSubLayers()
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) Derived from this type’s [TensorLayout] declarations.
void ResetState()
void UpdateParameters(T learningRate)