FactorizedSpatioTemporalAttention<T>
Models & Types · AiDotNet.Diffusion.Attention
Factorized spatio-temporal attention that applies spatial and temporal attention separately.
Inheritance: LayerBase<T> → FactorizedSpatioTemporalAttention<T>
Implements: IShapeContract
For Beginners
Factorized Spatio-Temporal Attention processes spatial (within-frame) and temporal (across-frame) relationships separately. This is much more efficient than joint attention while still capturing both spatial detail and temporal motion.
How It Works
References:
- Paper: “Scalable Diffusion Models with Transformers” (Peebles and Xie, 2023)
- Paper: “Video Diffusion Models” (Ho et al., 2022)
Factorized spatio-temporal attention decomposes full 3D attention into separate spatial and temporal components. This reduces computational complexity from O((THW)^2) to O(T*(HW)^2 + HW*T^2), making it feasible for high-resolution long videos.
Architecture:
- Spatial attention: self-attention within each frame (across H*W positions)
- Temporal attention: self-attention across frames (for each spatial position)
- LayerNorm + residual connections around each attention block
Constructors
| Constructor | Summary |
|---|---|
FactorizedSpatioTemporalAttention(int channels, int numHeads, int numFrames, int spatialSize) |
Initializes a new factorized spatio-temporal attention layer. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
|
int Channels { get; } |
Gets the number of channels. |
int NumFrames { get; } |
Gets the number of frames. |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
Derived from this type’s [TensorLayout] declarations. |
void ResetState() |
|
void UpdateParameters(T learningRate) |