TemporalSelfAttention<T>

Models & Types · AiDotNet.Diffusion.Attention

Temporal self-attention layer for video diffusion models.

Inheritance: LayerBase<T> → TemporalSelfAttention<T>

Implements: IShapeContract

For Beginners

Temporal Self-Attention computes attention exclusively along the time dimension - each spatial position attends to the same position across all frames. This captures how individual pixels or regions change over time.

How It Works

References:

  • Paper: “Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models” (Blattmann et al., 2023)
  • Paper: “AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models” (Guo et al., 2023)

Temporal self-attention applies attention across the time dimension of video features. For each spatial position, tokens from all frames attend to each other, enabling the model to learn temporal relationships and maintain consistency across frames.

Architecture:

  • Input: [batch * height * width, frames, channels]
  • Reshapes spatial dims into batch for per-position temporal attention
  • Each spatial location independently attends across frames
  • Output: same shape as input with temporal information mixed

Constructors

Constructor Summary
TemporalSelfAttention(int channels, int numHeads, int numFrames, int spatialSize) Initializes a new temporal self-attention layer.

Properties

Property Summary
bool SupportsTraining { get; }
int Channels { get; } Gets the number of channels.
int NumFrames { get; } Gets the number of frames.
int NumHeads { get; } Gets the number of attention heads.

Methods

Method Summary
Dictionary<string, string> GetDiagnostics()
IReadOnlyList<ILayer<T>> GetSubLayers()
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) Derived from this type’s [TensorLayout] declarations.
void ResetState()
void UpdateParameters(T learningRate)