TemporalSelfAttention<T>
Models & Types · AiDotNet.Diffusion.Attention
Temporal self-attention layer for video diffusion models.
Inheritance: LayerBase<T> → TemporalSelfAttention<T>
Implements: IShapeContract
For Beginners
Temporal Self-Attention computes attention exclusively along the time dimension - each spatial position attends to the same position across all frames. This captures how individual pixels or regions change over time.
How It Works
References:
- Paper: “Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models” (Blattmann et al., 2023)
- Paper: “AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models” (Guo et al., 2023)
Temporal self-attention applies attention across the time dimension of video features. For each spatial position, tokens from all frames attend to each other, enabling the model to learn temporal relationships and maintain consistency across frames.
Architecture:
- Input: [batch * height * width, frames, channels]
- Reshapes spatial dims into batch for per-position temporal attention
- Each spatial location independently attends across frames
- Output: same shape as input with temporal information mixed
Constructors
| Constructor | Summary |
|---|---|
TemporalSelfAttention(int channels, int numHeads, int numFrames, int spatialSize) |
Initializes a new temporal self-attention layer. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
|
int Channels { get; } |
Gets the number of channels. |
int NumFrames { get; } |
Gets the number of frames. |
int NumHeads { get; } |
Gets the number of attention heads. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, string> GetDiagnostics() |
|
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
Derived from this type’s [TensorLayout] declarations. |
void ResetState() |
|
void UpdateParameters(T learningRate) |