STDiTBlock<T>
Models & Types · AiDotNet.Diffusion.Attention
Spatial-Temporal DiT (STDiT) block for efficient video generation transformers.
Inheritance: LayerBase<T> → STDiTBlock<T>
Implements: IShapeContract
For Beginners
The STDiT (Spatial-Temporal DiT) Block alternates between spatial and temporal attention within a single transformer block. This efficient design is used in Open-Sora for balanced spatial and temporal processing.
How It Works
References:
- Paper: “Open-Sora: Democratizing Efficient Video Production for All” (2024)
- Paper: “Latte: Latent Diffusion Transformer for Video Generation” (Ma et al., 2024)
STDiT is the core building block for video DiT architectures. It combines spatial and temporal attention with cross-attention conditioning in a single transformer block:
- Spatial self-attention (within each frame)
- Temporal self-attention (across frames per spatial position)
- Cross-attention to text conditioning
- Feed-forward network
Each sub-layer uses adaptive layer normalization (adaLN-Zero) for timestep conditioning.
Constructors
| Constructor | Summary |
|---|---|
STDiTBlock(int channels, int numHeads, int contextDim, int numFrames, int spatialSize, int ffnMultiplier) |
Initializes a new STDiT block. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
|
int Channels { get; } |
Gets the hidden dimension. |
int ContextDim { get; } |
Gets the context dimension for cross-attention. |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
Derived from this type’s [TensorLayout] declarations. |
void ResetState() |
|
void UpdateParameters(T learningRate) |