SiTPredictor<T>
Models & Types · AiDotNet.Diffusion.NoisePredictors
Scalable Interpolant Transformer (SiT) noise predictor for flow-based diffusion.
Inheritance: NoisePredictorBase<T> → DiTNoisePredictor<T> → SiTPredictor<T>
For Beginners
SiT is “DiT with a more flexible noise process.” The neural network that predicts the noise is exactly the DiT transformer; what changes is the math used to add/remove noise during training and sampling, which the scheduler handles. Reusing the DiT backbone here means SiT gets real self-attention, real timestep conditioning (AdaLN), and the MLP expansion of a proper transformer — not a placeholder.
How It Works
SiT (Ma et al., ECCV 2024) deliberately reuses the DiT backbone unchanged
— the paper’s §3 states it adopts the DiT architecture and isolates its
contribution to the training/sampling side (a learnable interpolant
between data and noise that unifies score-based diffusion and flow matching).
The network itself is the identical patchify → adaptive-LayerNorm
self-attention transformer stack → final AdaLN layer. We therefore realize SiT
faithfully as a configured DiTNoisePredictor: the interpolant
difference lives in the diffusion model’s scheduler (flow-matching /
stochastic-interpolant), not in this noise-prediction network.
Reference: Ma et al., “SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers”, ECCV 2024 (arXiv:2401.08740).
Example
var predictor = new SiTPredictor<float>(inputChannels: 4, hiddenSize: 1152, numLayers: 28, numHeads: 16);
var noisyLatent = Tensor<float>.CreateRandom(new[] { 1, 4, 32, 32 });
var predicted = predictor.PredictNoise(noisyLatent, timestep: 500);
Constructors
| Constructor | Summary |
|---|---|
SiTPredictor(int inputChannels, int hiddenSize, int numLayers, int numHeads, int? seed) |
Initializes a new SiT predictor on the faithful DiT backbone. |