FlagDiTPredictor<T>

Models & Types · AiDotNet.Diffusion.NoisePredictors

Flag-DiT noise predictor for the Lumina-T2X image-generation architecture (Gao et al. 2024, “Lumina-T2X: Transforming Text into Any Modality via Flow-based Large Diffusion Transformers”, arXiv:2405.05945).

Inheritance: NoisePredictorBase<T> → FlagDiTPredictor<T>

For Beginners

Flag-DiT is the transformer behind Lumina. It cuts the image latent into little square patches (like words), uses rotary attention so it can run at any resolution, and conditions every layer on the timestep + text via lightweight scale/shift “knobs” that start at zero (so training begins from a clean identity). Sandwich normalization (a norm on both sides of each sub-layer) keeps the very deep stack numerically stable.

How It Works

Faithful Flag-DiT block stack (paper §3.1–3.2): the noisy latent is patchified into a token sequence, embedded, and processed by N transformer blocks. Each Flag-DiT block uses sandwich normalization (RMSNorm before AND after each sub-layer — Gao 2024 §3.1), grouped-query self-attention with rotary position embeddings (RoPE), a SwiGLU/GELU feed-forward, and zero-initialised adaLN conditioning (Peebles & Xie 2022; the adaLN projection produces per-block shift/scale/gate from the combined time + text embedding, and is zero-initialised so every block starts as identity). The final layer applies adaLN + RMSNorm and projects back to patch space, then unpatchifies to the latent shape. Designed for rectified-flow training (Lumina-T2X uses a flow-matching scheduler).

Constructors

Constructor Summary
FlagDiTPredictor(int inputChannels, int hiddenSize, int numLayers, int numHeads, int numKVHeads, int contextDim, int latentSize, int? seed) Initializes a new Flag-DiT predictor.

Properties

Property Summary
bool SupportsBatchedTimeEmbedding { get; }
bool SupportsCFG { get; }
bool SupportsCrossAttention { get; }
int BaseChannels { get; }
int ContextDimension { get; }
int InputChannels { get; }
int OutputChannels { get; }
int TimeEmbeddingDim { get; }

Methods

Method Summary
Tensor<T> PredictNoise(Tensor<T> noisySample, int timestep, Tensor<T> conditioning)
Tensor<T> PredictNoiseWithEmbedding(Tensor<T> noisySample, Tensor<T> timeEmbedding, Tensor<T> conditioning)