EMMDiTPredictor<T>

Models & Types · AiDotNet.Diffusion.NoisePredictors

E-MMDiT (Efficient Multimodal Diffusion Transformer) noise predictor — a compact configuration of the MMDiT architecture (Stable Diffusion 3, Esser et al. 2024) for parameter-efficient inference.

Inheritance: NoisePredictorBase<T> → MMDiTNoisePredictor<T> → EMMDiTPredictor<T>

For Beginners

This is a “small but real” version of the Stable Diffusion 3 transformer. It does exactly what the big one does — image and text tokens attend to each other jointly, and the timestep controls every layer — just with fewer/narrower layers so it runs lighter.

How It Works

E-MMDiT keeps the full MMDiT block — dual image/text streams with joint (concatenated) self-attention, separate per-stream MLPs, adaptive-LayerNorm modulation from the timestep + pooled-text conditioning, and QK-normalization — but at a smaller width/depth (hidden 1024, 12 joint blocks, 16 heads) than the full SD3 MMDiT. It is therefore realized as a configured MMDiTNoisePredictor: the architecture is identical to MMDiT, only the scale differs. (The earlier Dense-only placeholder had no attention or timestep conditioning and was not the MMDiT architecture it claimed.)

Reference: Esser et al., “Scaling Rectified Flow Transformers for High-Resolution Image Synthesis”, ICML 2024 (arXiv:2403.03206).

Constructors

Constructor Summary
EMMDiTPredictor(int inputChannels, int contextDim, int? seed) Initializes a new E-MMDiT predictor at the compact configuration: the constructor earlier releases compiled against, kept so they still bind after an upgrade.
EMMDiTPredictor(int inputChannels, int contextDim, int? seed, int hiddenSizeOverride, int numLayersOverride, int numHeadsOverride) Initializes a new E-MMDiT predictor on the faithful MMDiT backbone at the compact (hidden 1024, 12 joint blocks, 16 heads) configuration.