Diffusion

465 public types in this namespace, organized by kind.

Models & Types (418)

Type Summary
ARDiffusionModel<T> AR-Diffusion model combining autoregressive and diffusion generation in latent space.
ActivationPoolStats Statistics about activation pool usage.
AdaptiveProjectedGuidance<T> Adaptive Projected Guidance (APG) for diffusion model inference.
AdversarialDistillationTrainer<T> Trainer for Adversarial Diffusion Distillation (ADD) as used in SD/SDXL Turbo.
AllegroModel<T> Allegro efficient DiT-based video generation model.
AlphaCompositor<T> Alpha compositing for layered diffusion outputs with transparency support.
AnimateDiffModel<T> AnimateDiff model for text-to-video and image-to-video generation.
AnyEditModel<T> AnyEdit model for handling diverse editing types through unified instruction understanding.
AsymmDiTPredictor<T> Asymmetric Diffusion Transformer (AsymmDiT) noise predictor for video generation (Genmo Mochi 1 architecture).
AsyncOnlineDPO<T> Asynchronous Online DPO for diffusion model alignment with on-policy generation.
AudioLDM2Model<T> AudioLDM 2 - Enhanced Audio Latent Diffusion Model with dual text encoders.
AudioLDMModel<T> Audio Latent Diffusion Model (AudioLDM) for text-to-audio generation.
AudioProcessor<T> Complete audio processing pipeline for diffusion-based audio generation.
AudioVAE<T> AudioLDM’s variational autoencoder: the latent-diffusion KL autoencoder applied to a log-mel spectrogram as a one-channel image.
AuraFlowModel<T> AuraFlow model — open-source flow-matching text-to-image model.
AutoRegressiveMaskedDiffusion<T> Autoregressive Masked Diffusion for hybrid discrete-continuous image generation.
AutoencoderKL<T> The KL-regularized autoencoder of latent diffusion (Rombach et al., 2022), the VAE of Stable Diffusion.
BlendedDiffusionModel<T> Blended Diffusion model for text-guided local image editing within user-specified masks.
BrushEditModel<T> BrushEdit model combining multimodal LLM understanding with BrushNet inpainting.
BrushNetModel<T> BrushNet model for plug-and-play dual-branch inpainting with any diffusion backbone.
BrushNetXModel<T> BrushNet-X model extending BrushNet to SDXL architecture for high-resolution inpainting.
CATDMModel<T> CATDM model for controllable appearance transfer diffusion for virtual try-on.
CCSRModel<T> CCSR: Content-Consistent Super-Resolution with diffusion-based controllable restoration.
CLIPTextConditioner<T> CLIP text encoder conditioning module (Radford et al., ICML 2021).
CameraEmbedding<T> Camera embedding for view conditioning.
CameraPose Camera pose for rendering.
CameraPoseEncoder<T> Encodes camera pose (polar, azimuth, radius) into embeddings.
CannyEdgePreprocessor<T> Canny edge detection preprocessor for ControlNet conditioning.
CatVTONModel<T> CatVTON model for concatenation-based virtual try-on without warping modules.
CauchyNoiseSchedule<T> Cauchy noise schedule using extremely heavy-tailed Cauchy distribution for noise sampling.
Causal3DVAE<T> CogVideoX’s causal 3D VAE: video compressed 8x in height and width and 4x in time, every convolution causal in time.
CausalTemporalAttention<T> Causal temporal attention for autoregressive video generation.
ChatGLM3TextConditioner<T> ChatGLM3 text encoder conditioning module (Zeng et al.
CogVideoModel<T> CogVideo/CogVideoX model for text-to-video and image-to-video generation.
CogVideoX15Model<T> CogVideoX 1.5 model with 10-second any-resolution video generation.
CogView4Model<T> CogView-4 model for bilingual text-to-image generation by Zhipu AI.
ColorPalettePreprocessor<T> Color palette extraction preprocessor for ControlNet conditioning.
ConceptEraser<T> Concept erasure for removing unwanted concepts from diffusion model representations.
ConsisLoRAModel<T> ConsisLoRA model for consistent style transfer using LoRA with content-style disentanglement.
ConsistencyDistillationSampling<T> Consistency Distillation Sampling (CSD) for 3D generation with consistency constraints.
ConsistencyDistillationTrainer<T> Trainer for consistency distillation from a pretrained diffusion model.
ConsistencyModelScheduler<T> Consistency Model scheduler for single-step or few-step diffusion sampling.
ConsistencyModel<T> Consistency Model for single-step or few-step image generation.
ConsistencyTrainingTrainer<T> Trainer for consistency training from scratch without a pretrained teacher model.
ContentShufflePreprocessor<T> Content shuffle preprocessor for ControlNet conditioning.
ControlARModel<T> ControlAR model combining autoregressive generation with spatial control.
ControlNeXtModel<T> ControlNeXt model with improved efficiency and generalization.
ControlNetCondition<T> Represents a single control condition input for multi-control ControlNet composition.
ControlNetEncoder<T> ControlNet encoder per Zhang et al.
ControlNetFluxModel<T> ControlNet adapted for the FLUX.1 architecture with flow matching.
ControlNetInpaintingModel<T> ControlNet Inpainting model with mask-aware conditioning.
ControlNetLiteModel<T> Lightweight ControlNet model with reduced parameter count for faster inference.
ControlNetModel<T> ControlNet model for adding spatial conditioning to diffusion models.
ControlNetPlusPlusFluxModel<T> ControlNet++ adapted for FLUX architecture with reward-guided training.
ControlNetPlusPlusModel<T> ControlNet++ model with improved conditioning via reward-guided training.
ControlNetQRModel<T> ControlNet QR model specialized for embedding QR codes in generated artwork.
ControlNetSD3Model<T> ControlNet adapted for Stable Diffusion 3’s MMDiT architecture.
ControlNetTileModel<T> ControlNet Tile model for upscaling and detail enhancement.
ControlNetUnionModel<T> ControlNet Union model for unified multi-condition image generation with a single ControlNet.
ControlNetUnionProModel<T> ControlNet Union Pro model that supports multiple control types in a single model.
ControlNetXSModel<T> ControlNet-XS model — lightweight control network with minimal parameters.
CosineShiftedSchedule<T> Cosine-shifted noise schedule for resolution-adapted diffusion training.
CosmosModel<T> NVIDIA Cosmos physics-aware world generation model.
CubeDiffModel<T> CubeDiff model for cubemap-based panoramic generation with cross-face consistency.
CycleGANTurboModel<T> CycleGAN-Turbo model combining CycleGAN unpaired translation with a diffusion backbone.
D3PO<T> Direct Preference for Denoising Diffusion Policy Optimization (D3PO).
DDIMScheduler<T> DDIM (Denoising Diffusion Implicit Models) scheduler implementation.
DDPMModel<T> DDPM (Denoising Diffusion Probabilistic Models) implementation.
DDPMScheduler<T> DDPM (Denoising Diffusion Probabilistic Models) scheduler implementation.
DEISMultistepScheduler<T> Diffusion Exponential Integrator Sampler (DEIS) for fast diffusion model sampling.
DIAMONDModel<T> DIAMOND diffusion-based game engine from video with action conditioning.
DMD2Model<T> Distribution Matching Distillation v2 (DMD2) for single-step high-fidelity generation.
DPMSolverMultistepScheduler<T> DPM-Solver++ multistep scheduler for fast diffusion model sampling.
DPMSolverSDEScheduler<T> DPM++ 2M SDE scheduler — stochastic variant of DPM-Solver++ multistep.
DPMSolverSinglestepScheduler<T> DPM++ 2S Ancestral scheduler — single-step DPM-Solver++ with ancestral sampling.
DPMSolverV3Scheduler<T> DPM-Solver v3 scheduler with empirical model statistics for improved convergence.
DWPosePreprocessor<T> DWPose whole-body keypoint detection preprocessor for ControlNet conditioning.
DallE2Model<T> DALL-E 2 (unCLIP) model for text-to-image generation via CLIP-guided diffusion.
DallE3Model<T> DALL-E 3 style text-to-image generation model with advanced prompt understanding and high-fidelity image generation capabilities.
DeepCompressionVAE<T> Deep Compression Autoencoder (DC-AE) for extremely high spatial compression.
DeepFloydIFModel<T> DeepFloyd IF model for cascaded text-to-image generation in pixel space.
DenoisedScoreDistillation<T> Denoised Score Distillation (DSD) for artifact-free 3D generation.
DepthEstimationPreprocessor<T> Depth estimation preprocessor for ControlNet conditioning.
DiTBlock<T> Block structure for DiT transformer layers containing attention, MLP, and conditioning layers.
DiTNoisePredictor<T> Diffusion Transformer (DiT) noise predictor for diffusion models.
DiffBIRModel<T> DiffBIR model for blind image restoration with generative diffusion prior.
DiffEditModel<T> DiffEdit model for automatic mask generation and text-guided image editing.
DiffPanoModel<T> DiffPano model for scalable panorama generation with spherical epipolar-aware diffusion.
DiffusionAttention<T> Memory-efficient attention layer for diffusion models using Flash Attention.
DiffusionCrossAttention<T> Cross-attention layer for diffusion models with Flash Attention optimization.
DiffusionDPO<T> Direct Preference Optimization (DPO) adapted for diffusion models.
DiffusionMemoryManager<T> Memory management utilities for diffusion models including gradient checkpointing, activation pooling, and model sharding integration.
DiffusionRLHF<T> Reinforcement Learning from Human Feedback (RLHF) adapted for diffusion models.
DiffusionResBlock<T> Implements a residual block per the DDPM (Ho et al.
DiscretizedRFScheduler<T> Discretized Rectified Flow scheduler with optimized timestep selection.
DistilledT5TextConditioner<T> Distilled T5 text encoder conditioning module — same architecture as T5 but half the layer count, per the DistilBERT-style knowledge-distillation recipe (Sanh et al., 2019).
DistributionMatchingDistiller<T> Distribution Matching Distillation (DMD) trainer for single-step generation via distribution alignment.
DownBlock<T> Downsampling block for VAE encoder with multiple ResBlocks and strided convolution.
DreamFusionModel<T> DreamFusion model for text-to-3D generation via Score Distillation Sampling (SDS).
DreamGaussianModel<T> DreamGaussian model for fast 3D Gaussian splatting generation with Score Distillation Sampling.
DreamMesh<T> Simple mesh representation for DreamFusion.
DualTextConditioner<T> Dual text encoder conditioning module combining CLIP and T5 encoders.
DynamicCFGScheduler<T> Dynamic Classifier-Free Guidance that adjusts scale per timestep.
EDiffIModel<T> eDiff-I model — ensemble of expert denoisers for text-to-image diffusion.
ELLAAdapter<T> ELLA (Efficient Large Language Model Adapter) guidance adapter for enhanced text understanding.
EMMDiTPredictor<T> E-MMDiT (Efficient Multimodal Diffusion Transformer) noise predictor — a compact configuration of the MMDiT architecture (Stable Diffusion 3, Esser et al.
EQVAEModel<T> Equivariance-preserving VAE (EQ-VAE) with improved latent regularity.
EasyConsistencyModel<T> Easy Consistency Tuning (ECT) model for simple, stable consistency model training.
EmuVideo2Model<T> Emu Video 2 with improved generation quality and motion.
EmuVideoModel<T> Emu Video high-quality video generation with temporal consistency.
EulerAncestralDiscreteScheduler<T> Euler Ancestral discrete scheduler for diffusion model sampling.
EulerDiscreteScheduler<T> Euler discrete scheduler for diffusion model sampling.
ExFMScheduler<T> ExFM (Exponential Flow Matching) scheduler with exponential time discretization.
FactorizedSpatioTemporalAttention<T> Factorized spatio-temporal attention that applies spatial and temporal attention separately.
FashionVDMModel<T> FashionVDM model for video-based virtual try-on with temporal consistency.
FateZeroModel<T> FateZero zero-shot video editing via attention blending.
FlagDiTPredictor<T> Flag-DiT noise predictor for the Lumina-T2X image-generation architecture (Gao et al.
FlashDiffusionModel<T> Flash Diffusion model for rapid few-step generation via progressive attention distillation.
FlowDPMSolverScheduler<T> Flow DPM-Solver scheduler applying DPM-Solver acceleration to rectified flow models.
FlowEditModel<T> FlowEdit model for image editing via rectified flow inversion and re-generation.
FlowMapModel<T> FlowMap model for one-step generation via flow-map learning.
FlowMatchingScheduler<T> Flow matching scheduler implementing rectified flow ODE sampling.
FlowVidModel<T> FlowVid optical flow guided video-to-video synthesis.
Flux1Model<T> FLUX.1 model for high-quality text-to-image generation by Black Forest Labs.
Flux2Model<T> FLUX.2 model for next-generation text-to-image generation by Black Forest Labs.
Flux2SchnellModel<T> FLUX.2 Schnell for next-generation ultra-fast 1-4 step image generation.
FluxDoubleStreamPredictor<T> FLUX.1 double-stream transformer noise predictor (Black Forest Labs).
FluxInpaintingModel<T> FLUX Fill model for mask-guided inpainting using rectified flow transformers.
FluxSchnellModel<T> FLUX.1 Schnell for ultra-fast 1-4 step generation from the FLUX architecture.
FreeInpaintModel<T> Free-form inpainting model using masked diffusion with irregular mask support.
FreeNoiseModule<T> FreeNoise module for tuning-free longer video generation via noise rescheduling.
FreeNoiseVideoModel<T> FreeNoise extended video generation through noise rescheduling.
Full3DAttention<T> Full 3D attention across all spatio-temporal positions simultaneously.
GameGenXModel<T> GameGen-X open-world game video generation with interactive control.
GemmaTextConditioner<T> Gemma text encoder conditioning module (Gemma Team 2024).
Genie2Model<T> Genie 2 real-time interactive 3D environment generation.
GriffinLim<T> Griffin-Lim algorithm for audio reconstruction from magnitude spectrograms.
HDPainterModel<T> HD-Painter model for high-resolution inpainting with Prompt-Aware Introverted Attention (PAIntA).
HeunDiscreteScheduler<T> Heun discrete scheduler for diffusion model sampling using second-order Heun’s method.
HiDreamModel<T> HiDream-I1 model for high-quality imaginative text-to-image generation.
HunyuanDiTModel<T> Hunyuan-DiT model — bilingual (Chinese-English) DiT text-to-image model by Tencent.
HunyuanVideo15Model<T> HunyuanVideo 1.5 efficient video generation model for consumer GPUs.
HunyuanVideoModel<T> HunyuanVideo model for dual-stream DiT video generation with unified image-video capability.
HybridODESDEScheduler<T> Hybrid ODE/SDE scheduler that transitions between deterministic and stochastic sampling.
HyperSDModel<T> Hyper-SD model for unified 1-8 step generation via trajectory-segmented distillation.
ICEditModel<T> ICEdit model for in-context learning based image editing without per-task fine-tuning.
IDMVTONModel<T> IDM-VTON model for image-based virtual try-on with high-fidelity garment transfer.
IPAdapterFaceIDModel<T> IP-Adapter FaceID model for face-specific identity preservation using facial recognition embeddings.
IPAdapterFaceIDPlusModel<T> IP-Adapter FaceID Plus model for face-identity-preserving generation.
IPAdapterModel<T> IP-Adapter model for image-based prompt conditioning in diffusion models.
IPAdapterPlusModel<T> IP-Adapter Plus model for image prompt conditioning in diffusion models.
Ideogram3Model<T> Ideogram 3 model for text-to-image generation with superior text rendering.
ImageEncoder<T> Image encoder for extracting features from reference images.
ImageProjector<T> Projects image features to text embedding space.
Imagen2Model<T> Imagen 2 model for improved cascaded text-to-image generation.
Imagen3Model<T> Imagen 3 model for text-to-image generation by Google DeepMind.
ImagenModel<T> Imagen model for cascaded text-to-image generation with T5 text encoding.
ImagicModel<T> Imagic model for text-aligned real image editing via embedding optimization and model fine-tuning.
ImprovedConsistencyModel<T> Improved Consistency Training (iCT) model for single-step image generation.
ImprovedVideoVAE<T> Improved Video VAE with temporal-aware compression and motion consistency.
InpaintingMaskPreprocessor<T> Inpainting mask preprocessor for ControlNet conditioning.
InstaFlowModel<T> InstaFlow model for one-step text-to-image via rectified flow straightening.
Instant3DModel<T> Instant3D model – fast text-to-3D with feed-forward generation.
InstantIDModel<T> InstantID model — zero-shot identity-preserving image generation.
InstantStyleModel<T> InstantStyle model for zero-shot style transfer using IP-Adapter with style-content separation.
InstructPix2PixModel<T> InstructPix2Pix model for instruction-based image editing via natural language text prompts.
InstructVid2VidModel<T> InstructVid2Vid natural language instruction-based video editing.
IntervalScoreMatching<T> Interval Score Matching (ISM) for improved 3D score distillation.
JEN1Model<T> JEN-1 model for high-fidelity text-to-music generation.
KLoRAStyleModel<T> K-LoRA Style model for composable style transfer by merging multiple LoRA style adapters.
KandinskyModel<T> Kandinsky 2.2/3.0 model for text-to-image generation.
Kling26Model<T> Kling 2.6 with simultaneous audio-visual generation.
KlingModel<T> Kling model — 3D spatiotemporal attention video generation by Kuaishou.
KolorsModel<T> Kolors model — ChatGLM3-powered bilingual text-to-image model by Kwai.
LCMScheduler<T> LCM (Latent Consistency Model) scheduler for ultra-fast diffusion sampling.
LEDITSPPModel<T> LEDITS++ model for precise multi-concept editing of real images with automatic masking.
LGMModel<T> LGM (Large Gaussian Model) for feed-forward 3D Gaussian generation from multi-view images.
LMSDiscreteScheduler<T> Linear Multi-Step (LMS) discrete scheduler for diffusion model sampling.
LTXVideoModel<T> LTX-Video model for lightweight real-time video generation with extreme latent compression.
LaplaceNoiseSchedule<T> Laplace noise schedule using heavy-tailed Laplace distribution for noise sampling.
LatentConsistencyModel<T> Latent Consistency Model (LCM) for fast few-step image generation.
LatentInitializer<T> Initializes latent tensors for diffusion generation with various noise strategies.
LatentMaskBlender<T> Blends latent representations using a mask for seamless inpainting and region editing.
LatteModel<T> Latte model for Latent Diffusion Transformer video generation with factorized spatial-temporal attention.
LayerCheckpointState<T> State for layer-based gradient checkpointing.
LineArtPreprocessor<T> Line art extraction preprocessor for ControlNet conditioning.
LiteVAEModel<T> Lightweight VAE optimized for fast encoding/decoding on edge and mobile devices.
LogSNRImportanceSampling<T> Log-SNR importance sampling for efficient timestep selection during diffusion training.
LoongModel<T> Loong autoregressive LLM-based minute-long video generator.
LumaRay2Model<T> Luma Ray 2 video model with fast natural motion and better physics.
LumaRay3Model<T> Luma Ray 3 with Hi-Fi Diffusion for 4K HDR video.
LumiereModel<T> Lumiere Space-Time UNet for single-pass 80-frame video generation.
LuminaImage2Model<T> Lumina Image 2.0 model for high-resolution text-to-image generation.
LuminaT2XModel<T> Lumina-T2X unified framework for transforming text into any modality.
LuminaT2XModel<T> Lumina-T2X model — transformer-based text-to-any generation (image, video, 3D, audio).
MAGI1Model<T> MAGI-1 video model with strong temporal coherence and multi-task support.
MARModel<T> Masked Autoregressive (MAR) model for image generation via masked token prediction.
MLSDPreprocessor<T> MLSD (Mobile Line Segment Detection) preprocessor for ControlNet conditioning.
MMDiTBlock<T> Joint (double-stream) MMDiT block with separate image and text streams sharing attention but with independent MLPs and AdaLN.
MMDiTNoisePredictor<T> Multi-Modal Diffusion Transformer (MMDiT) noise predictor for SD3 and FLUX architectures.
MMDiTSingleBlock<T> Single-stream MMDiT block (FLUX-style) where text and image tokens are processed together through shared self-attention and parallel MLP.
MMDiTXNoisePredictor<T> Extended Multimodal Diffusion Transformer (MMDiT-X) noise predictor for the Stable Diffusion 3.5 architecture.
MVDreamModel<T> MVDream - Multi-View Diffusion Model for 3D-consistent image generation.
Magic3DModel<T> Magic3D model for high-quality text-to-3D generation using score distillation.
MagicBrushModel<T> MagicBrush model for instruction-based image editing with visual brush stroke guidance.
MakeAVideoModel<T> Make-A-Video model — text-to-video generation without paired text-video data.
MaskBinarizer<T> Converts a soft mask (continuous values) into a binary mask (0 or 1) using a threshold.
MaskBlur<T> Applies Gaussian blur to a mask for smooth transitions.
MaskDilation<T> Dilates a mask by expanding masked regions, filling small holes and connecting nearby regions.
MaskErosion<T> Erodes a mask by shrinking masked regions, removing thin protrusions and small artifacts.
MaskFeatherer<T> Applies feathering (soft blurring) to mask edges for smooth transitions in inpainting.
MaskFromBoundingBox<T> Generates a mask from one or more bounding box regions.
MaskFromPoints<T> Generates a mask from point prompts with circular regions, similar to SAM-style point selection.
MaskFromSegmentation<T> Generates a binary mask from a segmentation map by selecting specific class labels.
MaskInverter<T> Inverts a mask so masked regions become unmasked and vice versa.
MediaPipeFacePreprocessor<T> MediaPipe Face Mesh preprocessor for ControlNet conditioning.
MeissonicModel<T> Meissonic model for non-autoregressive masked image modeling (MIM) generation.
MelSpectrogram<T> Computes Mel spectrograms from audio signals.
MemoryEstimate Memory usage estimate.
MeshyModel<T> Meshy model for production-grade text/image to 3D generation with PBR texturing.
MidJourneyV7Model<T> MidJourney V7-style model architecture for artistic text-to-image generation.
MinimaxVideoModel<T> MiniMax Hailuo video model with strong image-to-video generation.
MoMaskModel<T> MoMask model for masked generative modeling of 3D human motion sequences.
Mochi1Model<T> Mochi 1 model for asymmetric DiT video generation with joint text-video attention.
Mochi1PreviewModel<T> Mochi 1 Preview with Asymmetric Diffusion Transformer (AsymmDiT).
ModelScopeT2VModel<T> ModelScope Text-to-Video model with temporal U-Net for short video clip generation.
ModelShard<T> Enables model sharding across multiple devices for large model inference.
MotionDiffuseModel<T> MotionDiffuse model for fine-grained text-driven motion generation with body part control.
MotionDiffusionModel<T> Motion Diffusion Model (MDM) for text-to-motion generation of human body movements.
MotionModule<T> AnimateDiff motion module for injecting temporal awareness into image diffusion models.
MovieGenModel<T> MovieGen 30B foundation model for video, audio, and editing.
MultiDiffusionModel<T> MultiDiffusion model for generating seamless panoramic and ultra-wide images.
MultiStepConsistencyModel<T> Multi-Step Consistency Model that bridges single-step and multi-step generation.
MultiViewAttention<T> Multi-view attention module for cross-view consistency.
MultiViewUNet<T> Multi-view aware U-Net for MVDream.
MultistepLCModel<T> Multistep Latent Consistency Model (MLCM) for high-quality few-step generation.
NeRFNetwork<T> Neural Radiance Field network for 3D representation.
NeRFResult<T> Result from DreamFusion generation.
NormalMapPreprocessor<T> Normal map estimation preprocessor for ControlNet conditioning.
NullTextInversionModel<T> Null-text Inversion model for editing real images by optimizing unconditional embeddings.
OSDSModel<T> One-Step Diffusion via Shortcut (OSDS) model for single-step high-quality generation.
OasisModel<T> Oasis playable AI game via next-frame prediction.
OffsetNoiseSchedule<T> Offset noise schedule that adds a global offset to noise for improved dark/bright image generation.
OmniGen2Model<T> OmniGen-2 unified model for multi-task image generation and editing with a single architecture.
OmniGenModel<T> OmniGen model — unified image generation model handling multiple tasks in one architecture.
One2345Model<T> One-2-3-45 model for single-image to 3D mesh generation in 45 seconds.
OpenPosePreprocessor<T> OpenPose body keypoint detection preprocessor for ControlNet conditioning.
OpenSora13Model<T> Open-Sora 1.3 with upgraded 3D-VAE and rectified flow.
OpenSora2Model<T> Open-Sora 2.0 commercial-level video generation model.
OpenSoraModel<T> Open-Sora model for open-source Sora-like video generation with STDiT architecture.
OutpaintingMaskGenerator<T> Generates masks for outpainting by marking regions outside the original image bounds.
PABCache<T> Pyramid Attention Broadcast (PAB) cache for accelerating video diffusion inference.
PASDModel<T> PASD: Pixel-Aware Stable Diffusion for real-world image super-resolution.
PCMModel<T> Phased Consistency Model (PCM) for flexible-step generation with phase-based training.
PNDMScheduler<T> PNDM (Pseudo Numerical Methods for Diffusion Models) scheduler implementation.
PaintByExampleModel<T> Paint-by-Example model for exemplar-based inpainting using reference images.
PeRFlowModel<T> PeRFlow (Piecewise Rectified Flow) model for few-step generation via flow straightening.
PeRFlowScheduler<T> PeRFlow (Piecewise Rectified Flow) scheduler for accelerated multi-segment flow sampling.
PerturbedAttentionGuidance<T> Perturbed Attention Guidance (PAG) for diffusion model inference.
PhotoMakerModel<T> PhotoMaker model — identity-customized photo generation with stacked ID embedding.
Pika21Model<T> Pika 2.1 short-form video with creative effects.
Pix2PixZeroModel<T> Pix2Pix-Zero model for zero-shot image-to-image translation without paired training data.
PixArtDeltaLCMModel<T> PixArt-Delta LCM for few-step generation from the efficient PixArt-Delta DiT architecture.
PixArtDeltaModel<T> PixArt-Delta model — LCM-distilled PixArt for fast 2-8 step generation.
PixArtModel<T> PixArt-α model for efficient high-quality text-to-image generation using DiT architecture.
PixArtSigmaModel<T> PixArt-Sigma model for high-resolution text-to-image generation with improved quality.
PlaygroundV25Model<T> Playground v2.5 model for aesthetically-focused text-to-image generation.
PlaygroundV3Model<T> Playground v3 model for aesthetically optimized text-to-image generation.
PointEModel<T> Point-E model for text-to-3D point cloud generation.
PowerPaintModel<T> PowerPaint v2 model for versatile task-aware image inpainting with learnable task prompts.
ProgressiveDistillationTrainer<T> Trainer for progressive distillation that halves the number of steps in each round.
PromptToPromptModel<T> Prompt-to-Prompt model for attention-based image editing by manipulating cross-attention maps.
PyramidFlowModel<T> Pyramid Flow multi-resolution video generation via pyramid flow matching.
QRCodePreprocessor<T> QR code pattern preprocessor for ControlNet conditioning.
Qwen2TextConditioner<T> Qwen2 text encoder conditioning module (Yang et al.
RACEEraser<T> RACE: Robust Adversarial Concept Erasure for removing concepts resilient to red-teaming attacks.
RADModel<T> Region-Aware Diffusion (RAD) model for spatially controlled inpainting and editing.
RAPHAELModel<T> RAPHAEL model — Mixture-of-Experts text-to-image diffusion model.
RBModulationModel<T> RB-Modulation model for training-free style transfer via reference-based attention modulation.
RecraftV3Model<T> Recraft V3 model for professional-grade text-to-image generation.
RectifiedFlowScheduler<T> Rectified flow scheduler for straight-path ODE sampling with velocity prediction.
ReferenceOnlyModel<T> Reference-Only model that uses a reference image’s self-attention features for conditioning.
RefinerStage<T> Refiner stage for late-stage noise-add-then-denoise detail improvement.
ReplaceAnythingModel<T> ReplaceAnything model for text-guided object replacement within SAM-segmented regions.
RescaledCFG<T> Rescaled Classifier-Free Guidance to prevent over-saturation.
RewardGuidance<T> Reward-guided sampling for inference-time alignment of diffusion models.
RewardScoreDistillation<T> Reward-weighted Score Distillation Sampling (RewardSDS) for preference-aligned 3D generation.
RiffusionModel<T> Riffusion model for music generation via spectrogram diffusion.
RunwayGen4Model<T> Runway Gen-4 multi-modal understanding and generation model.
RunwayGenModel<T> Runway Gen model for multi-modal video generation with structure and content disentanglement.
SAMPreprocessor<T> SAM (Segment Anything Model) preprocessor for ControlNet conditioning.
SANAModel<T> SANA model for efficient high-resolution text-to-image generation.
SANASprintModel<T> SANA Sprint model for ultra-fast 1-step generation from the SANA architecture.
SASTDModel<T> SASTD model for structure-aware style transfer via diffusion with edge-guided generation.
SASolverScheduler<T> SA-Solver (Stochastic Adams) scheduler using Adams-Bashforth/Moulton methods for SDE sampling.
SCOREFramework<T> SCORE: Selective Concept Obliteration for Responsible Editing in diffusion models.
SCottModel<T> SCott (Score Consistency via Optimal Transport) for efficient single/few-step generation.
SD3FlashModel<T> SD3 Flash for ultra-fast 1-4 step generation from SD3 architecture.
SD3InpaintingModel<T> Stable Diffusion 3 inpainting model using MMDiT architecture for mask-guided generation.
SD3TurboModel<T> SD3 Turbo for few-step generation from Stable Diffusion 3 via distillation.
SDEditModel<T> SDEdit model for image-guided synthesis via partial noise injection and guided denoising.
SDTurboModel<T> SD Turbo / SDXL Turbo model for real-time single-step image generation.
SDUpscalerModel<T> Stable Diffusion x4 Upscaler model for text-guided latent super-resolution.
SDXLInpaintingModel<T> SDXL Inpainting model for high-resolution 1024x1024 mask-based image inpainting.
SDXLLightningModel<T> SDXL Lightning model for 2-8 step high-quality generation via progressive distillation.
SDXLModel<T> Stable Diffusion XL (SDXL) model for high-resolution image generation.
SDXLRefiner<T> SDXL Refiner model for enhancing generated images.
SDXLTurboModel<T> SDXL Turbo model for real-time single-step high-resolution image generation.
SDXLVAEModel<T> SDXL-optimized VAE with improved decoder fidelity for 1024x1024 generation.
SGRACEEraser<T> S-GRACE: Style-aware GRACE for erasing artistic styles from diffusion models.
STDiTBlock<T> Spatial-Temporal DiT (STDiT) block for efficient video generation transformers.
SUPIRModel<T> SUPIR model for scaling up image restoration with SDXL for photo-realistic results.
ScoreDistillationSampling<T> Score Distillation Sampling (SDS) for text-to-3D and generator optimization.
ScoreDistillationTrainer<T> Trainer for Score Distillation Sampling (SDS) and its variants for generator training.
ScribblePreprocessor<T> Scribble/sketch preprocessor for ControlNet conditioning.
SeamlessBlender<T> Seamless blending for panoramic and tiled diffusion generation with overlap regions.
SeeSRModel<T> SeeSR: Semantics-aware super-resolution using diffusion-based image upscaling.
SeedEdit3Model<T> SeedEdit 3 model for high-fidelity instruction-based editing with structure preservation.
Seedance1Model<T> Seedance 1 ranked #1 on T2V and I2V leaderboards.
SelfAttentionGuidance<T> Self-Attention Guidance (SAG) for diffusion model inference.
SemanticScoreDistillation<T> Semantic Score Distillation (SemanticSDS) for semantically-guided 3D generation.
SemanticSegPreprocessor<T> Semantic segmentation preprocessor for ControlNet conditioning.
SenseFlowModel<T> SenseFlow model for accelerated flow-matching generation via knowledge distillation.
ShapEModel<T> Shap-E model for text-to-3D and image-to-3D generation with implicit neural representations.
ShortTimeFourierTransform<T> Short-Time Fourier Transform (STFT) for analyzing audio signals over time.
Show1Model<T> Show-1 marrying pixel and latent diffusion for text-to-video.
ShufflePreprocessor<T> Shuffle preprocessor for ControlNet conditioning.
SiDDiTModel<T> SiD-DiT: Score Identity Distillation applied to Diffusion Transformer (DiT) architectures.
SiDModel<T> Score Identity Distillation (SiD) model for single-step generation via score identity.
SiTPredictor<T> Scalable Interpolant Transformer (SiT) noise predictor for flow-based diffusion.
SigLIP2TextConditioner<T> SigLIP 2 text encoder conditioning module (Tschannen et al.
SigLIPTextConditioner<T> SigLIP text encoder conditioning module (Zhai et al., ICCV 2023).
SkyReelsV1Model<T> SkyReels V1 human-centric video generation model.
SnapVideoModel<T> Snap Video scaled spatiotemporal transformer for text-to-video.
SoftEdgePreprocessor<T> Soft edge detection preprocessor for ControlNet conditioning.
Sora2Model<T> Sora 2 cinematic video generation with physics simulation and synced audio.
SoraModel<T> Sora-architecture model for DiT-based video generation with native spatiotemporal patches.
SparseVideoGen<T> Sparse video generation with selective frame denoising for faster inference.
SpotDiffusionModel<T> SpotDiffusion model for spatially-organized text-guided panorama generation.
StableAudioModel<T> Stable Audio Open model — DiT-based latent diffusion for long-form audio generation.
StableCascadeModel<T> Stable Cascade (Würstchen v3) model for high-resolution text-to-image generation.
StableDiffusion15Model<T> Stable Diffusion 1.5 model for text-to-image generation.
StableDiffusion2Model<T> Stable Diffusion 2.0/2.1 model for text-to-image generation.
StableDiffusion35Model<T> Stable Diffusion 3.5 model with improved MMDiT-X architecture by Stability AI.
StableDiffusion3Model<T> Stable Diffusion 3 / SD 3.5 model for text-to-image generation using rectified flow and MMDiT.
StableSRModel<T> StableSR model for exploiting diffusion prior for real-world image super-resolution.
StableVITONModel<T> StableVITON model for learning semantic correspondence with Stable Diffusion for virtual try-on.
StableVideoDiffusion<T> Stable Video Diffusion (SVD) model for image-to-video generation.
StandardVAE<T> Standard Variational Autoencoder for latent diffusion models.
Step1XEditModel<T> Step1X-Edit model for one-step image editing using consistency distillation.
StepVideoModel<T> StepVideo text-to-video model with benchmark-leading quality.
StitchDiffusionModel<T> StitchDiffusion model for seamless 360-degree panorama generation with wrap-around consistency.
StochasticInterpolantScheduler<T> Stochastic Interpolant scheduler for generalized flow-based sampling with noise injection.
StreamingT2VModel<T> StreamingT2V autoregressive long video generation up to 1200 frames.
StrengthBasedScheduling<T> Strength-based scheduling for img2img and inpainting denoising control.
StudentTeacherFramework<T> Generic student-teacher framework for knowledge distillation in diffusion models.
StyDiffModel<T> StyDiff model for diffusion-based artistic style transfer with content preservation.
StyleAlignedEditModel<T> StyleAligned-Edit model for consistent multi-image style editing via shared self-attention.
StyleAlignedModel<T> Style-Aligned model for consistent style across multiple generated images.
StyleStudioModel<T> StyleStudio model for text-driven style generation and transfer with disentangled control.
SwiftBrushModel<T> SwiftBrush model for image-free one-step text-to-image distillation.
SyncDiffusionModel<T> SyncDiffusion model for coherent panorama generation with synchronized denoising.
SyncDreamerModel<T> SyncDreamer model for synchronized multi-view diffusion with 3D-consistent generation.
T2IAdapterModel<T> T2I-Adapter model for adding spatial control to text-to-image diffusion models.
T5TextConditioner<T> T5 text encoder conditioning module (Raffel et al., JMLR 2020).
TCDModel<T> Trajectory Consistency Distillation (TCD) model for high-quality few-step generation.
TLoRAAttentionAdapter<T> Wraps a UNet attention block and adds T-LoRA’s timestep-masked low-rank update to its output.
TLoRAModel<T> T-LoRA: timestep-dependent low-rank adaptation for customizing a diffusion model from a SINGLE image without overfitting to it.
TSDSRModel<T> TSD-SR: Timestep-Shifted Diffusion for fast and high-quality super-resolution.
TeaCache<T> Timestep Embedding Aware Cache (TeaCache) for accelerating video diffusion inference.
TemporalConvolution<T> 1D temporal convolution layer for video diffusion models.
TemporalInterpolationVAE<T> Stable Video Diffusion’s VAE: the latent-diffusion KL encoder applied to each frame, and a decoder that mixes information across frames so the decoded clip is temporally consistent.
TemporalSelfAttention<T> Temporal self-attention layer for video diffusion models.
TemporalVAE<T> Temporal-aware Variational Autoencoder for video diffusion models.
TilePreprocessor<T> Tile preprocessor for ControlNet conditioning.
TimestepDependentLora<T> Timestep-dependent low-rank adaptation: the mechanism of T-LoRA, in which the effective rank of the adapter update SHRINKS as the diffusion timestep rises.
TokenFlowModel<T> TokenFlow consistent video editing via token flow propagation.
TrainingEfficientLCM<T> Training-Efficient Latent Consistency Model for resource-constrained LCM distillation.
TrajectoryConsistencyDistiller<T> Trainer for Trajectory Consistency Distillation (TCD) with trajectory-aware loss.
TripleTextConditioner<T> Triple text encoder conditioning module combining two CLIP encoders and a T5 encoder.
TripoSRModel<T> TripoSR model for ultra-fast feed-forward single-image 3D reconstruction using LRM transformer.
TurboEditModel<T> TurboEdit model for fast few-step image editing using distilled SDXL Turbo.
TurboFillModel<T> TurboFill model for fast few-step inpainting using adversarial distillation on SDXL.
UNetBlock<T> Structure for U-Net blocks containing residual, attention, and sampling layers.
UNetNoisePredictor<T> U-Net architecture for noise prediction in diffusion models.
UViTBlock<T> Block structure for U-ViT transformer layers.
UViTNoisePredictor<T> U-shaped Vision Transformer (U-ViT) noise predictor for diffusion models.
UdioModel<T> Udio/Suno architecture model for full-song music generation with structural awareness.
UltraEditModel<T> UltraEdit model for fine-grained instruction-based image editing with region awareness.
UniControlNetModel<T> Uni-ControlNet model for simultaneous multi-condition control with condition-specific adapters.
UniPCScheduler<T> UniPC (Unified Predictor-Corrector) scheduler for fast, high-quality diffusion sampling.
UniSimModel<T> UniSim universal simulator from video and action pairs.
UniVSTAdaInStylization<T> UniVST’s training-free AdaIN-guided localized stylization (Song et al., arXiv:2410.20084, TPAMI 2025), operating at BOTH the latent level and the attention level.
UniVSTConsistentSmoothing<T> UniVST’s sliding-window consistent smoothing (Song et al., arXiv:2410.20084, TPAMI 2025): flow-warped temporal averaging in PIXEL space, fed back as refined noise.
UniVSTMaskPropagation<T> UniVST’s point-matching mask propagation (Song et al., arXiv:2410.20084, TPAMI 2025).
UniVSTModel<T> UniVST: training-free LOCALIZED video style transfer over a frozen latent diffusion model (Song, Lin, Zhan, Yan, Cao and Ji, arXiv:2410.20084, TPAMI 2025).
UniVSTQkvTransform<T> Attaches UniVST’s attention-level steering to a frozen attention block: query blending at every timestep, and key/value AdaIN on the beta ramp (Song et al., arXiv:2410.20084, TPAMI 2025).
UnifiedDistillationSampling<T> Unified Distillation Sampling (UDS) framework unifying SDS, VSD, CSD, and ISM variants.
UpBlock<T> Upsampling block for VAE decoder with transposed convolution and multiple ResBlocks.
UpscaleAVideoModel<T> Upscale-A-Video model for temporally consistent video super-resolution with diffusion.
VAEDecoder<T> Convolutional decoder for VAE that reconstructs images from latent space.
VAEEncoder<T> Convolutional encoder for VAE that compresses images to latent space.
VAEResBlock<T> Residual block for VAE encoder/decoder with GroupNorm and skip connections.
VariationalRFScheduler<T> Variational Rectified Flow scheduler with learned time-dependent noise injection.
VariationalScoreDistillation<T> Variational Score Distillation (VSD) for high-fidelity text-to-3D generation.
Veo3Model<T> Veo 3 with native audio generation and dialogue synchronization.
VeoModel<T> Veo model for Google’s high-fidelity cascaded video generation with temporal super-resolution.
VideoCrafter2Model<T> VideoCrafter 2 with improved quality and style fusion.
VideoCrafterModel<T> VideoCrafter model for high-quality text-to-video and image-to-video generation.
VideoP2PModel<T> VideoP2P prompt-to-prompt cross-attention control for video editing.
VideoPoetModel<T> VideoPoet LLM-based zero-shot video generation.
VideoUNetPredictor<T> 3D U-Net architecture for video noise prediction in diffusion models.
Wan21Model<T> Wan 2.1 video model with MoE denoising and full 3D attention.
Wan22Model<T> Wan 2.2 video model with timestep-specialized MoE experts.
WanVideoModel<T> Wan video model for Alibaba’s scalable DiT video generation with full 3D attention.
Wonder3DModel<T> Wonder3D model for multi-view cross-domain diffusion with simultaneous RGB and normal map generation.
Zero123Model<T> Zero-1-to-3 model for novel view synthesis from a single image.
ZeroTerminalSNRSchedule<T> Zero Terminal SNR noise schedule ensuring signal-to-noise ratio reaches exactly zero at the final timestep.

Layers (3)

Type Summary
DiffusionAttentionLayer<T> Lazy multi-head attention with distinct query and context dimensions.
TemporalModule3DLayer<T> Released Upscale-A-Video temporal module: temporal 3D ResNet, inflated spatial ResNet, and a zero-initialized residual projection.
VideoTransformer3DLayer<T> The released Upscale-A-Video Transformer3D block.

Base Classes (11)

Type Summary
AudioDiffusionModelBase<T> Base class for audio diffusion models that generate sound and music.
CompositeConditioningBase<T> Base class for conditioning modules that compose other conditioners rather than owning their own learnable weights.
DiffusionModelBase<T> Base class for diffusion-based generative models providing common functionality.
DiffusionPreprocessorBase<T> Base class for diffusion model condition preprocessors that convert input images into control signals (edge maps, depth maps, pose skeletons, etc.).
LatentDiffusionModelBase<T> Base class for latent diffusion models that operate in a compressed latent space.
NoisePredictorBase<T> Base class for noise prediction networks used in diffusion models.
NoiseSchedulerBase<T> Base class for diffusion model noise schedulers providing common functionality.
TextConditioningBase<T> Base class for text conditioning modules used in diffusion models.
ThreeDDiffusionModelBase<T> Base class for 3D diffusion models that generate 3D content like point clouds, meshes, and scenes.
VAEModelBase<T> Base class for Variational Autoencoder (VAE) models used in latent diffusion.
VideoDiffusionModelBase<T> Base class for video diffusion models that generate temporal sequences.

Interfaces (3)

Type Summary
IContextualLayer<T> Interface for layers that accept context (conditioning).
IGuidanceMethod<T> Interface for guidance methods that modify noise predictions during diffusion sampling.
IPoseExtractor<T> Pluggable keypoint extractor interface.

Enums (8)

Type Summary
AudioLDM2Variant AudioLDM 2 model variant.
BlendProfile Specifies the blending profile for overlap region transitions.
BlendingMode Blending modes for overlapping window regions.
ControlNetConditionType Specifies the type of conditioning input for multi-control ControlNet composition.
ControlType Types of control signals supported by ControlNet.
PaddingMode Padding mode for STFT centering.
ShardingStrategy Strategy for distributing layers across devices.
SparsityStrategy Strategy for selecting which transformer blocks to skip in sparse computation.

Structs (1)

Type Summary
DreamVector3<T> 3D vector type for DreamFusion.

Options & Configuration (18)

Type Summary
CLIPTextConditionerOptions Configuration options for the CLIPTextConditioner text conditioner.
ChatGLM3TextConditionerOptions Configuration options for the ChatGLM3TextConditioner text conditioner.
DiffusionMemoryConfig Configuration for diffusion model memory management.
DistilledT5TextConditionerOptions Configuration options for the DistilledT5TextConditioner text conditioner.
DreamFusionConfig Configuration for DreamFusion model.
GemmaTextConditionerOptions Configuration options for the GemmaTextConditioner text conditioner.
MVDreamConfig Configuration for MVDream model.
MotionModuleConfig Configuration for AnimateDiff motion modules.
PixArtOptions<T> Options for PixArt-α model configuration.
Qwen2TextConditionerOptions Configuration options for the Qwen2TextConditioner text conditioner.
SchedulerConfig<T> Configuration options for diffusion model step schedulers.
ShardingConfig Configuration for model sharding.
SigLIP2TextConditionerOptions Configuration options for the SigLIP2TextConditioner text conditioner.
SigLIPTextConditionerOptions Configuration options for the SigLIPTextConditioner text conditioner.
SpectrogramConfig Configuration for spectrogram generation.
T5TextConditionerOptions Configuration options for the T5TextConditioner text conditioner.
TextConditionerOptions Transformer dimensions shared by every text conditioner’s options, overriding the ones its variant selects.
UniVSTOptions Configuration for UniVSTModel.

Helpers & Utilities (3)

Type Summary
AiModelResultDiffusionExtensions Model-specific inference extensions on AiModelResult for the diffusion family (DDPM, LDM, Stable Diffusion, DiT, etc.).
ModelSizes<T> Standard DiT model sizes.
PointCounts<T> Standard Point-E point counts.