SUPIRModel<T>

Models & Types · AiDotNet.Diffusion.SuperResolution

SUPIR model for scaling up image restoration with SDXL for photo-realistic results.

Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → SUPIRModel<T>

For Beginners

SUPIR uses SDXL (the most capable open-source image model) combined with GPT language understanding for the highest quality image restoration.

How SUPIR works:

  1. A GPT model analyzes the degraded image and generates a quality description
  2. The description guides SDXL’s diffusion process for intelligent restoration
  3. SDXL’s 2048-dim dual text encoder provides rich semantic conditioning
  4. The result is photo-realistic detail at 1024x1024 native resolution

Key characteristics:

  • Based on SDXL (1024x1024 native resolution, much higher than SD 1.5’s 512x512)
  • GPT-guided semantic understanding of image content and degradation
  • Dual text encoders (2048-dim combined) for precise conditioning
  • DPM-Solver++ for efficient 50-step inference
  • SDXL VAE scale factor (0.13025, different from SD 1.5’s 0.18215)

When to use SUPIR:

  • Highest possible quality restoration is needed
  • Processing at 1024x1024 resolution
  • Complex degradation where semantic understanding helps
  • Photo-realistic detail generation for important images

Limitations:

  • Requires significantly more VRAM than SD 1.5-based models (~12GB+)
  • Slower than lighter SR methods due to SDXL backbone
  • GPT component adds complexity and potential dependency

How It Works

SUPIR (Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild) leverages the SDXL model’s generative prior with a GPT-guided restoration pipeline for photo-realistic super-resolution at high resolutions. It combines semantic understanding from a large language model with SDXL’s generation quality.

Architecture components:

  • SDXL U-Net backbone (320 base channels, [1, 2, 4] multipliers)
  • Dual text encoders: OpenCLIP ViT-bigG/14 + CLIP ViT-L/14 (2048-dim combined)
  • GPT-guided quality description for semantic restoration understanding
  • SDXL VAE (4-channel latent, 0.13025 scale factor for SDXL)
  • DPM-Solver++ multistep scheduler for efficient inference

Technical specifications:

  • Architecture: SDXL U-Net + GPT-guided conditioning
  • U-Net: 320 base channels, multipliers [1, 2, 4] (SDXL config)
  • Cross-attention dimension: 2048 (dual OpenCLIP + CLIP)
  • VAE: 4 latent channels, SDXL scale factor 0.13025
  • Native resolution: 1024x1024 pixels
  • Noise schedule: Scaled linear beta [0.00085, 0.012], 1000 timesteps
  • Default scheduler: DPM-Solver++ Multistep
  • Default guidance scale: 7.5

Reference: Yu et al., “Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild”, CVPR 2024

Example

var degradedImage = Tensor<float>.CreateRandom(1, 3, 32, 32);
// Create with industry-standard defaults (1024x1024 resolution)
var supir = new SUPIRModel<float>();
            
// High-quality 1024x1024 restoration
var restored = supir.ImageToImage(
    inputImage: degradedImage,
    prompt: "a high quality, detailed photograph with sharp focus",
    negativePrompt: "blurry, noisy, low quality, artifacts",
    strength: 0.5,
    numInferenceSteps: 50,
    guidanceScale: 7.5);
            
// Generate at SDXL native resolution
var result = supir.GenerateFromText(
    prompt: "a stunning photograph, masterpiece quality",
    width: 1024,
    height: 1024,
    numInferenceSteps: 50);

Constructors

Constructor Summary
SUPIRModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> unet, StandardVAE<T> vae, IConditioningModule<T> conditioner, int? seed) Initializes a new instance of SUPIRModel with full customization support.

Properties

Property Summary
IConditioningModule<T> Conditioner { get; }
INoisePredictor<T> NoisePredictor { get; }
IVAEModel<T> VAE { get; }
int LatentChannels { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()
Tensor<T> GenerateFromText(string prompt, string negativePrompt, int width, int height, int numInferenceSteps, double? guidanceScale, int? seed) Generates an image from a text prompt using SUPIR/SDXL defaults.
Tensor<T> ImageToImage(Tensor<T> inputImage, string prompt, string negativePrompt, double strength, int numInferenceSteps, double? guidanceScale, int? seed) Performs SDXL-quality image restoration using SUPIR.

Fields

Field Summary
int DefaultHeight Default image height for SUPIR output (1024, SDXL native).
int DefaultWidth Default image width for SUPIR output (1024, SDXL native).