DiffBIRModel<T>
Models & Types · AiDotNet.Diffusion.SuperResolution
DiffBIR model for blind image restoration with generative diffusion prior.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → DiffBIRModel<T>
For Beginners
DiffBIR restores degraded images (blur, noise, JPEG artifacts, old photos).
How DiffBIR works:
- Stage 1 (SwinIR): A regression network removes obvious degradation from the input
- Stage 2 (Diffusion): The SD prior adds realistic details to the cleaned image
- LAControlNet: Controls how much the diffusion stage can modify the Stage 1 output
Key characteristics:
- Handles blind (unknown) degradation types automatically
- Two-stage design separates degradation removal from detail generation
- Controllable balance between fidelity to input and generated quality
- Works for face restoration, general images, and old photo restoration
- Based on Stable Diffusion 1.5 backbone for high-quality detail generation
When to use DiffBIR:
- Restoring old or damaged photographs
- Removing JPEG compression artifacts
- Denoising and deblurring real-world images
- Face restoration in group photos or surveillance footage
Limitations:
- Two-stage pipeline is slower than single-stage methods
- May hallucinate details not present in the original image
- Best results at 512x512 (SD 1.5 native resolution)
How It Works
DiffBIR (Diffusion-Based Blind Image Restoration) uses a two-stage pipeline for real-world image restoration. The first stage removes degradation using a regression module (SwinIR), and the second stage refines details using a Stable Diffusion prior with a controllable module for balancing fidelity and quality.
Architecture components:
- Stage 1: SwinIR regression module for degradation removal
- Stage 2: SD 1.5 U-Net backbone (320 base channels, [1,2,4,4] multipliers)
- Controllable feature wrapping module (LAControlNet) for fidelity-quality balance
- Standard VAE (4-channel latent space, 0.18215 scale factor)
- CLIP ViT-L/14 text encoder (768-dim cross-attention)
Technical specifications:
- Architecture: Two-stage (SwinIR + SD 1.5 U-Net with LAControlNet)
- Stage 1: SwinIR with 6 RSTB blocks, 180 channels, window size 8
- Stage 2: SD 1.5 U-Net backbone, 320 base channels, [1,2,4,4] multipliers
- Cross-attention dimension: 768 (CLIP ViT-L/14)
- VAE: 4 latent channels, scale factor 0.18215, 8x downsampling
- Noise schedule: Scaled linear beta [0.00085, 0.012], 1000 timesteps
- Default guidance scale: 7.5
- Default resolution: 512x512
Reference: Lin et al., “DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior”, ECCV 2024
Example
var degradedPhoto = Tensor<float>.CreateRandom(1, 3, 32, 32);
// Create with industry-standard defaults
var diffbir = new DiffBIRModel<float>();
// Restore a degraded image using image-to-image pipeline
var restored = diffbir.ImageToImage(
inputImage: degradedPhoto,
prompt: "high quality, sharp, detailed",
negativePrompt: "blurry, noisy, artifacts",
strength: 0.5,
numInferenceSteps: 50,
guidanceScale: 7.5);
// Generate from text (for unconditional restoration with prompt guidance)
var result = diffbir.GenerateFromText(
prompt: "a sharp, detailed photograph",
width: 512,
height: 512,
numInferenceSteps: 50);
Constructors
| Constructor | Summary |
|---|---|
DiffBIRModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> unet, StandardVAE<T> vae, IConditioningModule<T> conditioner, int? seed) |
Initializes a new instance of DiffBIRModel with full customization support. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> VAE { get; } |
|
int CrossAttentionDim { get; } |
Gets the cross-attention dimension (768, matching CLIP ViT-L/14). |
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> GenerateFromText(string prompt, string negativePrompt, int width, int height, int numInferenceSteps, double? guidanceScale, int? seed) |
Generates an image from a text prompt using DiffBIR defaults. |
Tensor<T> ImageToImage(Tensor<T> inputImage, string prompt, string negativePrompt, double strength, int numInferenceSteps, double? guidanceScale, int? seed) |
Performs image restoration using the DiffBIR diffusion pipeline. |
Fields
| Field | Summary |
|---|---|
int DefaultHeight |
Default image height for DiffBIR restoration. |
int DefaultWidth |
Default image width for DiffBIR restoration. |