ControlNetUnionProModel<T>

Models & Types · AiDotNet.Diffusion.Control

ControlNet Union Pro model that supports multiple control types in a single model.

Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → ControlNetUnionProModel<T>

For Beginners

Instead of needing a different model file for edges, depth, poses, etc., this single model handles all control types. You just tell it which type of control image you’re providing, and it adapts automatically.

How It Works

ControlNet Union Pro consolidates multiple control types into a single unified model, eliminating the need to load separate ControlNet checkpoints for each control type. Supports switching between and combining control modes at inference time.

Example

// Create a unified ControlNet that handles multiple control types
var options = new DiffusionModelOptions<float>
{
    LatentChannels = 4,
    DefaultInferenceSteps = 25
};
            
// Generate with any control type using the same model
var controlInput = Tensor<float>.CreateRandom(new[] { 1, 1, 512, 512 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
    .ConfigureModel(new ControlNetUnionProModel<float>(options: options))
    .Build(trainX, trainY);
var prediction = result.Predict(controlInput);

Constructors

Constructor Summary
ControlNetUnionProModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> baseUNet, StandardVAE<T> vae, IConditioningModule<T> conditioner, ControlType[] supportedTypes, int? seed) Initializes a new ControlNet Union Pro model.

Properties

Property Summary
IConditioningModule<T> Conditioner { get; }
INoisePredictor<T> NoisePredictor { get; }
IVAEModel<T> VAE { get; }
int LatentChannels { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()