MultiDiffusionModel<T>

Models & Types · AiDotNet.Diffusion.Panorama

MultiDiffusion model for generating seamless panoramic and ultra-wide images.

Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → MultiDiffusionModel<T>

For Beginners

MultiDiffusion creates panoramic images wider than what the model normally generates. It works by generating overlapping patches and blending them together seamlessly, like taking multiple photos and stitching them into a panorama.

How It Works

MultiDiffusion generates arbitrarily wide/tall images by running overlapping diffusion passes and averaging the denoised results in the overlap regions. This produces seamless panoramas without visible seam artifacts.

Reference: Bar-Tal et al., “MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation”, ICML 2023

Example

var options = new DiffusionModelOptions<float> { LatentChannels = 4, DefaultInferenceSteps = 30 };
var noise = Tensor<float>.CreateRandom(new[] { 1, 4, 64, 256 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
    .ConfigureModel(new MultiDiffusionModel<float>(options: options))
    .Build(trainX, trainY);
var panorama = result.Predict(noise);

Constructors

Constructor Summary
MultiDiffusionModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> predictor, StandardVAE<T> vae, IConditioningModule<T> conditioner, int? seed)

Properties

Property Summary
IConditioningModule<T> Conditioner { get; }
INoisePredictor<T> NoisePredictor { get; }
IVAEModel<T> VAE { get; }
int LatentChannels { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()