SyncDiffusionModel<T>

Models & Types · AiDotNet.Diffusion.Panorama

SyncDiffusion model for coherent panorama generation with synchronized denoising.

Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → SyncDiffusionModel<T>

For Beginners

SyncDiffusion creates panoramas that look more consistent than MultiDiffusion. While MultiDiffusion just averages overlapping patches, SyncDiffusion adds an extra step to make sure all patches agree on the overall style and color, preventing abrupt style changes across the panorama.

How It Works

SyncDiffusion improves on MultiDiffusion by adding a gradient descent synchronization step that ensures global style consistency across patches. After each denoising step, patches are updated to minimize perceptual difference with a shared anchor view.

Reference: Lee et al., “SyncDiffusion: Coherent Montage via Synchronized Joint Diffusions”, NeurIPS 2023

Example

var options = new DiffusionModelOptions<float> { LatentChannels = 4, DefaultInferenceSteps = 30 };
var noise = Tensor<float>.CreateRandom(new[] { 1, 4, 64, 256 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
    .ConfigureModel(new SyncDiffusionModel<float>(options: options))
    .Build(trainX, trainY);
var panorama = result.Predict(noise);

Constructors

Constructor Summary
SyncDiffusionModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> predictor, StandardVAE<T> vae, IConditioningModule<T> conditioner, int? seed)

Properties

Property Summary
IConditioningModule<T> Conditioner { get; }
INoisePredictor<T> NoisePredictor { get; }
IVAEModel<T> VAE { get; }
int LatentChannels { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()