SyncDiffusionModel<T>
Models & Types · AiDotNet.Diffusion.Panorama
SyncDiffusion model for coherent panorama generation with synchronized denoising.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → SyncDiffusionModel<T>
For Beginners
SyncDiffusion creates panoramas that look more consistent than MultiDiffusion. While MultiDiffusion just averages overlapping patches, SyncDiffusion adds an extra step to make sure all patches agree on the overall style and color, preventing abrupt style changes across the panorama.
How It Works
SyncDiffusion improves on MultiDiffusion by adding a gradient descent synchronization step that ensures global style consistency across patches. After each denoising step, patches are updated to minimize perceptual difference with a shared anchor view.
Reference: Lee et al., “SyncDiffusion: Coherent Montage via Synchronized Joint Diffusions”, NeurIPS 2023
Example
var options = new DiffusionModelOptions<float> { LatentChannels = 4, DefaultInferenceSteps = 30 };
var noise = Tensor<float>.CreateRandom(new[] { 1, 4, 64, 256 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
.ConfigureModel(new SyncDiffusionModel<float>(options: options))
.Build(trainX, trainY);
var panorama = result.Predict(noise);
Constructors
| Constructor | Summary |
|---|---|
SyncDiffusionModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> predictor, StandardVAE<T> vae, IConditioningModule<T> conditioner, int? seed) |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> VAE { get; } |
|
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |