UniVSTConsistentSmoothing<T>

Models & Types · AiDotNet.Diffusion.StyleTransfer

UniVST’s sliding-window consistent smoothing (Song et al., arXiv:2410.20084, TPAMI 2025): flow-warped temporal averaging in PIXEL space, fed back as refined noise.

For Beginners

Independently stylized frames flicker. This lines neighbouring frames up using optical flow, averages them, and feeds the steadier result back into the generator.

How It Works

The distinguishing choice is that smoothing happens on decoded PIXELS and is then folded back into the latent trajectory as a corrected noise estimate, rather than being applied to latents directly. Averaging latents would blur structure, because neighbouring latents are not aligned; averaging flow-warped pixels aligns them first.

The averaged frames are re-encoded and used to re-derive epsilon, so the correction re-enters the diffusion process through the noise rather than overwriting the latent. That keeps the sampler’s own update rule intact.

Constructors

Constructor Summary
UniVSTConsistentSmoothing(UniVSTOptions options) Creates the smoother.

Properties

Property Summary
int HalfWindow { get; } Gets m, the window half-width; the full window spans 2m + 1 frames.
int WindowLength { get; } Gets the full window length, 2m + 1.

Methods

Method Summary
IReadOnlyList<Tensor<T>> SmoothSequence(IReadOnlyList<Tensor<T>> frames, Func<int, int, Tensor<T>> flowTo) Smooths a decoded frame sequence: each frame becomes the mean of its window’s neighbours, warped into its own frame of reference.
IReadOnlyList<int> WindowIndices(int index, int frameCount) Window indices for frame index, clamped to the sequence.
Tensor<T> RefineNoise(Tensor<T> latentT, Tensor<T> smoothedZeroPrediction, double alphaCumulativeT) Re-derives epsilon from a smoothed x0 prediction: epsbar = (z_t - sqrt(alpha_t) * z0bar) / sqrt(1 - alpha_t).
Tensor<T> StepWithRefinedNoise(Tensor<T> smoothedZeroPrediction, Tensor<T> refinedNoise, double alphaCumulativePrevious) The DDIM step rebuilt from the smoothed x0 and refined noise: z_{t-1} = sqrt(alpha_{t-1}) * z0bar + sqrt(1 - alpha_{t-1}) * epsbar.
bool IsActive(double timestepFraction) True when smoothing applies at the given progress through the schedule (fraction of T).