UniVSTConsistentSmoothing<T>
Models & Types · AiDotNet.Diffusion.StyleTransfer
UniVST’s sliding-window consistent smoothing (Song et al., arXiv:2410.20084, TPAMI 2025): flow-warped temporal averaging in PIXEL space, fed back as refined noise.
For Beginners
Independently stylized frames flicker. This lines neighbouring frames up using optical flow, averages them, and feeds the steadier result back into the generator.
How It Works
The distinguishing choice is that smoothing happens on decoded PIXELS and is then folded back into the latent trajectory as a corrected noise estimate, rather than being applied to latents directly. Averaging latents would blur structure, because neighbouring latents are not aligned; averaging flow-warped pixels aligns them first.
The averaged frames are re-encoded and used to re-derive epsilon, so the correction re-enters the diffusion process through the noise rather than overwriting the latent. That keeps the sampler’s own update rule intact.
Constructors
| Constructor | Summary |
|---|---|
UniVSTConsistentSmoothing(UniVSTOptions options) |
Creates the smoother. |
Properties
| Property | Summary |
|---|---|
int HalfWindow { get; } |
Gets m, the window half-width; the full window spans 2m + 1 frames. |
int WindowLength { get; } |
Gets the full window length, 2m + 1. |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<Tensor<T>> SmoothSequence(IReadOnlyList<Tensor<T>> frames, Func<int, int, Tensor<T>> flowTo) |
Smooths a decoded frame sequence: each frame becomes the mean of its window’s neighbours, warped into its own frame of reference. |
IReadOnlyList<int> WindowIndices(int index, int frameCount) |
Window indices for frame index, clamped to the sequence. |
Tensor<T> RefineNoise(Tensor<T> latentT, Tensor<T> smoothedZeroPrediction, double alphaCumulativeT) |
Re-derives epsilon from a smoothed x0 prediction: epsbar = (z_t - sqrt(alpha_t) * z0bar) / sqrt(1 - alpha_t). |
Tensor<T> StepWithRefinedNoise(Tensor<T> smoothedZeroPrediction, Tensor<T> refinedNoise, double alphaCumulativePrevious) |
The DDIM step rebuilt from the smoothed x0 and refined noise: z_{t-1} = sqrt(alpha_{t-1}) * z0bar + sqrt(1 - alpha_{t-1}) * epsbar. |
bool IsActive(double timestepFraction) |
True when smoothing applies at the given progress through the schedule (fraction of T). |