StreamDiffVSROptions

Options & Configuration · AiDotNet.Video.Options

Options for StreamDiffVSR: the authors’ released Stream-DiffVSR (jamichss/Stream-DiffVSR, Jamichsu/Stream-DiffVSR on the Hugging Face Hub).

Inheritance: ModelOptions → NeuralNetworkOptions → StreamDiffVSROptions

For Beginners

the defaults build the released model. Shrink the widths for experiments on modest hardware; every value stays settable.

How It Works

The defaults are the release’s configuration: the four-step distilled UNet in the Stable Diffusion x4 upscaler’s layout (widths 256, 512, 512, 1024, eight heads, 1024-wide text, only_cross_attention at the first three levels), the Auto-regressive Temporal Guidance ControlNet copied from it (conditioning widths 64, 128, 256), the temporal tiny autoencoder (TAESD-x4 with the Temporal Processor Module in its decoder), DDIM with v-prediction on the scaled-linear schedule at trailing timesteps, RAFT-large flows with twelve updates, and the ARTG stage’s training recipe (train/train_artg.sh).

Constructors

Constructor Summary
StreamDiffVSROptions() Creates the released configuration.
StreamDiffVSROptions(StreamDiffVSROptions other) Copies other.

Properties

Property Summary
OnnxModelOptions OnnxOptions { get; set; } The ONNX runtime options.
bool[] OnlyCrossAttention { get; set; } Per level, whether both attentions read the text (released: true, true, true, false).
double AdamBeta1 { get; set; } AdamW’s β1 (0.9).
double AdamBeta2 { get; set; } AdamW’s β2 (0.999).
double AdamEpsilon { get; set; } AdamW’s ε (1e-8).
double BetaEnd { get; set; } β at the last timestep (0.02).
double BetaStart { get; set; } β at the first timestep of the scaled-linear schedule (1e-4).
double ConditioningScale { get; set; } The scale on the ARTG residuals (released: 1).
double GanWeight { get; set; } The adversarial weight (0.025); zero leaves the discriminator out.
double LatentMseWeight { get; set; } The weight of the latent v-prediction MSE (1).
double LearningRate { get; set; } AdamW’s learning rate (released ARTG stage: 5e-5).
double LpipsWeight { get; set; } The weight of LPIPS on the decoded frame (0.1); zero leaves AlexNet out.
double MaxGradientNorm { get; set; } The gradient-norm clip (1); zero disables it.
double WeightDecay { get; set; } AdamW’s weight decay (1e-2).
int AttentionHeads { get; set; } The attention heads of every transformer (released attention_head_dim: 8).
int CrossAttentionDim { get; set; } The text embedding’s width (released: 1024, OpenCLIP ViT-H).
int DiscriminatorInterval { get; set; } The discriminator trains, and the adversarial term applies, on every this many steps (5).
int FlowIterations { get; set; } RAFT’s flow updates (torchvision’s raft_large: 12).
int GanStartIteration { get; set; } The step from which the adversarial term counts (the paper’s 20k; the release’s ARTG script passes 2000).
int LayersPerBlock { get; set; } The residual blocks per encoder level; the decoder has one more (released: 2).
int NumTrainTimesteps { get; set; } The training schedule’s timesteps (1000).
int PromptLength { get; set; } The empty prompt’s tokens (77).
int SamplingSteps { get; set; } The DDIM steps at trailing timesteps (released: 4).
int ScaleFactor { get; } The upscaling factor: the tiny autoencoder’s compression, 2^(levels - 1) of EncoderBlocks (released: 4).
int VaeChannels { get; set; } The tiny autoencoder’s width (released: 64).
int WarmupSteps { get; set; } The linear warmup before the constant rate (constant_with_warmup, 1000 steps).
int[] BlockChannels { get; set; } The UNet’s level widths (released: 256, 512, 512, 1024); each splits into 32 groups.
int[] ConditioningChannels { get; set; } The ARTG condition embedding’s widths (released: 64, 128, 256).
int[] DecoderBlocks { get; set; } The decoder’s temporal blocks per level (released: 3, 3, 1).
int[] EncoderBlocks { get; set; } The encoder’s blocks per level (released: 1, 3, 3).
string ModelPath { get; set; } The ONNX model’s path in inference mode.