VITS2Options

Options & Configuration · AiDotNet.TextToSpeech.EndToEnd

Options for VITS2 (Kong et al. 2023): VITS with an adversarially trained stochastic duration predictor, noise-scaled monotonic alignment search, a Transformer block in the normalizing flows and a speaker-conditioned text encoder.

Inheritance: ModelOptions → TtsModelOptions → EndToEndTtsOptions → VitsModelOptions → VITS2Options

For Beginners

These options configure the VITS2 model. Default values follow the original paper settings.

How It Works

The shared settings are in VitsModelOptions with VITS2’s changes (§3): no blank token, the 80-band mel spectrogram as the posterior encoder’s input, 256 training instances per step (49 updates per LJSpeech epoch of 12,500 clips), the waveform networks trained for 800k steps and the duration predictor for 30k steps after them (“separately trained as the last training step”, §2.1).

What the paper leaves open follows its community reference implementation (p0p4k/vits2_pytorch, configs/vits2_ljs_nosdp.json): the duration predictor’s layers (DurationPredictor(hidden, 256, 3, 0.5)), the time-step-wise discriminator (DurationDiscriminatorV2(hidden, hidden, 3)), the alignment noise schedule (the paper’s 0.01 falling by 2e-6 per step), the flow’s Transformer (pre_conv: 2 layers, 2 heads, kernel 3, dropout 0.1 over the coupling’s half channels) and the speaker conditioning at the third encoder block.

Constructors

Constructor Summary
VITS2Options() Creates the paper’s configuration.
VITS2Options(VITS2Options other) Initializes a new instance by copying from another instance.

Properties

Property Summary
double AlignmentNoiseDecay { get; set; } Gets or sets how much the alignment noise scale falls per step (2e-6, §2.2); it stops at zero.
double AlignmentNoiseScale { get; set; } Gets or sets the alignment noise scale at the first step (0.01, §2.2).
double DurationNoiseScale { get; set; } Gets or sets the scale of the duration predictor’s noise z_d at synthesis (1: the training distribution).
double FlowTransformerDropout { get; set; } Gets or sets the flow Transformer’s dropout (0.1).
int DurationDiscriminatorKernelSize { get; set; } Gets or sets the duration discriminator’s kernel (3; its width is the text encoder’s).
int DurationPredictorFilterChannels { get; set; } Gets or sets the duration predictor’s convolution width (256).
int FlowTransformerHeads { get; set; } Gets or sets the flow Transformer’s heads (2).
int FlowTransformerKernelSize { get; set; } Gets or sets the flow Transformer’s feed-forward kernel (3).
int FlowTransformerLayers { get; set; } Gets or sets the Transformer layers in each flow coupling (2; 0 removes the block).
int SpeakerConditionedEncoderBlock { get; set; } Gets or sets the text encoder block before which a multi-speaker model adds the speaker embedding (2: the third block, §2.4).
long AcousticTrainingSteps { get; set; } Gets or sets the steps that train the waveform networks before the duration predictor trains on its own (800,000; the duration predictor then trains for the paper’s 30,000 steps).