StyleTTS2Options

Options & Configuration · AiDotNet.TextToSpeech.StyleEmotion

Options for StyleTTS2.

Inheritance: ModelOptions → StyleTtsBaseOptions → StyleTtsWaveformOptions → StyleTTS2Options

For Beginners

These settings size the networks and control how speech is generated; the defaults reproduce the published multispeaker model.

How It Works

The defaults are the paper’s multispeaker model as the released LibriTTS configuration builds it (yl4579/StyleTTS2 Configs/config_libritts.yml): HiFi-GAN decoder, 128-dimensional acoustic and prosodic styles, a 3-layer style-diffusion transformer conditioned on PL-BERT, 50 duration bins. The LJSpeech configuration is Multispeaker false with the iSTFTNet decoder (DecoderType, UpsampleRates {10, 6}, UpsampleKernels {20, 12}).

Constructors

Constructor Summary
StyleTTS2Options() Creates the released LibriTTS configuration.
StyleTTS2Options(StyleTTS2Options other) Copies another instance.

Properties

Property Summary
IReadOnlyList<int[]> OutOfDistributionTexts { get; set; } Phoneme id sequences of other texts (framed by the pad symbol) for SLM adversarial training’s out-of- distribution half (the reference’s OOD_texts.txt); without them every SLM step reads the training text.
bool EstimateSigmaData { get; set; } Whether training re-estimates σ_data as the mean standard deviation of the target styles (true).
bool Multispeaker { get; set; } Whether the style denoiser reads a reference style (the LibriTTS model, true) or not (LJSpeech).
double Alpha { get; set; } Weight of the sampled acoustic style against the reference’s (0.3; multispeaker only).
double BertBeta1 { get; set; } PL-BERT’s AdamW β₁ (0.9; the other modules use 0).
double BertLearningRate { get; set; } PL-BERT’s learning rate in joint training (1e-5).
double BertWeightDecay { get; set; } PL-BERT’s AdamW weight decay (0.01).
double Beta { get; set; } Weight of the sampled prosodic style against the reference’s (0.7; multispeaker only).
double DiffusionLossWeight { get; set; } λ_diff, the EDM score-matching loss (1).
double DropoutRate { get; set; } Native prosody-predictor dropout probability.
double DurationCrossEntropyWeight { get; set; } λ_ce, the duration bins’ binary cross-entropy (20).
double DurationLossWeight { get; set; } λ_dur, the duration L1 loss (1).
double EmbeddingMaskProbability { get; set; } Probability of training the denoiser unconditionally (0.1), for classifier-free guidance.
double EmbeddingScale { get; set; } Classifier-free guidance scale at inference (1: none).
double F0LossWeight { get; set; } λ_F0, the pitch reconstruction loss (1; the loss itself is ÷10).
double FineTuneLearningRate { get; set; } The decoder’s and acoustic style encoder’s learning rate in joint training (1e-5).
double GeneratorLossWeight { get; set; } λ_gen, the wave discriminators’ adversarial loss (1).
double LogSigmaMean { get; set; } Mean of ln σ for the training noise levels (−3).
double LogSigmaStd { get; set; } Standard deviation of ln σ (1).
double MelLossWeight { get; set; } λ_mel, the multi-resolution mel loss (5).
double MonotonicLossWeight { get; set; } λ_mono, the aligner’s monotonic loss (1; the loss itself is ×10).
double NormLossWeight { get; set; } λ_norm, the energy reconstruction loss (1).
double S2sLossWeight { get; set; } λ_s2s, the aligner’s token cross-entropy (1).
double SigmaData { get; set; } σ_data of the EDM preconditioning (0.2, replaced by each batch’s estimate when EstimateSigmaData).
double SlmDurationSigma { get; set; } σ of the Gaussian upsampling of differentiable duration modeling (1.5).
double SlmGradientScale { get; set; } Scale of the SLM step’s gradients on the duration head, its LSTM and the diffusion model (0.01).
double SlmGradientThreshold { get; set; } The predictor’s gradient norm above which the SLM step’s gradients are divided by it (5).
double SlmLossWeight { get; set; } λ_slm, WavLM feature matching (1).
double StyleLossWeight { get; set; } λ_sty, the sampled style’s L1 reconstruction loss (1).
int DiffusionHeadFeatures { get; set; } Its per-head width (64).
int DiffusionHeads { get; set; } Its attention heads (8).
int DiffusionLayers { get; set; } Blocks of the style-diffusion transformer (3).
int DiffusionMaxLength { get; set; } Length of the denoiser’s unconditional embedding table (512, StyleTransformer1d’s default).
int DiffusionMultiplier { get; set; } Its feed-forward multiplier (2).
int DiffusionStartStep { get; set; } Joint-training updates before the style diffusion and the wave discriminators start (the reference’s diff_epoch: 10 epochs for LibriTTS, 20 for LJSpeech, whose length depends on your data — set it for yours).
int DiffusionSteps { get; set; } Sampling steps of the style diffusion at inference (5).
int DurationBins { get; set; } Duration bins of the duration head (max_dur, 50): the duration is the sum of their sigmoids.
int FinalPhonemeExtraFrames { get; set; } Frames added to the last phoneme’s duration (0; the LJSpeech notebook adds 5).
int HiddenDim { get; set; } Native text and prosody hidden width.
int HopSize { get; set; } Samples per mel frame, paired with the waveform generator’s upsampling and iSTFT hop.
int IstftHopSize { get; set; } iSTFT waveform hop, paired with the generator’s rates and mel hop.
int JointStartStep { get; set; } Joint-training updates before the decoder and style encoder are fine-tuned and SLM adversarial training starts (the reference’s joint_epoch: 15 epochs for LibriTTS, 50 for LJSpeech).
int MaxConvDim { get; set; } Maximum width of each native style encoder.
int SlmAdversarialMaxFrames { get; set; } Mel frames of the longest SLM adversarial segment (500).
int SlmAdversarialMinFrames { get; set; } Mel frames of the shortest SLM adversarial segment (slmadv_params.min_len, 400); utterances shorter than this skip the step.
int SlmDiscriminatorChannels { get; set; } First width of the SLM discriminator head (64).
int SlmDiscriminatorInterval { get; set; } Joint updates per SLM discriminator update (iter: 20 for LibriTTS, 10 for LJSpeech).
int StyleEncoderDim { get; set; } Initial width of each native style encoder.
int TextKernel { get; set; } Text convolution kernel.
int TextLayers { get; set; } Text convolution and duration encoder layers.
int TrimEndSamples { get; set; } Samples cut from the end of the waveform (50, the LibriTTS notebook’s fix for a pulse at the end).
int[] UpsampleKernels { get; set; } The released transposed-convolution kernels, twice each upsampling rate.
int[] UpsampleRates { get; set; } Native waveform upsampling factors, paired with their doubled kernels and mel hop.