StyleTTS2Options
Options & Configuration · AiDotNet.TextToSpeech.StyleEmotion
Options for StyleTTS2.
Inheritance: ModelOptions → StyleTtsBaseOptions → StyleTtsWaveformOptions → StyleTTS2Options
For Beginners
These settings size the networks and control how speech is generated; the defaults reproduce the published multispeaker model.
How It Works
The defaults are the paper’s multispeaker model as the released LibriTTS configuration builds it
(yl4579/StyleTTS2 Configs/config_libritts.yml): HiFi-GAN decoder, 128-dimensional acoustic and prosodic styles,
a 3-layer style-diffusion transformer conditioned on PL-BERT, 50 duration bins. The LJSpeech configuration is
Multispeaker false with the iSTFTNet decoder (DecoderType, UpsampleRates
{10, 6}, UpsampleKernels {20, 12}).
Constructors
| Constructor | Summary |
|---|---|
StyleTTS2Options() |
Creates the released LibriTTS configuration. |
StyleTTS2Options(StyleTTS2Options other) |
Copies another instance. |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<int[]> OutOfDistributionTexts { get; set; } |
Phoneme id sequences of other texts (framed by the pad symbol) for SLM adversarial training’s out-of- distribution half (the reference’s OOD_texts.txt); without them every SLM step reads the training text. |
bool EstimateSigmaData { get; set; } |
Whether training re-estimates σ_data as the mean standard deviation of the target styles (true). |
bool Multispeaker { get; set; } |
Whether the style denoiser reads a reference style (the LibriTTS model, true) or not (LJSpeech). |
double Alpha { get; set; } |
Weight of the sampled acoustic style against the reference’s (0.3; multispeaker only). |
double BertBeta1 { get; set; } |
PL-BERT’s AdamW β₁ (0.9; the other modules use 0). |
double BertLearningRate { get; set; } |
PL-BERT’s learning rate in joint training (1e-5). |
double BertWeightDecay { get; set; } |
PL-BERT’s AdamW weight decay (0.01). |
double Beta { get; set; } |
Weight of the sampled prosodic style against the reference’s (0.7; multispeaker only). |
double DiffusionLossWeight { get; set; } |
λ_diff, the EDM score-matching loss (1). |
double DropoutRate { get; set; } |
Native prosody-predictor dropout probability. |
double DurationCrossEntropyWeight { get; set; } |
λ_ce, the duration bins’ binary cross-entropy (20). |
double DurationLossWeight { get; set; } |
λ_dur, the duration L1 loss (1). |
double EmbeddingMaskProbability { get; set; } |
Probability of training the denoiser unconditionally (0.1), for classifier-free guidance. |
double EmbeddingScale { get; set; } |
Classifier-free guidance scale at inference (1: none). |
double F0LossWeight { get; set; } |
λ_F0, the pitch reconstruction loss (1; the loss itself is ÷10). |
double FineTuneLearningRate { get; set; } |
The decoder’s and acoustic style encoder’s learning rate in joint training (1e-5). |
double GeneratorLossWeight { get; set; } |
λ_gen, the wave discriminators’ adversarial loss (1). |
double LogSigmaMean { get; set; } |
Mean of ln σ for the training noise levels (−3). |
double LogSigmaStd { get; set; } |
Standard deviation of ln σ (1). |
double MelLossWeight { get; set; } |
λ_mel, the multi-resolution mel loss (5). |
double MonotonicLossWeight { get; set; } |
λ_mono, the aligner’s monotonic loss (1; the loss itself is ×10). |
double NormLossWeight { get; set; } |
λ_norm, the energy reconstruction loss (1). |
double S2sLossWeight { get; set; } |
λ_s2s, the aligner’s token cross-entropy (1). |
double SigmaData { get; set; } |
σ_data of the EDM preconditioning (0.2, replaced by each batch’s estimate when EstimateSigmaData). |
double SlmDurationSigma { get; set; } |
σ of the Gaussian upsampling of differentiable duration modeling (1.5). |
double SlmGradientScale { get; set; } |
Scale of the SLM step’s gradients on the duration head, its LSTM and the diffusion model (0.01). |
double SlmGradientThreshold { get; set; } |
The predictor’s gradient norm above which the SLM step’s gradients are divided by it (5). |
double SlmLossWeight { get; set; } |
λ_slm, WavLM feature matching (1). |
double StyleLossWeight { get; set; } |
λ_sty, the sampled style’s L1 reconstruction loss (1). |
int DiffusionHeadFeatures { get; set; } |
Its per-head width (64). |
int DiffusionHeads { get; set; } |
Its attention heads (8). |
int DiffusionLayers { get; set; } |
Blocks of the style-diffusion transformer (3). |
int DiffusionMaxLength { get; set; } |
Length of the denoiser’s unconditional embedding table (512, StyleTransformer1d’s default). |
int DiffusionMultiplier { get; set; } |
Its feed-forward multiplier (2). |
int DiffusionStartStep { get; set; } |
Joint-training updates before the style diffusion and the wave discriminators start (the reference’s diff_epoch: 10 epochs for LibriTTS, 20 for LJSpeech, whose length depends on your data — set it for yours). |
int DiffusionSteps { get; set; } |
Sampling steps of the style diffusion at inference (5). |
int DurationBins { get; set; } |
Duration bins of the duration head (max_dur, 50): the duration is the sum of their sigmoids. |
int FinalPhonemeExtraFrames { get; set; } |
Frames added to the last phoneme’s duration (0; the LJSpeech notebook adds 5). |
int HiddenDim { get; set; } |
Native text and prosody hidden width. |
int HopSize { get; set; } |
Samples per mel frame, paired with the waveform generator’s upsampling and iSTFT hop. |
int IstftHopSize { get; set; } |
iSTFT waveform hop, paired with the generator’s rates and mel hop. |
int JointStartStep { get; set; } |
Joint-training updates before the decoder and style encoder are fine-tuned and SLM adversarial training starts (the reference’s joint_epoch: 15 epochs for LibriTTS, 50 for LJSpeech). |
int MaxConvDim { get; set; } |
Maximum width of each native style encoder. |
int SlmAdversarialMaxFrames { get; set; } |
Mel frames of the longest SLM adversarial segment (500). |
int SlmAdversarialMinFrames { get; set; } |
Mel frames of the shortest SLM adversarial segment (slmadv_params.min_len, 400); utterances shorter than this skip the step. |
int SlmDiscriminatorChannels { get; set; } |
First width of the SLM discriminator head (64). |
int SlmDiscriminatorInterval { get; set; } |
Joint updates per SLM discriminator update (iter: 20 for LibriTTS, 10 for LJSpeech). |
int StyleEncoderDim { get; set; } |
Initial width of each native style encoder. |
int TextKernel { get; set; } |
Text convolution kernel. |
int TextLayers { get; set; } |
Text convolution and duration encoder layers. |
int TrimEndSamples { get; set; } |
Samples cut from the end of the waveform (50, the LibriTTS notebook’s fix for a pulse at the end). |
int[] UpsampleKernels { get; set; } |
The released transposed-convolution kernels, twice each upsampling rate. |
int[] UpsampleRates { get; set; } |
Native waveform upsampling factors, paired with their doubled kernels and mel hop. |