MegaTTS2Options
Options & Configuration · AiDotNet.TextToSpeech.Latest
Options for MegaTTS2. The sizes are the paper’s (Jiang et al. 2024, Appendix A.4 and Table 5);
what it leaves unstated comes from the authors’ code (NATSpeech, SyntaSpeech, MegaTTS 3’s release) and from CVQ-VAE.
Inheritance: ModelOptions → TtsModelOptions → MegaTTS2Options
For Beginners
Mega-TTS 2 speaks in a voice it has never heard from a few seconds, or a few minutes, of it: the more prompt sentences you give it, the closer it gets to the speaker. These options size its parts; train it on recordings with phoneme durations.
Constructors
| Constructor | Summary |
|---|---|
MegaTTS2Options() |
Creates an instance with the default settings. |
Properties
| Property | Summary |
|---|---|
MegaTts2CodeAnchor CodeAnchor { get; set; } |
Where the dynamic initialization draws its anchors (CVQ-VAE’s closest features). |
MegaTts2CodeDistance CodeDistance { get; set; } |
How a vector finds its code (CVQ-VAE’s cosine similarity). |
MegaTts2Stage Stage { get; set; } |
The stage MegaTTS2 trains. |
MegaTtsLanguage Language { get; set; } |
The text front end (English; the paper trains on LibriLight). |
double AdversarialWeight { get; set; } |
The adversarial loss weight (SyntaSpeech’s 0.05). |
double Beta1 { get; set; } |
Adam’s β1 (0.9), β2 (0.999) and ε (1e-9), on the Transformer schedule (§4.1). |
double Beta2 { get; set; } |
See Beta1. |
double CodeUsageDecay { get; set; } |
The decay of the codes’ running usage (CVQ-VAE’s 0.99). |
double CommitmentWeight { get; set; } |
The VQ loss’s commitment weight β (0.25, van den Oord et al. |
double DiscriminatorDecay { get; set; } |
See DiscriminatorLearningRate. |
double DiscriminatorEpsilon { get; set; } |
See DiscriminatorLearningRate. |
double DiscriminatorLearningRate { get; set; } |
The discriminator’s learning rate (1e-4), ε (1e-6) and step decay (halved every 40000 steps), as SyntaSpeech trains it. |
double Epsilon { get; set; } |
See Beta1. |
double MaxGradientNorm { get; set; } |
The gradient norm every step clips to (NATSpeech’s clip_grad_norm of 1); 0 turns clipping off. |
double MelMaxFrequency { get; set; } |
The mel filterbank’s highest frequency (NATSpeech’s 7600 Hz). |
double MelMinFrequency { get; set; } |
The mel filterbank’s lowest frequency (NATSpeech’s 80 Hz). |
double ProsodyInterpolation { get; set; } |
Prosody interpolation’s weight γ on the auxiliary prosody prompt (0.8 in the paper’s transfer experiments, Eq. |
double ScheduleFactor { get; set; } |
The Transformer schedule’s scale (1: d^−½ · min(t^−½, t · warmup^−1.5)). |
int CodeDimension { get; set; } |
See Codes. |
int Codes { get; set; } |
The codebook’s entries (1024) and their width (256). |
int Compression { get; set; } |
The VQ encoder’s compression rate r: the max pooling of the mel spectrogram (8). |
int ContentFilter { get; set; } |
See ContentLayers. |
int ContentHeads { get; set; } |
See ContentLayers. |
int ContentLayers { get; set; } |
The content encoder’s layers (8), heads (NATSpeech’s 2) and feed-forward filter (1024); kernel 5. |
int DecoderBlocks { get; set; } |
The mel decoder’s blocks (4 blocks of 2 layers, kernel 5). |
int DiscriminatorDecaySteps { get; set; } |
See DiscriminatorLearningRate. |
int DiscriminatorHidden { get; set; } |
See DiscriminatorWindows. |
int Downsampling { get; set; } |
See TimbreHidden. |
int DurationCodes { get; set; } |
The largest duration the duration model reads as a token; longer ones read as it (MegaTTS 3’s 128). |
int DurationHeads { get; set; } |
See DurationLayers. |
int DurationHidden { get; set; } |
See DurationLayers. |
int DurationLayers { get; set; } |
The autoregressive duration model’s layers (8), width (512) and heads (8, MegaTTS 3’s). |
int LmHeads { get; set; } |
See LmLayers. |
int LmHidden { get; set; } |
See LmLayers. |
int LmLayers { get; set; } |
The P-LLM’s layers (12), width (1024) and heads (16). |
int NumDecoderLayers { get; set; } |
Gets or sets the mel decoder’s convolutional block count. |
int NumEncoderLayers { get; set; } |
Gets or sets the FastSpeech content encoder depth. |
int NumHeads { get; set; } |
Gets or sets the FastSpeech content encoder attention head count. |
int ProsodyBlocks { get; set; } |
See ProsodyHidden. |
int ProsodyHidden { get; set; } |
The VQ prosody encoder’s width (384) and blocks per stack (3 blocks of 2 layers, kernel 5). |
int ProsodyTokenBudget { get; set; } |
The most prosody tokens one language-model step reads (the paper’s 4000 per GPU). |
int SamplingSeed { get; set; } |
The seed of the prosody sampling. |
int TimbreBlocks { get; set; } |
See TimbreHidden. |
int TimbreHeads { get; set; } |
The timbre-to-content attention’s heads (NATSpeech’s 2). |
int TimbreHidden { get; set; } |
The MRTE’s acoustic width (the key encoder’s 256), blocks per stack (5 blocks of 2 layers, kernel 3) and downsampling rate d (16). |
int TimbreReferenceFrames { get; set; } |
The frames of the other recordings the MRTE reads in training, drawn at random (2000). |
int TopK { get; set; } |
Top-k sampling of the prosody codes (10, Appendix A.6). |
int TrainingSeed { get; set; } |
The seed of training’s draws (the discriminator windows, the MRTE’s frames, CVQ-VAE’s anchors). |
int VocoderInitialChannels { get; set; } |
HiFi-GAN V1’s first width (512), upsampling rates and kernels, and residual-block kernels and dilations (Kong et al. |
int WarmupSteps { get; set; } |
The Transformer schedule’s warm-up steps (Vaswani et al.’s 4000). |
int WindowSize { get; set; } |
The analysis window, in samples (NATSpeech’s 1024). |
int[] DiscriminatorWindows { get; set; } |
The multi-length discriminator’s windows (32, 64, 128 frames) and width (192). |
int[] VocoderKernels { get; set; } |
See VocoderInitialChannels. |
int[] VocoderRates { get; set; } |
See VocoderInitialChannels. |
int[] VocoderResBlockKernels { get; set; } |
See VocoderInitialChannels. |
int[][] VocoderResBlockDilations { get; set; } |
See VocoderInitialChannels. |
string[] PhoneSet { get; set; } |
The phone set (NATSpeech’s ph_set.json): its sorted phones follow the three reserved tokens. |