MegaTTSOptions

Options & Configuration · AiDotNet.TextToSpeech.Latest

Options for MegaTTS. The sizes are the paper’s (Jiang et al. 2023, Table 7); what it leaves unstated comes from the authors’ NATSpeech and SyntaSpeech code.

Inheritance: ModelOptions → TtsModelOptions → MegaTTSOptions

For Beginners

Mega-TTS speaks in a voice from a few seconds of it, copying both how the person sounds and how they tend to speak. These options size its parts; train it on recordings with phoneme durations.

Constructors

Constructor Summary
MegaTTSOptions() Creates an instance with the default settings.

Properties

Property Summary
MegaTtsLanguage Language { get; set; } The text front end (English).
MegaTtsStage Stage { get; set; } The model MegaTTS trains.
double AdversarialWeight { get; set; } The adversarial loss weight (SyntaSpeech’s 0.05).
double Beta1 { get; set; } Adam’s β1 (0.9), β2 (0.98) and ε (1e-9), on the Transformer schedule.
double Beta2 { get; set; } See Beta1.
double DiscriminatorDecay { get; set; } See DiscriminatorLearningRate.
double DiscriminatorEpsilon { get; set; } See DiscriminatorLearningRate.
double DiscriminatorLearningRate { get; set; } The discriminator’s learning rate (1e-4), ε (1e-6), and its step decay (halved every 40000 steps), as SyntaSpeech trains it.
double DurationWeight { get; set; } The phoneme-duration loss weight (NATSpeech’s 0.1); word and sentence durations weigh 1.
double Epsilon { get; set; } See Beta1.
double MaxGradientNorm { get; set; } The gradient norm every step clips to (NATSpeech’s clip_grad_norm of 1); 0 turns clipping off.
double MelMaxFrequency { get; set; } The mel filterbank’s highest frequency (NATSpeech’s 7600 Hz).
double MelMinFrequency { get; set; } The mel filterbank’s lowest frequency (NATSpeech’s 80 Hz).
double ScheduleFactor { get; set; } The Transformer schedule’s scale (1: Vaswani et al.’s rate, d^−½ · min(t^−½, t · warmup^−1.5)).
double SentenceDurationWeight { get; set; } See WordDurationWeight.
double WordDurationWeight { get; set; } The word- and sentence-duration loss weights (NATSpeech’s 1 and 1).
int CodeDimension { get; set; } See Codes.
int Codes { get; set; } VQ codebook entries (2048) and their width (256).
int ContentHeads { get; set; } See ContentLayers.
int ContentLayers { get; set; } Content encoder layers (4) and heads (2), feed-forward kernel 5.
int ContextSentences { get; set; } The sentences of one speaker the P-LLM reads as one sequence in training (Table 7’s 7 contextual sentences); see TrainProsodyModel.
int ConvBlocks { get; set; } Convolution blocks of the prosody encoder’s two stacks, the timbre encoder and the mel decoder (5).
int DiscriminatorDecaySteps { get; set; } See DiscriminatorLearningRate.
int DiscriminatorHidden { get; set; } See DiscriminatorWindows.
int DurationKernel { get; set; } See DurationLayers.
int DurationLayers { get; set; } The duration predictor’s convolutions (3) and their kernel (NATSpeech’s 3).
int LmHeads { get; set; } See LmLayers.
int LmHidden { get; set; } See LmLayers.
int LmLayers { get; set; } The prosody language model’s layers (8), width (512) and heads (8), feed-forward kernel 5.
int NumDecoderLayers { get; set; } Gets or sets the convolutional depth shared by the acoustic decoder and encoders.
int NumEncoderLayers { get; set; } Gets or sets the FastSpeech content encoder depth.
int NumHeads { get; set; } Gets or sets the FastSpeech content encoder attention head count.
int ProsodyBins { get; set; } The low mel bins the prosody encoder reads (the first 20).
int SamplingSeed { get; set; } The seed of the prosody sampling.
int TimbreKernel { get; set; } The timbre encoder’s kernel (31).
int TopK { get; set; } Top-k sampling of the prosody codes (5).
int TrainingSeed { get; set; } The seed of training’s draws (the discriminator windows).
int VocoderInitialChannels { get; set; } HiFi-GAN V1’s first width (512), upsampling rates and kernels, and residual-block kernels and dilations (Kong et al.
int WarmupSteps { get; set; } The Transformer schedule’s warm-up steps (4000).
int WindowSize { get; set; } The analysis window, in samples (NATSpeech’s 1024).
int[] DiscriminatorWindows { get; set; } The discriminator’s windows (32, 64, 128 frames) and width (192).
int[] VocoderKernels { get; set; } See VocoderInitialChannels.
int[] VocoderRates { get; set; } See VocoderInitialChannels.
int[] VocoderResBlockKernels { get; set; } See VocoderInitialChannels.
int[][] VocoderResBlockDilations { get; set; } See VocoderInitialChannels.
string[] PhoneSet { get; set; } The phone set (NATSpeech’s ph_set.json): its sorted phones follow the three reserved tokens.