NaturalSpeechOptions
Options & Configuration · AiDotNet.TextToSpeech.CodecBased
Options for NaturalSpeech.
Inheritance: ModelOptions → TtsModelOptions → NaturalSpeechOptions
For Beginners
NaturalSpeech reads text aloud in one voice (it is trained on one speaker, as on LJSpeech). Train it in stages: optionally pre-train the phoneme encoder on text, then warm up, initialize the memory bank, train jointly and finish with tuning.
How It Works
The defaults are the paper’s (Tan et al. 2022, Table 8 and §4.1). It leaves some details unstated; those come from
heatz123/naturalspeech, and where that port departs from the paper its choice is an option
(Heatz123 selects all of them).
Constructors
| Constructor | Summary |
|---|---|
NaturalSpeechOptions() |
Creates the paper’s configuration. |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<ValueTuple<int, int>> SupPhonemeMerges { get; set; } |
The sup-phoneme merges (pairs of ids in merge order; see LearnSupPhonemes); null reads phonemes alone and pre-trains with masked phoneme prediction only. |
NaturalSpeechPhonemeEncoder PhonemeEncoder { get; set; } |
The phoneme encoder (the paper’s FFT blocks). |
NaturalSpeechStage CurrentStage { get; set; } |
The stage training runs. |
NaturalSpeechUpsampling Upsampling { get; set; } |
The learnable upsampling layer (the paper’s). |
bool AddBlank { get; set; } |
Whether a blank symbol separates the phonemes (VITS’s add_blank, as heatz123 trains; off by default since the paper reads phonemes only). |
bool DurationLossAfterWarmup { get; set; } |
Whether the duration loss stays after warm-up (the paper trains the predictor on alignment labels only in warm-up; heatz123 keeps it). |
bool UseSoftDtw { get; set; } |
Whether the backward and forward KLs use soft dynamic time warping (App. |
double BackwardKlWeight { get; set; } |
The backward KL’s weight (1). |
double Beta1 { get; set; } |
AdamW’s β₁ (0.8). |
double Beta2 { get; set; } |
AdamW’s β₂ (0.99). |
double DurationDropout { get; set; } |
The duration predictor’s dropout (0.5). |
double DurationLossWeight { get; set; } |
The duration loss’s weight (1; heatz123 5). |
double EndToEndWeight { get; set; } |
The end-to-end GAN loss’s weight (1; heatz123 0.1). |
double Epsilon { get; set; } |
AdamW’s ε (1e-9, VITS’s). |
double ForwardKlWeight { get; set; } |
The forward KL’s weight (1: Eq. |
double LearningRateDecay { get; set; } |
The per-epoch learning-rate decay (0.999875). |
double LengthScale { get; set; } |
The durations’ scale at inference (1). |
double MaskRatio { get; set; } |
The fraction of sup-phonemes (or phonemes) masked in pre-training (15%). |
double MelLossWeight { get; set; } |
The mel loss’s weight (45, HiFi-GAN’s, which the paper’s waveform loss follows). |
double NoiseScale { get; set; } |
The prior’s noise scale at inference (1, heatz123’s default). |
double SoftDtwGamma { get; set; } |
The soft-DTW smoothing γ (0.01). |
double SoftDtwWarp { get; set; } |
The soft-DTW warp penalty (0.07). |
int CouplingLayers { get; set; } |
The WaveNet layers in each coupling (4). |
int Couplings { get; set; } |
The flow’s affine coupling layers (4). |
int DiscriminatorWidthDivisor { get; set; } |
A divisor on every discriminator width (1: the paper’s widths). |
int DurationFilter { get; set; } |
The duration predictor’s filters (192; heatz123 256). |
int DurationKernel { get; set; } |
The duration predictor’s kernel (3). |
int DurationLayers { get; set; } |
The duration predictor’s convolution layers (3; heatz123 2). |
int EncoderKernel { get; set; } |
The phoneme encoder’s convolution kernel (3). |
int FilterChannels { get; set; } |
The phoneme encoder’s feed-forward filter (768). |
int HopSize { get; set; } |
The native HiFi-GAN decoder’s upsampling product (256 in the released configuration). |
int MemoryHeads { get; set; } |
The memory attention’s heads (2). |
int MemoryKMeansIterations { get; set; } |
K-means iterations of the memory bank’s initialization (20). |
int MemorySize { get; set; } |
The memory bank’s size (1000). |
int NumDecoderLayers { get; set; } |
The native waveform decoder’s upsampling stages (four in the released configuration). |
int SegmentFrames { get; set; } |
The decoder’s training segment in frames (32: heatz123’s 8192 samples). |
int UpsampleInitialChannels { get; set; } |
The waveform decoder’s first width (256). |
int UpsamplingFilter { get; set; } |
The upsampling convolution’s output channels (8). |
int UpsamplingKernel { get; set; } |
The upsampling convolution’s kernel (3). |
int UpsamplingP { get; set; } |
The auxiliary context width p of the upsampling (2). |
int UpsamplingQ { get; set; } |
The attention heads q of the upsampling (4). |
int? UpdatesPerEpoch { get; set; } |
Training steps per epoch (null: no decay). |
int[] DiscriminatorPeriods { get; set; } |
The multi-period discriminator’s periods (2, 3, 5, 7, 11, with VITS’s scale discriminator for the paper’s period 1). |
int[] ResblockKernels { get; set; } |
The waveform decoder’s residual-block kernels (3, 7, 11). |
int[] UpsampleKernels { get; set; } |
The waveform decoder’s transposed-convolution kernels (16, 16, 4, 4). |
int[] UpsampleRates { get; set; } |
The waveform decoder’s upsampling rates (8, 8, 2, 2). |
int[][] ResblockDilations { get; set; } |
The waveform decoder’s residual-block dilations (1, 3, 5 each). |
string EspeakLanguage { get; set; } |
eSpeak NG’s voice for the phonemizer (the paper’s phonemizer with the espeak backend; “en-us”). |
Methods
| Method | Summary |
|---|---|
NaturalSpeechOptions Heatz123() |
heatz123/naturalspeech’s choices where they depart from the paper: blank-separated phonemes, the VITS phoneme encoder, a 2-layer, 256-filter duration predictor, its learnable upsampling, the plain KL instead of soft-DTW (with the warp scale… |