XTTSv2Options
Options & Configuration · AiDotNet.TextToSpeech.VoiceCloning
Configuration of Casanova et al.’s XTTS-v2 and Coqui’s released checkpoint.
Inheritance: ModelOptions → TtsModelOptions → CodecTtsOptions → XTTSv2Options
Known derived types: XTTSv2CloneOptions
How It Works
Defaults describe coqui/XTTS-v2: a 30-layer, 1024-wide GPT-2, a 1024-entry single-codebook
dVAE, the six-block conditioning encoder and 32-latent Perceiver, H/ASP speaker encoder, and the
speaker-conditioned 24 kHz HiFi-GAN decoder. The paper reports 6,681 text symbols; the released
vocab.json and checkpoint both have 6,681 embedding rows.
Constructors
| Constructor | Summary |
|---|---|
XTTSv2Options() |
Creates the released XTTS-v2 configuration and the paper’s full-training optimizer recipe. |
XTTSv2Options(XTTSv2Options other) |
Creates a deep copy. |
Properties
| Property | Summary |
|---|---|
double AcousticTokenRate { get; } |
Gets or sets the acoustic token rate. |
double AdamBeta1 { get; set; } |
Gets or sets the first-moment decay for the adam optimizer. |
double AdamBeta2 { get; set; } |
Gets or sets the second-moment decay for the adam optimizer. |
double AdamEpsilon { get; set; } |
Gets or sets the numerical stability constant for adam. |
double AudioLossWeight { get; set; } |
Gets or sets the weight of the audio loss. |
double ConditioningChunkSeconds { get; set; } |
Gets or sets the conditioning chunk duration, in seconds. |
double ConditioningSeconds { get; set; } |
Gets or sets the conditioning duration, in seconds. |
double LearningRateGamma { get; set; } |
Gets or sets the learning rate gamma. |
double LengthPenalty { get; set; } |
Gets or sets the length penalty. |
double MaximumReferenceSeconds { get; set; } |
Gets or sets the maximum reference duration, in seconds. |
double MelMaxFrequency { get; set; } |
Gets or sets the mel max frequency. |
double MinimumConditioningChunkSeconds { get; set; } |
Gets or sets the minimum conditioning chunk duration, in seconds. |
double RepetitionPenalty { get; set; } |
Gets or sets the penalty applied to repeated tokens during generation. |
double Speed { get; set; } |
Gets or sets the speed. |
double Temperature { get; set; } |
Gets or sets the logit temperature used during sampling. |
double TextLossWeight { get; set; } |
Gets or sets the weight of the text loss. |
double TopP { get; set; } |
Gets or sets the cumulative probability threshold for nucleus sampling. |
int AudioVocabularySize { get; set; } |
The dVAE codes plus start and stop: CodebookSize + 2 (1026) unless set. |
int CodebookSize { get; set; } |
The dVAE vocabulary, paired with the audio vocabulary and special IDs. |
int ConditioningBlocks { get; set; } |
Gets or sets the conditioning blocks. |
int ConditioningFftSize { get; set; } |
Perceiver conditioning mel parameters from the released inference path. |
int ConditioningHeads { get; set; } |
Gets or sets the number of attention heads in the conditioning encoder. |
int ConditioningHopSize { get; set; } |
Gets or sets the conditioning hop size. |
int ConditioningWindowSize { get; set; } |
Gets or sets the conditioning window size. |
int DecoderInitialChannels { get; set; } |
Gets or sets the decoder initial channels. |
int DvaeCodebookDim { get; set; } |
Gets or sets the feature width of the discrete VAE codebook. |
int DvaeFftSize { get; set; } |
Gets or sets the discrete VAE FFT size. |
int DvaeHiddenDim { get; set; } |
Gets or sets the feature width of the discrete VAE hidden. |
int DvaeHopSize { get; set; } |
Gets or sets the discrete VAE hop size. |
int DvaeLayers { get; set; } |
Gets or sets the number of layers in the discrete VAE. |
int DvaeResidualBlocks { get; set; } |
Gets or sets the discrete VAE residual blocks. |
int DvaeWindowSize { get; set; } |
Gets or sets the discrete VAE window size. |
int GptCodeStride { get; set; } |
Samples represented by one GPT/dVAE code at 22.05 kHz (21.5332 codes/s). |
int GradientAccumulationSteps { get; set; } |
Gets or sets the gradient accumulation steps. |
int InputSampleRate { get; set; } |
Input/reference and dVAE sample rate (22.05 kHz); generated audio is 24 kHz. |
int MaxPromptTokens { get; set; } |
Gets or sets the max prompt tokens. |
int NumCodebooks { get; set; } |
The released dVAE’s single codebook. |
int NumEncoderLayers { get; set; } |
The GPT is decoder-only; use ConditioningBlocks for the reference encoder. |
int PaperTextVocabularySize { get; set; } |
Vocabulary size stated by the paper and present in the pinned official checkpoint. |
int PerceiverDepth { get; set; } |
Gets or sets the perceiver depth. |
int PerceiverFeedForwardMultiple { get; set; } |
Gets or sets the perceiver feed forward multiple. |
int PerceiverHeadDim { get; set; } |
Gets or sets the feature width of the perceiver head. |
int PerceiverHeads { get; set; } |
Gets or sets the number of attention heads in the perceiver. |
int PerceiverLatents { get; set; } |
Gets or sets the perceiver latents. |
int SamplingSeed { get; set; } |
Gets or sets the random seed used during sampling. |
int StartAudioToken { get; set; } |
Gets or sets the token ID for start audio: CodebookSize (1024) unless set. |
int StartTextToken { get; set; } |
Resolved from [START] in the released tokenizer when it is loaded. |
int StopAudioToken { get; set; } |
Gets or sets the token ID for stop audio: CodebookSize + 1 (1025) unless set. |
int StopTextToken { get; set; } |
Resolved from [STOP] in the released tokenizer when it is loaded. |
int TextVocabularySize { get; set; } |
Embedding rows in the released checkpoint. |
int TopK { get; set; } |
Gets or sets the maximum number of highest-probability tokens retained during sampling. |
int VocabSize { get; set; } |
The text embedding rows, paired with the loaded tokenizer vocabulary. |
int[] DecoderResblockKernels { get; set; } |
Gets or sets the convolution kernel sizes for the decoder resblock. |
int[] DecoderUpsampleKernels { get; set; } |
Gets or sets the convolution kernel sizes for the decoder upsample. |
int[] DecoderUpsampleRates { get; set; } |
Gets or sets the decoder upsample rates. |
int[] LearningRateMilestones { get; set; } |
Gets or sets the learning rate milestones. |
int[] SpeakerEncoderFilters { get; set; } |
H/ASP SE-ResNet stage widths from the released config_se.json. |
int[][] DecoderResblockDilations { get; set; } |
Gets or sets the convolution dilations for the decoder resblock. |
string Language { get; set; } |
Gets or sets the language code used for speech processing. |
string MelStatsPath { get; set; } |
Optional released mel_stats.pth. |
string TokenizerPath { get; set; } |
Optional Hugging Face/tokenizers tokenizer JSON shipped as vocab.json. |
Methods
| Method | Summary |
|---|---|
XTTSv2Options OfficialGptFineTuning() |
The public GPT fine-tuning recipe (the frozen dVAE and decoder are not updated). |
Fields
| Field | Summary |
|---|---|
string[] ReleasedLanguages |
The language ids in the released checkpoint, in model-card order. |