XTTSv2Options

Options & Configuration · AiDotNet.TextToSpeech.VoiceCloning

Configuration of Casanova et al.’s XTTS-v2 and Coqui’s released checkpoint.

Inheritance: ModelOptions → TtsModelOptions → CodecTtsOptions → XTTSv2Options

Known derived types: XTTSv2CloneOptions

How It Works

Defaults describe coqui/XTTS-v2: a 30-layer, 1024-wide GPT-2, a 1024-entry single-codebook dVAE, the six-block conditioning encoder and 32-latent Perceiver, H/ASP speaker encoder, and the speaker-conditioned 24 kHz HiFi-GAN decoder. The paper reports 6,681 text symbols; the released vocab.json and checkpoint both have 6,681 embedding rows.

Constructors

Constructor Summary
XTTSv2Options() Creates the released XTTS-v2 configuration and the paper’s full-training optimizer recipe.
XTTSv2Options(XTTSv2Options other) Creates a deep copy.

Properties

Property Summary
double AcousticTokenRate { get; } Gets or sets the acoustic token rate.
double AdamBeta1 { get; set; } Gets or sets the first-moment decay for the adam optimizer.
double AdamBeta2 { get; set; } Gets or sets the second-moment decay for the adam optimizer.
double AdamEpsilon { get; set; } Gets or sets the numerical stability constant for adam.
double AudioLossWeight { get; set; } Gets or sets the weight of the audio loss.
double ConditioningChunkSeconds { get; set; } Gets or sets the conditioning chunk duration, in seconds.
double ConditioningSeconds { get; set; } Gets or sets the conditioning duration, in seconds.
double LearningRateGamma { get; set; } Gets or sets the learning rate gamma.
double LengthPenalty { get; set; } Gets or sets the length penalty.
double MaximumReferenceSeconds { get; set; } Gets or sets the maximum reference duration, in seconds.
double MelMaxFrequency { get; set; } Gets or sets the mel max frequency.
double MinimumConditioningChunkSeconds { get; set; } Gets or sets the minimum conditioning chunk duration, in seconds.
double RepetitionPenalty { get; set; } Gets or sets the penalty applied to repeated tokens during generation.
double Speed { get; set; } Gets or sets the speed.
double Temperature { get; set; } Gets or sets the logit temperature used during sampling.
double TextLossWeight { get; set; } Gets or sets the weight of the text loss.
double TopP { get; set; } Gets or sets the cumulative probability threshold for nucleus sampling.
int AudioVocabularySize { get; set; } The dVAE codes plus start and stop: CodebookSize + 2 (1026) unless set.
int CodebookSize { get; set; } The dVAE vocabulary, paired with the audio vocabulary and special IDs.
int ConditioningBlocks { get; set; } Gets or sets the conditioning blocks.
int ConditioningFftSize { get; set; } Perceiver conditioning mel parameters from the released inference path.
int ConditioningHeads { get; set; } Gets or sets the number of attention heads in the conditioning encoder.
int ConditioningHopSize { get; set; } Gets or sets the conditioning hop size.
int ConditioningWindowSize { get; set; } Gets or sets the conditioning window size.
int DecoderInitialChannels { get; set; } Gets or sets the decoder initial channels.
int DvaeCodebookDim { get; set; } Gets or sets the feature width of the discrete VAE codebook.
int DvaeFftSize { get; set; } Gets or sets the discrete VAE FFT size.
int DvaeHiddenDim { get; set; } Gets or sets the feature width of the discrete VAE hidden.
int DvaeHopSize { get; set; } Gets or sets the discrete VAE hop size.
int DvaeLayers { get; set; } Gets or sets the number of layers in the discrete VAE.
int DvaeResidualBlocks { get; set; } Gets or sets the discrete VAE residual blocks.
int DvaeWindowSize { get; set; } Gets or sets the discrete VAE window size.
int GptCodeStride { get; set; } Samples represented by one GPT/dVAE code at 22.05 kHz (21.5332 codes/s).
int GradientAccumulationSteps { get; set; } Gets or sets the gradient accumulation steps.
int InputSampleRate { get; set; } Input/reference and dVAE sample rate (22.05 kHz); generated audio is 24 kHz.
int MaxPromptTokens { get; set; } Gets or sets the max prompt tokens.
int NumCodebooks { get; set; } The released dVAE’s single codebook.
int NumEncoderLayers { get; set; } The GPT is decoder-only; use ConditioningBlocks for the reference encoder.
int PaperTextVocabularySize { get; set; } Vocabulary size stated by the paper and present in the pinned official checkpoint.
int PerceiverDepth { get; set; } Gets or sets the perceiver depth.
int PerceiverFeedForwardMultiple { get; set; } Gets or sets the perceiver feed forward multiple.
int PerceiverHeadDim { get; set; } Gets or sets the feature width of the perceiver head.
int PerceiverHeads { get; set; } Gets or sets the number of attention heads in the perceiver.
int PerceiverLatents { get; set; } Gets or sets the perceiver latents.
int SamplingSeed { get; set; } Gets or sets the random seed used during sampling.
int StartAudioToken { get; set; } Gets or sets the token ID for start audio: CodebookSize (1024) unless set.
int StartTextToken { get; set; } Resolved from [START] in the released tokenizer when it is loaded.
int StopAudioToken { get; set; } Gets or sets the token ID for stop audio: CodebookSize + 1 (1025) unless set.
int StopTextToken { get; set; } Resolved from [STOP] in the released tokenizer when it is loaded.
int TextVocabularySize { get; set; } Embedding rows in the released checkpoint.
int TopK { get; set; } Gets or sets the maximum number of highest-probability tokens retained during sampling.
int VocabSize { get; set; } The text embedding rows, paired with the loaded tokenizer vocabulary.
int[] DecoderResblockKernels { get; set; } Gets or sets the convolution kernel sizes for the decoder resblock.
int[] DecoderUpsampleKernels { get; set; } Gets or sets the convolution kernel sizes for the decoder upsample.
int[] DecoderUpsampleRates { get; set; } Gets or sets the decoder upsample rates.
int[] LearningRateMilestones { get; set; } Gets or sets the learning rate milestones.
int[] SpeakerEncoderFilters { get; set; } H/ASP SE-ResNet stage widths from the released config_se.json.
int[][] DecoderResblockDilations { get; set; } Gets or sets the convolution dilations for the decoder resblock.
string Language { get; set; } Gets or sets the language code used for speech processing.
string MelStatsPath { get; set; } Optional released mel_stats.pth.
string TokenizerPath { get; set; } Optional Hugging Face/tokenizers tokenizer JSON shipped as vocab.json.

Methods

Method Summary
XTTSv2Options OfficialGptFineTuning() The public GPT fine-tuning recipe (the frozen dVAE and decoder are not updated).

Fields

Field Summary
string[] ReleasedLanguages The language ids in the released checkpoint, in model-card order.