FireRedTTSOptions
Options & Configuration · AiDotNet.TextToSpeech.CodecBased
Options for FireRedTTS: the release’s sizes (FireRedTeam/FireRedTTS config_24k.json),
decoding as its synthesize runs it, and the training recipes.
Inheritance: ModelOptions → FireRedTTSOptions
For Beginners
the defaults build the released model; load its weights with
LoadReferenceWeights. Smaller values make a model to train from scratch.
Constructors
| Constructor | Summary |
|---|---|
FireRedTTSOptions() |
Initializes a new instance with default values. |
Properties
| Property | Summary |
|---|---|
FireRedTtsMelDecoder MelDecoder { get; set; } |
The Mel decoder synthesis uses. |
FireRedTtsStage Stage { get; set; } |
The network a training step updates. |
FireRedTtsVocoder Vocoder { get; set; } |
The vocoder: the release’s 24 kHz or the paper’s 48 kHz. |
OnnxModelOptions OnnxOptions { get; set; } |
Settings for the exported ONNX graph runtime. |
bool ConstrainToSemanticTokens { get; set; } |
Keeps decoding to semantic tokens and the stop token (off: the release samples from every audio token, and a start token would fail in the flow). |
bool HubertNormalizeWave { get; set; } |
Whether HuBERT’s input is layer-normalized (the checkpoint’s task.normalize; chinese-HuBERT-large: true). |
bool InstructionTuning { get; set; } |
Adds the emotion and paralinguistic embeddings and the paper’s tags to the tokenizer. |
bool ReleaseTextSplit { get; set; } |
The release’s text_split drops a last piece of 15 characters or more; false (the default) keeps it. |
bool Sample { get; set; } |
Sample the semantic tokens (the release: true); false takes the most likely token. |
double AcousticLmDropout { get; set; } |
Gets or sets the dropout probability for the acoustic language-model. |
double AcousticTemperature { get; set; } |
Gets or sets the logit temperature used for acoustic-token sampling. |
double AudioLossWeight { get; set; } |
Gets or sets the weight of the audio loss. |
double Beta1 { get; set; } |
The GPT’s AdamW (XTTS: 0.9, 0.96, 1e-8; rate LearningRate, decay on weights). |
double Beta2 { get; set; } |
Gets or sets the second-moment decay for the training optimizer. |
double CodebookEmaDecay { get; set; } |
The codebooks’ EMA decay (DPQ-VAE §4.1.2: 0.9 for PQ-VAE). |
double CodecAdversarialWeight { get; set; } |
Gets or sets the weight of the codec adversarial objective. |
double CodecCommitmentWeight { get; set; } |
Gets or sets the weight of the codec commitment objective. |
double CodecEmaDecay { get; set; } |
The codec’s EMA decay and commitment weight (SoCodec’s config: 0.99, 1.0). |
double CodecReconstructionWeight { get; set; } |
The codec’s losses: L1 mel reconstruction and the LSGAN generator loss (no paper or code gives them). |
double DeadCodeThreshold { get; set; } |
The cluster size below which a codeword is replaced (the 1S code’s threshold_ema_dead_code). |
double DropoutRate { get; set; } |
The GPT training dropout. |
double DualDecodingFinalWeight { get; set; } |
Dual decoding’s weight γ on the pre-quantized reconstruction (DPQ-VAE eq. |
double Epsilon { get; set; } |
Gets or sets the numerical stability constant of the optimizer. |
double FlowDropout { get; set; } |
Gets or sets the dropout probability for the flow. |
double InferenceCfgRate { get; set; } |
Classifier-free guidance at inference (eq. |
double LearningRate { get; set; } |
The text-to-speech GPT learning rate. |
double LearningRateDecay { get; set; } |
Gets or sets the learning rate decay. |
double MelLossWeight { get; set; } |
Gets or sets the weight of the mel loss. |
double RepetitionPenalty { get; set; } |
Gets or sets the penalty applied to repeated tokens during generation. |
double SslLossWeight { get; set; } |
Gets or sets the weight of the self-supervised loss. |
double Temperature { get; set; } |
Gets or sets the logit temperature used during sampling. |
double TextLossWeight { get; set; } |
XTTS’s loss weights: text 0.01, audio 1. |
double TopP { get; set; } |
Gets or sets the cumulative probability threshold for nucleus sampling. |
double TrainingCfgRate { get; set; } |
The probability training drops μ (§3.3.1: 20 %). |
double VqLossWeight { get; set; } |
Eq. |
double WeightDecay { get; set; } |
The text-to-speech GPT weight decay. |
int AcousticLmDim { get; set; } |
The acoustic LM’s sizes (MusicGen-small: 24 layers, 1024 wide, 16 heads, a 4096 feed-forward). |
int AcousticLmFeedForward { get; set; } |
Gets or sets the acoustic language-model feed forward. |
int AcousticLmHeads { get; set; } |
Gets or sets the number of attention heads in the acoustic language-model. |
int AcousticLmLayers { get; set; } |
Gets or sets the number of layers in the acoustic language-model. |
int AcousticLmWarmupSteps { get; set; } |
Gets or sets the acoustic language-model warmup steps. |
int AcousticTopK { get; set; } |
The acoustic LM’s sampling (MusicGen’s defaults: top-k 250, temperature 1). |
int AudioTokens { get; set; } |
The audio vocabulary: the tokenizer’s 16,384 tokens plus the start and stop tokens. |
int CandidateRank { get; set; } |
The kept sequence’s rank by length, shortest first (the release keeps the second shortest, 1). |
int Candidates { get; set; } |
Sequences sampled per segment (num_return_sequences: 7). |
int CodebookDim { get; set; } |
Gets or sets the feature width of the codebook. |
int CodebookSize { get; set; } |
The speech tokenizer’s codewords per sub-codebook (two sub-codebooks make 16,384 tokens). |
int CodecCodebookDim { get; set; } |
Gets or sets the feature width of the codec codebook. |
int CodecConvGroups { get; set; } |
The residual units’ convolution groups (SoCodec: 32). |
int CodecDim { get; set; } |
The Mel codec’s width (SoCodec’s n_model_size at the speaker embedding’s width, which its decoder adds). |
int CodecStreams { get; set; } |
The streams (§3.3.2: four of 16,384 codewords, each two codebooks of CodebookSize). |
int DiscriminatorHidden { get; set; } |
Gets or sets the discriminator hidden. |
int DualDecodingDecayEnd { get; set; } |
Gets or sets the dual decoding decay end. |
int DualDecodingDecayStart { get; set; } |
Gets or sets the dual decoding decay start. |
int FlowBlocks { get; set; } |
Gets or sets the flow blocks. |
int FlowCrossAttentionBlocks { get; set; } |
The encoder layers with cross-attention to the prompt (srcattention_end_index + 1). |
int FlowDecoderBlocksPerLevel { get; set; } |
Gets or sets the flow decoder blocks per level. |
int FlowDecoderHeadDim { get; set; } |
Gets or sets the feature width of the flow decoder head. |
int FlowDecoderHeads { get; set; } |
Gets or sets the number of attention heads in the flow decoder. |
int FlowDecoderMidBlocks { get; set; } |
Gets or sets the flow decoder mid blocks. |
int FlowDim { get; set; } |
Gets or sets the feature width of the flow. |
int FlowHeads { get; set; } |
Gets or sets the number of attention heads in the flow. |
int FlowLinearUnits { get; set; } |
Gets or sets the flow linear units. |
int FlowRegulatorBlocks { get; set; } |
Gets or sets the flow regulator blocks. |
int FlowSteps { get; set; } |
The Euler steps of the flow (the release’s n_timesteps=10). |
int FlowWarmupSteps { get; set; } |
The flow’s and the acoustic LM’s warm-up steps (their recipes: CosyVoice’s 2,500 and MusicGen’s 4,000; the other recipe values are the stages’ declared optimizers). |
int HiddenDim { get; set; } |
The text-to-speech GPT hidden width. |
int HopSize { get; set; } |
The output samples per 40 ms semantic token, paired with the vocoder preset. |
int HubertDim { get; set; } |
Gets or sets the feature width of the HuBERT. |
int HubertHeads { get; set; } |
Gets or sets the number of attention heads in the HuBERT. |
int HubertIntermediate { get; set; } |
Gets or sets the HuBERT intermediate. |
int HubertLayers { get; set; } |
Gets or sets the number of layers in the HuBERT. |
int HubertPositionalGroups { get; set; } |
Gets or sets the HuBERT positional groups. |
int HubertPositionalKernel { get; set; } |
Gets or sets the HuBERT positional kernel. |
int Lookahead { get; set; } |
d in sᵢ₋d + aᵢ (unstated in the paper; 3 spans MusicGen’s delay pattern over four streams). |
int MaxAudioTokens { get; set; } |
The most semantic tokens the GPT’s positions hold (gpt_max_audio_tokens). |
int MaxNewTokens { get; set; } |
The most semantic tokens per segment (the release: the GPT’s limit, 627). |
int MaxTextLength { get; set; } |
The input text limit below the GPT’s position capacity. |
int MaxTextTokens { get; set; } |
The most text tokens the GPT’s positions hold (gpt_max_text_tokens). |
int MelChannels { get; set; } |
The flow/codec mel channels; the tokenizer’s speaker frontend separately uses 80 bins. |
int NumDecoderLayers { get; set; } |
The text-to-speech GPT depth. |
int NumHeads { get; set; } |
The text-to-speech GPT attention heads. |
int PromptEncoderBlocks { get; set; } |
Gets or sets the prompt encoder blocks. |
int PromptEncoderHidden { get; set; } |
Gets or sets the prompt encoder hidden. |
int PromptEncoderKernel { get; set; } |
Gets or sets the prompt encoder kernel. |
int PromptEncoderLayers { get; set; } |
Gets or sets the number of layers in the prompt encoder. |
int PromptEncoderReduction { get; set; } |
Gets or sets the prompt encoder reduction. |
int ReferenceHidden { get; set; } |
The speaker embedding’s projection’s hidden width (reference_embedding). |
int SampleRate { get; set; } |
The output waveform rate selected with the vocoder preset. |
int SamplingSeed { get; set; } |
The seed of the sampled tokens and the flow’s noise. |
int SpeakerAttentionChannels { get; set; } |
Gets or sets the speaker attention channels. |
int SpeakerDim { get; set; } |
The speaker embedding’s width (also the tokenizer’s model width, whose decoder adds it). |
int SpeakerRes2NetScale { get; set; } |
Gets or sets the speaker res 2 net scale. |
int SpeakerSeChannels { get; set; } |
Gets or sets the speaker squeeze-and-excitation channels. |
int SslDim { get; set; } |
The HuBERT features’ width (chinese-HuBERT-large: 1024). |
int StartTextToken { get; set; } |
The text start token ([START]). |
int StopTextToken { get; set; } |
The text stop token ([STOP]). |
int TokenizerDim { get; set; } |
The tokenizer encoder’s width (n_model_size). |
int TopK { get; set; } |
Gets or sets the maximum number of highest-probability tokens retained during sampling. |
int TrainingSeed { get; set; } |
Gets or sets the random seed used during training. |
int VocabSize { get; set; } |
The text vocabulary size. |
int VocoderInitialChannels { get; set; } |
Gets or sets the vocoder initial channels. |
int[] CodecStrides { get; set; } |
The codec’s strides: the 10 ms mel to 20 ms frames. |
int[] DiscriminatorWindows { get; set; } |
The mel discriminator’s window lengths (frames) and width (MegaTTS’s). |
int[] FlowDecoderChannels { get; set; } |
Gets or sets the flow decoder channels. |
int[] HubertConvDims { get; set; } |
Gets or sets the HuBERT conv dims. |
int[] LearningRateMilestones { get; set; } |
XTTS’s MultiStepLR milestones (×0.5 each). |
int[] SpeakerChannels { get; set; } |
The ECAPA-TDNN’s channels per stage (the last aggregates). |
int[] SpeakerDilations { get; set; } |
Gets or sets the convolution dilations for the speaker. |
int[] SpeakerKernels { get; set; } |
Gets or sets the convolution kernel sizes for the speaker. |
int[] TokenizerDownsampleStrides { get; set; } |
Gets or sets the convolution strides for the tokenizer downsample. |
int[] VocoderResBlockKernels { get; set; } |
Gets or sets the convolution kernel sizes for the vocoder res block. |
int[] VocoderUpsampleKernels { get; set; } |
Gets or sets the convolution kernel sizes for the vocoder upsample. |
int[] VocoderUpsampleRates { get; set; } |
Gets or sets the vocoder upsample rates. |
int[][] TokenizerUpsampleStrides { get; set; } |
The SSL decoder’s and the mel decoder’s upsampling strides (the release config’s upsample_scales). |
int[][] VocoderResBlockDilations { get; set; } |
Gets or sets the convolution dilations for the vocoder res block. |
string ModelPath { get; set; } |
The exported ONNX graph path, when a graph is loaded. |