FireRedTTSOptions

Options & Configuration · AiDotNet.TextToSpeech.CodecBased

Options for FireRedTTS: the release’s sizes (FireRedTeam/FireRedTTS config_24k.json), decoding as its synthesize runs it, and the training recipes.

Inheritance: ModelOptions → FireRedTTSOptions

For Beginners

the defaults build the released model; load its weights with LoadReferenceWeights. Smaller values make a model to train from scratch.

Constructors

Constructor Summary
FireRedTTSOptions() Initializes a new instance with default values.

Properties

Property Summary
FireRedTtsMelDecoder MelDecoder { get; set; } The Mel decoder synthesis uses.
FireRedTtsStage Stage { get; set; } The network a training step updates.
FireRedTtsVocoder Vocoder { get; set; } The vocoder: the release’s 24 kHz or the paper’s 48 kHz.
OnnxModelOptions OnnxOptions { get; set; } Settings for the exported ONNX graph runtime.
bool ConstrainToSemanticTokens { get; set; } Keeps decoding to semantic tokens and the stop token (off: the release samples from every audio token, and a start token would fail in the flow).
bool HubertNormalizeWave { get; set; } Whether HuBERT’s input is layer-normalized (the checkpoint’s task.normalize; chinese-HuBERT-large: true).
bool InstructionTuning { get; set; } Adds the emotion and paralinguistic embeddings and the paper’s tags to the tokenizer.
bool ReleaseTextSplit { get; set; } The release’s text_split drops a last piece of 15 characters or more; false (the default) keeps it.
bool Sample { get; set; } Sample the semantic tokens (the release: true); false takes the most likely token.
double AcousticLmDropout { get; set; } Gets or sets the dropout probability for the acoustic language-model.
double AcousticTemperature { get; set; } Gets or sets the logit temperature used for acoustic-token sampling.
double AudioLossWeight { get; set; } Gets or sets the weight of the audio loss.
double Beta1 { get; set; } The GPT’s AdamW (XTTS: 0.9, 0.96, 1e-8; rate LearningRate, decay on weights).
double Beta2 { get; set; } Gets or sets the second-moment decay for the training optimizer.
double CodebookEmaDecay { get; set; } The codebooks’ EMA decay (DPQ-VAE §4.1.2: 0.9 for PQ-VAE).
double CodecAdversarialWeight { get; set; } Gets or sets the weight of the codec adversarial objective.
double CodecCommitmentWeight { get; set; } Gets or sets the weight of the codec commitment objective.
double CodecEmaDecay { get; set; } The codec’s EMA decay and commitment weight (SoCodec’s config: 0.99, 1.0).
double CodecReconstructionWeight { get; set; } The codec’s losses: L1 mel reconstruction and the LSGAN generator loss (no paper or code gives them).
double DeadCodeThreshold { get; set; } The cluster size below which a codeword is replaced (the 1S code’s threshold_ema_dead_code).
double DropoutRate { get; set; } The GPT training dropout.
double DualDecodingFinalWeight { get; set; } Dual decoding’s weight γ on the pre-quantized reconstruction (DPQ-VAE eq.
double Epsilon { get; set; } Gets or sets the numerical stability constant of the optimizer.
double FlowDropout { get; set; } Gets or sets the dropout probability for the flow.
double InferenceCfgRate { get; set; } Classifier-free guidance at inference (eq.
double LearningRate { get; set; } The text-to-speech GPT learning rate.
double LearningRateDecay { get; set; } Gets or sets the learning rate decay.
double MelLossWeight { get; set; } Gets or sets the weight of the mel loss.
double RepetitionPenalty { get; set; } Gets or sets the penalty applied to repeated tokens during generation.
double SslLossWeight { get; set; } Gets or sets the weight of the self-supervised loss.
double Temperature { get; set; } Gets or sets the logit temperature used during sampling.
double TextLossWeight { get; set; } XTTS’s loss weights: text 0.01, audio 1.
double TopP { get; set; } Gets or sets the cumulative probability threshold for nucleus sampling.
double TrainingCfgRate { get; set; } The probability training drops μ (§3.3.1: 20 %).
double VqLossWeight { get; set; } Eq.
double WeightDecay { get; set; } The text-to-speech GPT weight decay.
int AcousticLmDim { get; set; } The acoustic LM’s sizes (MusicGen-small: 24 layers, 1024 wide, 16 heads, a 4096 feed-forward).
int AcousticLmFeedForward { get; set; } Gets or sets the acoustic language-model feed forward.
int AcousticLmHeads { get; set; } Gets or sets the number of attention heads in the acoustic language-model.
int AcousticLmLayers { get; set; } Gets or sets the number of layers in the acoustic language-model.
int AcousticLmWarmupSteps { get; set; } Gets or sets the acoustic language-model warmup steps.
int AcousticTopK { get; set; } The acoustic LM’s sampling (MusicGen’s defaults: top-k 250, temperature 1).
int AudioTokens { get; set; } The audio vocabulary: the tokenizer’s 16,384 tokens plus the start and stop tokens.
int CandidateRank { get; set; } The kept sequence’s rank by length, shortest first (the release keeps the second shortest, 1).
int Candidates { get; set; } Sequences sampled per segment (num_return_sequences: 7).
int CodebookDim { get; set; } Gets or sets the feature width of the codebook.
int CodebookSize { get; set; } The speech tokenizer’s codewords per sub-codebook (two sub-codebooks make 16,384 tokens).
int CodecCodebookDim { get; set; } Gets or sets the feature width of the codec codebook.
int CodecConvGroups { get; set; } The residual units’ convolution groups (SoCodec: 32).
int CodecDim { get; set; } The Mel codec’s width (SoCodec’s n_model_size at the speaker embedding’s width, which its decoder adds).
int CodecStreams { get; set; } The streams (§3.3.2: four of 16,384 codewords, each two codebooks of CodebookSize).
int DiscriminatorHidden { get; set; } Gets or sets the discriminator hidden.
int DualDecodingDecayEnd { get; set; } Gets or sets the dual decoding decay end.
int DualDecodingDecayStart { get; set; } Gets or sets the dual decoding decay start.
int FlowBlocks { get; set; } Gets or sets the flow blocks.
int FlowCrossAttentionBlocks { get; set; } The encoder layers with cross-attention to the prompt (srcattention_end_index + 1).
int FlowDecoderBlocksPerLevel { get; set; } Gets or sets the flow decoder blocks per level.
int FlowDecoderHeadDim { get; set; } Gets or sets the feature width of the flow decoder head.
int FlowDecoderHeads { get; set; } Gets or sets the number of attention heads in the flow decoder.
int FlowDecoderMidBlocks { get; set; } Gets or sets the flow decoder mid blocks.
int FlowDim { get; set; } Gets or sets the feature width of the flow.
int FlowHeads { get; set; } Gets or sets the number of attention heads in the flow.
int FlowLinearUnits { get; set; } Gets or sets the flow linear units.
int FlowRegulatorBlocks { get; set; } Gets or sets the flow regulator blocks.
int FlowSteps { get; set; } The Euler steps of the flow (the release’s n_timesteps=10).
int FlowWarmupSteps { get; set; } The flow’s and the acoustic LM’s warm-up steps (their recipes: CosyVoice’s 2,500 and MusicGen’s 4,000; the other recipe values are the stages’ declared optimizers).
int HiddenDim { get; set; } The text-to-speech GPT hidden width.
int HopSize { get; set; } The output samples per 40 ms semantic token, paired with the vocoder preset.
int HubertDim { get; set; } Gets or sets the feature width of the HuBERT.
int HubertHeads { get; set; } Gets or sets the number of attention heads in the HuBERT.
int HubertIntermediate { get; set; } Gets or sets the HuBERT intermediate.
int HubertLayers { get; set; } Gets or sets the number of layers in the HuBERT.
int HubertPositionalGroups { get; set; } Gets or sets the HuBERT positional groups.
int HubertPositionalKernel { get; set; } Gets or sets the HuBERT positional kernel.
int Lookahead { get; set; } d in sᵢ₋d + aᵢ (unstated in the paper; 3 spans MusicGen’s delay pattern over four streams).
int MaxAudioTokens { get; set; } The most semantic tokens the GPT’s positions hold (gpt_max_audio_tokens).
int MaxNewTokens { get; set; } The most semantic tokens per segment (the release: the GPT’s limit, 627).
int MaxTextLength { get; set; } The input text limit below the GPT’s position capacity.
int MaxTextTokens { get; set; } The most text tokens the GPT’s positions hold (gpt_max_text_tokens).
int MelChannels { get; set; } The flow/codec mel channels; the tokenizer’s speaker frontend separately uses 80 bins.
int NumDecoderLayers { get; set; } The text-to-speech GPT depth.
int NumHeads { get; set; } The text-to-speech GPT attention heads.
int PromptEncoderBlocks { get; set; } Gets or sets the prompt encoder blocks.
int PromptEncoderHidden { get; set; } Gets or sets the prompt encoder hidden.
int PromptEncoderKernel { get; set; } Gets or sets the prompt encoder kernel.
int PromptEncoderLayers { get; set; } Gets or sets the number of layers in the prompt encoder.
int PromptEncoderReduction { get; set; } Gets or sets the prompt encoder reduction.
int ReferenceHidden { get; set; } The speaker embedding’s projection’s hidden width (reference_embedding).
int SampleRate { get; set; } The output waveform rate selected with the vocoder preset.
int SamplingSeed { get; set; } The seed of the sampled tokens and the flow’s noise.
int SpeakerAttentionChannels { get; set; } Gets or sets the speaker attention channels.
int SpeakerDim { get; set; } The speaker embedding’s width (also the tokenizer’s model width, whose decoder adds it).
int SpeakerRes2NetScale { get; set; } Gets or sets the speaker res 2 net scale.
int SpeakerSeChannels { get; set; } Gets or sets the speaker squeeze-and-excitation channels.
int SslDim { get; set; } The HuBERT features’ width (chinese-HuBERT-large: 1024).
int StartTextToken { get; set; } The text start token ([START]).
int StopTextToken { get; set; } The text stop token ([STOP]).
int TokenizerDim { get; set; } The tokenizer encoder’s width (n_model_size).
int TopK { get; set; } Gets or sets the maximum number of highest-probability tokens retained during sampling.
int TrainingSeed { get; set; } Gets or sets the random seed used during training.
int VocabSize { get; set; } The text vocabulary size.
int VocoderInitialChannels { get; set; } Gets or sets the vocoder initial channels.
int[] CodecStrides { get; set; } The codec’s strides: the 10 ms mel to 20 ms frames.
int[] DiscriminatorWindows { get; set; } The mel discriminator’s window lengths (frames) and width (MegaTTS’s).
int[] FlowDecoderChannels { get; set; } Gets or sets the flow decoder channels.
int[] HubertConvDims { get; set; } Gets or sets the HuBERT conv dims.
int[] LearningRateMilestones { get; set; } XTTS’s MultiStepLR milestones (×0.5 each).
int[] SpeakerChannels { get; set; } The ECAPA-TDNN’s channels per stage (the last aggregates).
int[] SpeakerDilations { get; set; } Gets or sets the convolution dilations for the speaker.
int[] SpeakerKernels { get; set; } Gets or sets the convolution kernel sizes for the speaker.
int[] TokenizerDownsampleStrides { get; set; } Gets or sets the convolution strides for the tokenizer downsample.
int[] VocoderResBlockKernels { get; set; } Gets or sets the convolution kernel sizes for the vocoder res block.
int[] VocoderUpsampleKernels { get; set; } Gets or sets the convolution kernel sizes for the vocoder upsample.
int[] VocoderUpsampleRates { get; set; } Gets or sets the vocoder upsample rates.
int[][] TokenizerUpsampleStrides { get; set; } The SSL decoder’s and the mel decoder’s upsampling strides (the release config’s upsample_scales).
int[][] VocoderResBlockDilations { get; set; } Gets or sets the convolution dilations for the vocoder res block.
string ModelPath { get; set; } The exported ONNX graph path, when a graph is loaded.