FishSpeechFamilyOptions

Options & Configuration · AiDotNet.TextToSpeech.CodecBased

Options shared by the Fish Speech models: the Dual-AR transformer, the Firefly-GAN codec, the text front end, decoding as the release’s API runs it, and training. Each model’s options set its release’s values.

Inheritance: ModelOptions → TtsModelOptions → FishSpeechFamilyOptions

Known derived types: FishSpeechOptions, FishSpeechV15Options

Properties

Property Summary
FishSpeechRecipe Recipe { get; set; } The recipe ApplyRecipe last set.
bool ConstrainToSemanticTokens { get; set; } Whether every generated frame is speech: the token is sampled among the semantic tokens only (1.4: and each code among those a frame can store), so decoding runs to MaxNewTokens.
bool CosineDecay { get; set; } Whether the rate decays along a cosine after warm-up (otherwise it stays constant).
bool NormalizeChineseText { get; set; } Whether text is read through the Chinese number normalizer first (the 1.4 API’s ChnNormedText, its normalize default; the 1.5 engine ignores the flag).
double Beta1 { get; set; } AdamW’s first-moment decay.
double Beta2 { get; set; } AdamW’s second-moment decay.
double DpoBeta { get; set; } Standard DPO’s β (TrainPreference).
double Epsilon { get; set; } AdamW’s epsilon.
double FinalLearningRateRatio { get; set; } The cosine’s floor as a fraction of the peak rate (final_lr_ratio).
double MaxGradientNorm { get; set; } The gradient norm is clipped to this (the release’s gradient_clip_val).
double NormEpsilon { get; set; } RMSNorm’s epsilon (norm_eps).
double ReferenceFreeDpoBeta { get; set; } The release’s reference-free DPO’s β (dpo_beta).
double RepetitionPenalty { get; set; } The repetition penalty over the last 16 frames, in (0, 2).
double RopeBase { get; set; } The rotary positions’ base (rope_base).
double Temperature { get; set; } The sampling temperature, in (0, 2).
double TopP { get; set; } Nucleus sampling’s probability mass, in (0, 1].
int ChunkLength { get; set; } The text is split into segments of at most this many UTF-8 bytes, each generated in turn (chunk_length; 0 speaks the whole text as one segment).
int CodebookSize { get; set; } Each codebook’s codes (codebook_size).
int CodecChannels { get; set; } The generator’s initial channels (upsample_initial_channel).
int CodecPostKernel { get; set; } The generator’s last convolution’s kernel.
int CodecPreKernel { get; set; } The generator’s first convolution’s kernel.
int CodecQuantizerDim { get; } The quantizer’s input width (input_dim, 512): the encoder’s last stage width (CodecEncoderDims), which the quantizer reads.
int ContextLength { get; set; } The token budget generate_long keeps earlier segments within (max_length: what remains after 1,024 tokens for the new segment and the references).
int FastLayers { get; set; } The fast transformer’s layers (n_fast_layer).
int IntermediateSize { get; set; } The SwiGLU feed-forward’s width (intermediate_size).
int KeyValueHeads { get; set; } The slow transformer’s key-value heads (n_local_heads; grouped-query attention).
int MaxNewTokens { get; set; } The most frames generated per segment (max_new_tokens; 0 fills the context).
int MaxSequenceLength { get; set; } The transformer’s context (max_seq_len).
int NumCodebooks { get; set; } The codec’s codebooks, one per quantizer group (num_codebooks).
int SamplingSeed { get; set; } The seed of decoding’s draws.
int TrainingSeed { get; set; } The seed of training’s draws (dropout).
int TrainingSteps { get; set; } The schedule’s length; with CosineDecay, the step at which the rate reaches its floor.
int WarmupSteps { get; set; } Linear warm-up steps.
int[] CodecDownsampleFactors { get; set; } The quantizer’s strided downsampling before quantization (and upsampling after).
int[] CodecEncoderDepths { get; set; } The ConvNeXt encoder’s blocks per stage.
int[] CodecEncoderDims { get; set; } The ConvNeXt encoder’s widths per stage.
int[] CodecLevels { get; set; } The finite scalar quantizer’s levels per dimension (8·5·5·5 = 1,000 codes).
int[] CodecResBlockKernels { get; set; } The kernels of the residual blocks each ParallelBlock averages.
int[] CodecUpsampleKernels { get; set; } The HiFi-GAN generator’s upsampling kernels.
int[] CodecUpsampleRates { get; set; } The HiFi-GAN generator’s upsampling strides.
int[][] CodecResBlockDilations { get; set; } The dilations of each residual block.
string TokenizerPath { get; set; } The release’s tokenizer file (1.5: tokenizer.tiktoken; 1.4: tokenizer.json).

Methods

Method Summary
void ApplyRecipe(FishSpeechRecipe recipe) Sets the optimizer’s values to recipe’s.