KaniTTSOptions

Options & Configuration · AiDotNet.TextToSpeech.Latest

Options for KaniTTS.

Inheritance: ModelOptions → KaniTTSOptions

Known derived types: KaniTTS2Options

For Beginners

These settings size KaniTTS’s language model and its audio codec, how it picks each next sound token, and how it fine-tunes; the defaults reproduce the released models.

How It Works

KaniTTS has no paper: the defaults are its release (nineninesix-ai/kani-tts and the kani-tts 0.0.4 package; nineninesix/kani-tts-370m’s config.json, which every Kani 1 checkpoint shares). The language model is LFM2-350M (width 1024, 16 layers of which 6 attend, 16 heads over 8 key-value heads, a short-convolution kernel of 3, SwiGLU 4608 after LFM2’s adjustment of 6656, rotary base 10⁶, a head tied to its 80,539-token table: the text tokenizer’s 64,400, ten control tokens and four codebooks of 4032 audio codes). It speaks NVIDIA’s NanoCodec (nemo-nano-codec-22khz-0.6kbps-12.5fps: 22.05 kHz, 1764 samples a frame, four FSQ codebooks of 9·8·8·7 codes), sampled at temperature 1, top-k 50, top-p 0.95 and a repetition penalty of 1.1 for up to 1200 tokens. Training follows the release’s fine-tuning pipeline (KaniTTS-Finetune-pipeline): rank-stabilized LoRA of rank 16 (alpha 16, dropout 0.1) on q_proj, w1 and w2, AdamW at 5e-5 with weight decay 0.02, a cosine schedule after a 10% warm-up, the gradient norm clipped to 1 and four samples accumulated per step.

Constructors

Constructor Summary
KaniTTSOptions() Creates the released configuration.
KaniTTSOptions(KaniTTSOptions other) Copies another instance.

Properties

Property Summary
OnnxModelOptions OnnxOptions { get; set; } Options for an exported ONNX model.
bool AutoAdjustFeedForward { get; set; } LFM2’s feed-forward adjustment: 2/3 of Intermediate, times FeedForwardMultiplier, rounded up to a multiple of FeedForwardMultipleOf (on).
bool FrameLevelPositions { get; set; } KaniTTS 2’s frame-level positions: the four tokens of an audio frame share one rotary position, each frame AudioStep after the last (off for Kani 1).
bool LearnableRope { get; set; } KaniTTS 2’s learnable rotary scale per attention layer (off for Kani 1).
bool LenientSpeechSpans { get; set; } How generated tokens become audio.
bool RankStabilizedLora { get; set; } peft’s rsLoRA: updates scaled by alpha / √rank rather than alpha / rank (on).
bool RemoveRepeatedFrames { get; set; } Whether a training recording’s consecutive identical frames are kept once (the release’s remove_consecutive_duplicates, on).
bool UseLora { get; set; } Whether training adapts LoRA weights (the release’s fine-tuning) rather than every language-model weight.
bool[] AttentionLayers { get; set; } Which of the language model’s layers attend (the rest are short convolutions): LFM2-350M’s 16 layers attend at 2, 5, 8, 10, 12 and 14.
double AudioStep { get; set; } How far each audio frame’s position advances (audio_step, 1).
double Beta1 { get; set; } AdamW β₁ (0.9).
double Beta2 { get; set; } AdamW β₂ (0.999).
double Epsilon { get; set; } AdamW ε (1e-8).
double FeedForwardMultiplier { get; set; } block_ffn_dim_multiplier (1).
double LearningRate { get; set; } Initial language-model optimizer learning rate (5e-5).
double LoraAlpha { get; set; } LoRA alpha (16).
double LoraDropout { get; set; } LoRA dropout (0.1).
double MaxGradNorm { get; set; } The gradient norm clipped to (1).
double NormEpsilon { get; set; } RMSNorm ε (1e-5).
double RepetitionPenalty { get; set; } Repetition penalty over every token read so far (1.1).
double RopeAlphaLearningRateRatio { get; set; } The learnable rotary scales’ learning rate as a multiple of ModelOptions’s (alpha_lr_ratio, 1.5), with no weight decay.
double RopeAlphaMax { get; set; } The learnable rotary scale’s greatest value (alpha_max, 2).
double RopeAlphaMin { get; set; } The learnable rotary scale’s least value (alpha_min, 0.1).
double RopeTheta { get; set; } Rotary base (10⁶).
double Temperature { get; set; } Sampling temperature (1; 0 samples greedily).
double TopP { get; set; } Top-p (0.95).
double WarmupRatio { get; set; } The share of TrainingSteps the learning rate warms up over linearly (0.1).
double WeightDecay { get; set; } AdamW weight decay (0.02).
int Codebooks { get; set; } NanoCodec’s codebooks (4).
int CodecDecoderChannels { get; set; } The codec decoder’s first width (864).
int CodecEncodedDim { get; } The codec’s encoded width, fixed by the number of FSQ codebooks and dimensions per codebook.
int CodecEncoderChannels { get; set; } The codec encoder’s first width (24).
int ConvKernel { get; set; } The short convolution’s kernel (3).
int EndOfText { get; set; } The text tokenizer’s end token (2, <|endoftext|>).
int FeedForwardMultipleOf { get; set; } block_multiple_of (256).
int GradientAccumulationSteps { get; set; } Samples whose gradients make one optimizer step (4).
int Heads { get; set; } Attention heads (16).
int HiddenDim { get; set; } LFM2 hidden width (1,024).
int HopSize { get; } Codec samples per frame, derived from the encoder’s downsampling rates.
int Intermediate { get; set; } The feed-forward width as configured (6656; LFM2 builds 4608 from it).
int KeyValueHeads { get; set; } Key-value heads (8).
int LoraRank { get; set; } LoRA rank (16).
int MaxNewTokens { get; set; } Tokens generated at most (1200).
int MaxTextLength { get; set; } Maximum input text-token count (1,000).
int SampleRate { get; set; } Audio sample rate in Hz (22,050).
int SamplingSeed { get; set; } Seed of sampling and of LoRA’s dropout.
int StartOfText { get; set; } The text tokenizer’s start token (1, <|startoftext|>).
int StreamingChunkFrames { get; set; } New frames per streaming window (25).
int StreamingLookbackFrames { get; set; } Earlier frames each streaming window re-reads for context (15).
int TextVocabulary { get; set; } The text tokenizer’s size (64,400); the control tokens and the audio codes follow it.
int TopK { get; set; } Top-k (50, Hugging Face generate()’s default, which the release keeps).
int TrainingSteps { get; set; } Optimizer steps the cosine schedule spans (0: a constant rate).
int VocabSize { get; set; } Combined text, control-token and audio-code vocabulary size (80,539).
int WarmupSteps { get; set; } Linear warm-up steps (KaniTTS 2’s 2000); 0 to warm up over WarmupRatio of the steps.
int[] CodecDownsampleRates { get; set; } The codec encoder’s downsampling (2, 3, 6, 7, 7: 1764 samples a frame).
int[] CodecLevels { get; set; } The levels of each FSQ dimension of a codebook (9, 8, 8, 7: 4032 codes).
int[] CodecUpsampleRates { get; set; } The codec decoder’s upsampling (7, 7, 6, 3, 2).
string TokenizerJson { get; set; } The text tokenizer’s tokenizer.json, which LoadReferenceWeights or LoadTokenizer fills (LFM2’s tokenizer comes under its own license, so none is built in); null until then.
string[] LoraTargets { get; set; } The modules LoRA adapts (q_proj, w1, w2), by peft’s names: q_proj, k_proj, v_proj, out_proj, in_proj, w1, w2, w3.
string[] SpeakerNames { get; set; } A multi-speaker checkpoint’s speakers (config.json’s speaker_settings.speaker_list, which LoadReferenceWeights reads), indexed by a voice’s or a training sample’s SpeakerId.