KaniTTSOptions
Options & Configuration · AiDotNet.TextToSpeech.Latest
Options for KaniTTS.
Inheritance: ModelOptions → KaniTTSOptions
Known derived types: KaniTTS2Options
For Beginners
These settings size KaniTTS’s language model and its audio codec, how it picks each next sound token, and how it fine-tunes; the defaults reproduce the released models.
How It Works
KaniTTS has no paper: the defaults are its release (nineninesix-ai/kani-tts and the kani-tts 0.0.4 package;
nineninesix/kani-tts-370m’s config.json, which every Kani 1 checkpoint shares). The language model is LFM2-350M
(width 1024, 16 layers of which 6 attend, 16 heads over 8 key-value heads, a short-convolution kernel of 3, SwiGLU
4608 after LFM2’s adjustment of 6656, rotary base 10⁶, a head tied to its 80,539-token table: the text tokenizer’s
64,400, ten control tokens and four codebooks of 4032 audio codes). It speaks NVIDIA’s NanoCodec
(nemo-nano-codec-22khz-0.6kbps-12.5fps: 22.05 kHz, 1764 samples a frame, four FSQ codebooks of 9·8·8·7 codes),
sampled at temperature 1, top-k 50, top-p 0.95 and a repetition penalty of 1.1 for up to 1200 tokens. Training follows
the release’s fine-tuning pipeline (KaniTTS-Finetune-pipeline): rank-stabilized LoRA of rank 16 (alpha 16, dropout 0.1)
on q_proj, w1 and w2, AdamW at 5e-5 with weight decay 0.02, a cosine schedule after a 10% warm-up, the gradient norm
clipped to 1 and four samples accumulated per step.
Constructors
| Constructor | Summary |
|---|---|
KaniTTSOptions() |
Creates the released configuration. |
KaniTTSOptions(KaniTTSOptions other) |
Copies another instance. |
Properties
| Property | Summary |
|---|---|
OnnxModelOptions OnnxOptions { get; set; } |
Options for an exported ONNX model. |
bool AutoAdjustFeedForward { get; set; } |
LFM2’s feed-forward adjustment: 2/3 of Intermediate, times FeedForwardMultiplier, rounded up to a multiple of FeedForwardMultipleOf (on). |
bool FrameLevelPositions { get; set; } |
KaniTTS 2’s frame-level positions: the four tokens of an audio frame share one rotary position, each frame AudioStep after the last (off for Kani 1). |
bool LearnableRope { get; set; } |
KaniTTS 2’s learnable rotary scale per attention layer (off for Kani 1). |
bool LenientSpeechSpans { get; set; } |
How generated tokens become audio. |
bool RankStabilizedLora { get; set; } |
peft’s rsLoRA: updates scaled by alpha / √rank rather than alpha / rank (on). |
bool RemoveRepeatedFrames { get; set; } |
Whether a training recording’s consecutive identical frames are kept once (the release’s remove_consecutive_duplicates, on). |
bool UseLora { get; set; } |
Whether training adapts LoRA weights (the release’s fine-tuning) rather than every language-model weight. |
bool[] AttentionLayers { get; set; } |
Which of the language model’s layers attend (the rest are short convolutions): LFM2-350M’s 16 layers attend at 2, 5, 8, 10, 12 and 14. |
double AudioStep { get; set; } |
How far each audio frame’s position advances (audio_step, 1). |
double Beta1 { get; set; } |
AdamW β₁ (0.9). |
double Beta2 { get; set; } |
AdamW β₂ (0.999). |
double Epsilon { get; set; } |
AdamW ε (1e-8). |
double FeedForwardMultiplier { get; set; } |
block_ffn_dim_multiplier (1). |
double LearningRate { get; set; } |
Initial language-model optimizer learning rate (5e-5). |
double LoraAlpha { get; set; } |
LoRA alpha (16). |
double LoraDropout { get; set; } |
LoRA dropout (0.1). |
double MaxGradNorm { get; set; } |
The gradient norm clipped to (1). |
double NormEpsilon { get; set; } |
RMSNorm ε (1e-5). |
double RepetitionPenalty { get; set; } |
Repetition penalty over every token read so far (1.1). |
double RopeAlphaLearningRateRatio { get; set; } |
The learnable rotary scales’ learning rate as a multiple of ModelOptions’s (alpha_lr_ratio, 1.5), with no weight decay. |
double RopeAlphaMax { get; set; } |
The learnable rotary scale’s greatest value (alpha_max, 2). |
double RopeAlphaMin { get; set; } |
The learnable rotary scale’s least value (alpha_min, 0.1). |
double RopeTheta { get; set; } |
Rotary base (10⁶). |
double Temperature { get; set; } |
Sampling temperature (1; 0 samples greedily). |
double TopP { get; set; } |
Top-p (0.95). |
double WarmupRatio { get; set; } |
The share of TrainingSteps the learning rate warms up over linearly (0.1). |
double WeightDecay { get; set; } |
AdamW weight decay (0.02). |
int Codebooks { get; set; } |
NanoCodec’s codebooks (4). |
int CodecDecoderChannels { get; set; } |
The codec decoder’s first width (864). |
int CodecEncodedDim { get; } |
The codec’s encoded width, fixed by the number of FSQ codebooks and dimensions per codebook. |
int CodecEncoderChannels { get; set; } |
The codec encoder’s first width (24). |
int ConvKernel { get; set; } |
The short convolution’s kernel (3). |
int EndOfText { get; set; } |
The text tokenizer’s end token (2, <|endoftext|>). |
int FeedForwardMultipleOf { get; set; } |
block_multiple_of (256). |
int GradientAccumulationSteps { get; set; } |
Samples whose gradients make one optimizer step (4). |
int Heads { get; set; } |
Attention heads (16). |
int HiddenDim { get; set; } |
LFM2 hidden width (1,024). |
int HopSize { get; } |
Codec samples per frame, derived from the encoder’s downsampling rates. |
int Intermediate { get; set; } |
The feed-forward width as configured (6656; LFM2 builds 4608 from it). |
int KeyValueHeads { get; set; } |
Key-value heads (8). |
int LoraRank { get; set; } |
LoRA rank (16). |
int MaxNewTokens { get; set; } |
Tokens generated at most (1200). |
int MaxTextLength { get; set; } |
Maximum input text-token count (1,000). |
int SampleRate { get; set; } |
Audio sample rate in Hz (22,050). |
int SamplingSeed { get; set; } |
Seed of sampling and of LoRA’s dropout. |
int StartOfText { get; set; } |
The text tokenizer’s start token (1, <|startoftext|>). |
int StreamingChunkFrames { get; set; } |
New frames per streaming window (25). |
int StreamingLookbackFrames { get; set; } |
Earlier frames each streaming window re-reads for context (15). |
int TextVocabulary { get; set; } |
The text tokenizer’s size (64,400); the control tokens and the audio codes follow it. |
int TopK { get; set; } |
Top-k (50, Hugging Face generate()’s default, which the release keeps). |
int TrainingSteps { get; set; } |
Optimizer steps the cosine schedule spans (0: a constant rate). |
int VocabSize { get; set; } |
Combined text, control-token and audio-code vocabulary size (80,539). |
int WarmupSteps { get; set; } |
Linear warm-up steps (KaniTTS 2’s 2000); 0 to warm up over WarmupRatio of the steps. |
int[] CodecDownsampleRates { get; set; } |
The codec encoder’s downsampling (2, 3, 6, 7, 7: 1764 samples a frame). |
int[] CodecLevels { get; set; } |
The levels of each FSQ dimension of a codebook (9, 8, 8, 7: 4032 codes). |
int[] CodecUpsampleRates { get; set; } |
The codec decoder’s upsampling (7, 7, 6, 3, 2). |
string TokenizerJson { get; set; } |
The text tokenizer’s tokenizer.json, which LoadReferenceWeights or LoadTokenizer fills (LFM2’s tokenizer comes under its own license, so none is built in); null until then. |
string[] LoraTargets { get; set; } |
The modules LoRA adapts (q_proj, w1, w2), by peft’s names: q_proj, k_proj, v_proj, out_proj, in_proj, w1, w2, w3. |
string[] SpeakerNames { get; set; } |
A multi-speaker checkpoint’s speakers (config.json’s speaker_settings.speaker_list, which LoadReferenceWeights reads), indexed by a voice’s or a training sample’s SpeakerId. |