KaniTTS2<T>
Models & Types · AiDotNet.TextToSpeech.Latest
KaniTTS 2 (nineninesix.ai, 2026): Kani’s LFM2 language model writing NVIDIA NanoCodec codes, with frame-level rotary positions, a learnable rotary scale per attention layer and a speaker embedding from a recording (WavLM-Large speaker embeddings) inserted after the first token, so it speaks in a reference speaker’s voice.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → KaniTTSBase<T> → KaniTTS2<T>
For Beginners
Give KaniTTS 2 a few seconds of someone speaking and it reads text aloud in that voice; optionally a language or accent tag picks how it pronounces the text.
How It Works
KaniTTS 2 has no paper; its release is the reference: kani_tts/model.py’s KaniTTS2ForCausalLM (positions,
learnable rotary scale, speaker insertion, generation), kani_tts/core.py (the prompt with a language tag, sampling, the
speech span), kani_tts/speaker_embedder.py, and kani-tts-2-pretrain (the data collator and trainer).
Example
var architecture = new NeuralNetworkArchitecture<float>(InputType.OneDimensional, NeuralNetworkTaskType.Regression, inputSize: 1, outputSize: 1);
var model = new KaniTTS2<float>(architecture, new KaniTTS2Options());
model.LoadReferenceWeights("kani-tts-2-en"); // model.safetensors, tokenizer.json, config.json
model.LoadCodecWeights("nemo-nano-codec-22khz-0.6kbps-12.5fps.nemo");
model.LoadSpeakerEmbedderWeights("speaker-emb-tbr"); // nineninesix/speaker-emb-tbr
model.SpeakerEmbedding = model.EmbedSpeaker(referenceRecording); // 22.05 kHz
model.LanguageTag = "en_us";
var audio = model.Synthesize("Hello, how are you doing today?");
Constructors
| Constructor | Summary |
|---|---|
KaniTTS2(NeuralNetworkArchitecture<T> architecture) |
Calls the fully specified overload with the omitted parameters’ defaults. |
KaniTTS2(NeuralNetworkArchitecture<T> architecture, KaniTTS2Options options) |
Creates a native, trainable KaniTTS 2. |
KaniTTS2(NeuralNetworkArchitecture<T> architecture, string modelPath, KaniTTS2Options options) |
Creates an ONNX-backed KaniTTS 2 for inference. |
Properties
| Property | Summary |
|---|---|
IDictionary<int, Tensor<T>> TrainingSpeakerEmbeddings { get; } |
Speaker embeddings by training sample SpeakerId — the release trains each sample on its speaker’s averaged embedding (AverageSpeakerEmbedding). |
Tensor<T> SpeakerEmbedding { get; set; } |
The speaker embedding synthesis speaks with (SpeakerEmbeddingDim values, such as EmbedSpeaker returns); null to use the voice’s reference recording, or no speaker. |
string LanguageTag { get; set; } |
The language or accent tag synthesis puts before the text (“en_us: text”); null for none. |
Methods
| Method | Summary |
|---|---|
Tensor<T> AverageSpeakerEmbedding(IEnumerable<Tensor<T>> recordings) |
The release’s per-speaker embedding (kanitts-2-dataset-pipeline’s regrouping): the plain mean of the speaker’s recordings’ embeddings. |
Tensor<T> EmbedSpeaker(Tensor<T> recording) |
The speaker embedding of a recording at the model’s sample rate (kani_tts’s SpeakerEmbedder.embed_audio: resampled to 16 kHz by torchaudio’s resampler, at most 30 seconds, through the WavLM-Large embedder). |
void LoadSpeakerEmbedderWeights(string path) |
Loads the speaker embedder: nineninesix/speaker-emb-tbr’s model.safetensors (a file, or the folder holding it). |