OuteTTS<T>
Models & Types · AiDotNet.TextToSpeech.Latest
OuteTTS 1.0 (OuteAI): text-to-speech as plain language modeling — a Llama 3.2 or Qwen3 language model reads the text and continues it with each word’s duration, energy, spectral-centroid and pitch tokens and its DAC codes, which a DAC decoder turns into 24 kHz audio. A voice is a short recording’s words with their codes (a speaker profile), timed by Whisper.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → OuteTTS<T>
Implements: ICodecTts<T>
For Beginners
OuteTTS treats speech as text a language model can write: after the sentence to speak it writes, word by word, little tokens that describe the sound, and a codec turns them into audio. Give it a few seconds of someone speaking and it continues in that voice.
How It Works
OuteTTS has no paper; its release (edwko/OuteTTS 1.0, OuteAI/Llama-OuteTTS-1.0-1B and OuteTTS-1.0-0.6B) is the reference: the prompt format, text normalization and chunking, sampling, the DAC interface and speaker profiles are ported from it, with Whisper’s transcription and word timing from openai-whisper.
Example
var architecture = new NeuralNetworkArchitecture<float>(InputType.OneDimensional, NeuralNetworkTaskType.Regression, inputSize: 1, outputSize: 1);
var model = new OuteTTS<float>(architecture, new OuteTTSOptions());
model.LoadReferenceWeights("Llama-OuteTTS-1.0-1B"); // model.safetensors and tokenizer.json
model.LoadCodecWeights("weights_24khz_1.5kbps_v1.0.pth"); // ibm-research/DAC.speech.v1.0
model.Speaker = OuteTTSSpeaker.DefaultEnglishFemale();
var audio = model.Synthesize("Hello, how are you doing today?");
Constructors
| Constructor | Summary |
|---|---|
OuteTTS(NeuralNetworkArchitecture<T> architecture) |
Calls the fully specified overload with the omitted parameters’ defaults. |
OuteTTS(NeuralNetworkArchitecture<T> architecture, OuteTTSOptions options) |
Creates a native, trainable OuteTTS. |
OuteTTS(NeuralNetworkArchitecture<T> architecture, string modelPath, OuteTTSOptions options) |
Creates an ONNX-backed OuteTTS for inference. |
Properties
| Property | Summary |
|---|---|
OuteTTSSpeaker Speaker { get; set; } |
The speaker profile synthesis continues (null: no speaker, as the release allows). |
bool TrainsOnSampleObjective { get; } |
|
int CodebookSize { get; } |
|
int CodecFrameRate { get; } |
Codec frames per second (75). |
int MaxTextLength { get; } |
|
int NumCodebooks { get; } |
Methods
| Method | Summary |
|---|---|
LayerInputDomain GetInputDomain(int[] inputShape) |
|
ModelMetadata<T> GetModelMetadata() |
|
OuteTTSSpeaker CreateSpeaker(Tensor<T> audio, string transcript) |
A speaker profile from a recording at the model’s sample rate (the release’s create_speaker): Whisper transcribes it with word timestamps (or, given transcript, times that transcript’s words), DAC encodes it, and each word takes its fra… |
T EvaluateTrainingObjective(TtsTrainingSample<T> sample) |
|
Tensor<T> DecodeFromTokens(Tensor<T> tokens) |
|
Tensor<T> EncodeToTokens(Tensor<T> audio) |
|
Tensor<T> PredictFromTokenIds(Tensor<int> tokenIds) |
Generates audio from exact integer text token ids using the native language model. |
Tensor<T> Synthesize(string text) |
|
Tensor<int> TextToTokenIds(string text) |
Tokenizes text as exact integer ids, including ids not representable by Half. |
void LoadAlignerWeights(string path, string alignmentHeads) |
Loads Whisper for speaker profiles: an openai-whisper checkpoint (its model_state_dict, such as large-v3-turbo.pt) and the alignment heads of the named release (turbo by default). |
void LoadCodecWeights(string path) |
Loads the codec: descript-audio-codec’s checkpoint (ibm-research/DAC.speech.v1.0’s weights_24khz_1.5kbps_v1.0.pth). |
void LoadReferenceWeights(string directory) |
Loads a released OuteTTS 1.0 model directory: model.safetensors (the Hugging Face LlamaForCausalLM or Qwen3ForCausalLM state dict) and, for Llama, its tokenizer.json. |
void LoadTokenizer(string path) |
Reads the language model’s tokenizer from a Hugging Face tokenizer.json. |