OuteTTS<T>

Models & Types · AiDotNet.TextToSpeech.Latest

OuteTTS 1.0 (OuteAI): text-to-speech as plain language modeling — a Llama 3.2 or Qwen3 language model reads the text and continues it with each word’s duration, energy, spectral-centroid and pitch tokens and its DAC codes, which a DAC decoder turns into 24 kHz audio. A voice is a short recording’s words with their codes (a speaker profile), timed by Whisper.

Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → OuteTTS<T>

Implements: ICodecTts<T>

For Beginners

OuteTTS treats speech as text a language model can write: after the sentence to speak it writes, word by word, little tokens that describe the sound, and a codec turns them into audio. Give it a few seconds of someone speaking and it continues in that voice.

How It Works

OuteTTS has no paper; its release (edwko/OuteTTS 1.0, OuteAI/Llama-OuteTTS-1.0-1B and OuteTTS-1.0-0.6B) is the reference: the prompt format, text normalization and chunking, sampling, the DAC interface and speaker profiles are ported from it, with Whisper’s transcription and word timing from openai-whisper.

Example

var architecture = new NeuralNetworkArchitecture<float>(InputType.OneDimensional, NeuralNetworkTaskType.Regression, inputSize: 1, outputSize: 1);
var model = new OuteTTS<float>(architecture, new OuteTTSOptions());
model.LoadReferenceWeights("Llama-OuteTTS-1.0-1B");          // model.safetensors and tokenizer.json
model.LoadCodecWeights("weights_24khz_1.5kbps_v1.0.pth");      // ibm-research/DAC.speech.v1.0
model.Speaker = OuteTTSSpeaker.DefaultEnglishFemale();
var audio = model.Synthesize("Hello, how are you doing today?");

Constructors

Constructor Summary
OuteTTS(NeuralNetworkArchitecture<T> architecture) Calls the fully specified overload with the omitted parameters’ defaults.
OuteTTS(NeuralNetworkArchitecture<T> architecture, OuteTTSOptions options) Creates a native, trainable OuteTTS.
OuteTTS(NeuralNetworkArchitecture<T> architecture, string modelPath, OuteTTSOptions options) Creates an ONNX-backed OuteTTS for inference.

Properties

Property Summary
OuteTTSSpeaker Speaker { get; set; } The speaker profile synthesis continues (null: no speaker, as the release allows).
bool TrainsOnSampleObjective { get; }
int CodebookSize { get; }
int CodecFrameRate { get; } Codec frames per second (75).
int MaxTextLength { get; }
int NumCodebooks { get; }

Methods

Method Summary
LayerInputDomain GetInputDomain(int[] inputShape)
ModelMetadata<T> GetModelMetadata()
OuteTTSSpeaker CreateSpeaker(Tensor<T> audio, string transcript) A speaker profile from a recording at the model’s sample rate (the release’s create_speaker): Whisper transcribes it with word timestamps (or, given transcript, times that transcript’s words), DAC encodes it, and each word takes its fra…
T EvaluateTrainingObjective(TtsTrainingSample<T> sample)
Tensor<T> DecodeFromTokens(Tensor<T> tokens)
Tensor<T> EncodeToTokens(Tensor<T> audio)
Tensor<T> PredictFromTokenIds(Tensor<int> tokenIds) Generates audio from exact integer text token ids using the native language model.
Tensor<T> Synthesize(string text)
Tensor<int> TextToTokenIds(string text) Tokenizes text as exact integer ids, including ids not representable by Half.
void LoadAlignerWeights(string path, string alignmentHeads) Loads Whisper for speaker profiles: an openai-whisper checkpoint (its model_state_dict, such as large-v3-turbo.pt) and the alignment heads of the named release (turbo by default).
void LoadCodecWeights(string path) Loads the codec: descript-audio-codec’s checkpoint (ibm-research/DAC.speech.v1.0’s weights_24khz_1.5kbps_v1.0.pth).
void LoadReferenceWeights(string directory) Loads a released OuteTTS 1.0 model directory: model.safetensors (the Hugging Face LlamaForCausalLM or Qwen3ForCausalLM state dict) and, for Llama, its tokenizer.json.
void LoadTokenizer(string path) Reads the language model’s tokenizer from a Hugging Face tokenizer.json.