OrpheusTTS<T>

Models & Types · AiDotNet.TextToSpeech.CodecBased

Orpheus (Canopy Labs, 2025): text-to-speech as plain language modeling — Llama 3.2 3B reads “{voice}: {text}” and continues it with SNAC codes, seven tokens per frame, which SNAC’s decoder turns into 24 kHz audio.

Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → OrpheusTTS<T>

Implements: ICodecTts<T>

For Beginners

Orpheus treats speech as a language a chatbot-style model can write: after the sentence to speak it writes “sound tokens” one at a time, and a neural codec turns those tokens into audio.

How It Works

Orpheus has no paper; canopyai/Orpheus-TTS (release 0.1) is the reference: the prompt and the token layout, the sampling and the streaming decoder come from its orpheus_tts package, the full decode and the training sequence from the notebooks it links, and the recipe from finetune/train.py. SNAC (Siuzdak et al. 2024) is ported from hubertsiuzdak/snac.

Each frame’s seven tokens are SNAC’s coarse code, the first medium code, two fine codes, the second medium code and two more fine codes, token = 128266 + 4096 × slot + code. Emotive tags such as <laugh> and <sigh> are written in the text.

Example

var architecture = new NeuralNetworkArchitecture<float>(InputType.OneDimensional, NeuralNetworkTaskType.Regression, inputSize: 1, outputSize: 1);
var model = new OrpheusTTS<float>(architecture, new OrpheusTTSOptions { Voice = "tara" });
model.LoadReferenceWeights("orpheus-3b-0.1-ft");      // the model.safetensors shards (and tokenizer.json)
model.LoadCodecWeights("snac_24khz/pytorch_model.bin");
var audio = model.Synthesize("Hey there, I'm Tara. <laugh> How are you doing today?");

Constructors

Constructor Summary
OrpheusTTS(NeuralNetworkArchitecture<T> architecture) Calls the fully specified overload with the omitted parameters’ defaults.
OrpheusTTS(NeuralNetworkArchitecture<T> architecture, OrpheusTTSOptions options) Creates a native, trainable Orpheus.
OrpheusTTS(NeuralNetworkArchitecture<T> architecture, string modelPath, OrpheusTTSOptions options) Creates an ONNX-backed Orpheus for inference.

Properties

Property Summary
bool TrainsOnSampleObjective { get; }
int CodebookSize { get; }
int CodecFrameRate { get; } Frames per second (one coarse code and six finer ones each: about 12 at 24 kHz).
int CodecTokenCodebooks { get; }
int CodecTokenVocabulary { get; }
int MaxTextLength { get; }
int NumCodebooks { get; }

Methods

Method Summary
LayerInputDomain GetInputDomain(int[] inputShape)
ModelMetadata<T> GetModelMetadata()
T EvaluateTrainingObjective(TtsTrainingSample<T> sample)
Tensor<T> DecodeFromTokens(Tensor<T> tokens)
Tensor<T> EncodeToTokens(Tensor<T> audio)
Tensor<T> Synthesize(string text)
void LoadCodecWeights(string path) Loads SNAC: hubertsiuzdak/snac_24khz’s pytorch_model.bin.
void LoadReferenceWeights(string directory) Loads a released Orpheus model directory: its *.safetensors (the Hugging Face LlamaForCausalLM state dict) and, when present, its tokenizer.json.
void LoadTokenizer(string path) Reads the language model’s tokenizer from a Hugging Face tokenizer.json (a fine-tune’s own).