OrpheusTTS<T>
Models & Types · AiDotNet.TextToSpeech.CodecBased
Orpheus (Canopy Labs, 2025): text-to-speech as plain language modeling — Llama 3.2 3B reads “{voice}: {text}”
and continues it with SNAC codes, seven tokens per frame, which SNAC’s decoder turns into 24 kHz audio.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → OrpheusTTS<T>
Implements: ICodecTts<T>
For Beginners
Orpheus treats speech as a language a chatbot-style model can write: after the sentence to speak it writes “sound tokens” one at a time, and a neural codec turns those tokens into audio.
How It Works
Orpheus has no paper; canopyai/Orpheus-TTS (release 0.1) is the reference: the prompt and the token layout, the
sampling and the streaming decoder come from its orpheus_tts package, the full decode and the training sequence
from the notebooks it links, and the recipe from finetune/train.py. SNAC (Siuzdak et al. 2024) is ported from
hubertsiuzdak/snac.
Each frame’s seven tokens are SNAC’s coarse code, the first medium code, two fine codes, the second medium code
and two more fine codes, token = 128266 + 4096 × slot + code. Emotive tags such as <laugh> and
<sigh> are written in the text.
Example
var architecture = new NeuralNetworkArchitecture<float>(InputType.OneDimensional, NeuralNetworkTaskType.Regression, inputSize: 1, outputSize: 1);
var model = new OrpheusTTS<float>(architecture, new OrpheusTTSOptions { Voice = "tara" });
model.LoadReferenceWeights("orpheus-3b-0.1-ft"); // the model.safetensors shards (and tokenizer.json)
model.LoadCodecWeights("snac_24khz/pytorch_model.bin");
var audio = model.Synthesize("Hey there, I'm Tara. <laugh> How are you doing today?");
Constructors
| Constructor | Summary |
|---|---|
OrpheusTTS(NeuralNetworkArchitecture<T> architecture) |
Calls the fully specified overload with the omitted parameters’ defaults. |
OrpheusTTS(NeuralNetworkArchitecture<T> architecture, OrpheusTTSOptions options) |
Creates a native, trainable Orpheus. |
OrpheusTTS(NeuralNetworkArchitecture<T> architecture, string modelPath, OrpheusTTSOptions options) |
Creates an ONNX-backed Orpheus for inference. |
Properties
| Property | Summary |
|---|---|
bool TrainsOnSampleObjective { get; } |
|
int CodebookSize { get; } |
|
int CodecFrameRate { get; } |
Frames per second (one coarse code and six finer ones each: about 12 at 24 kHz). |
int CodecTokenCodebooks { get; } |
|
int CodecTokenVocabulary { get; } |
|
int MaxTextLength { get; } |
|
int NumCodebooks { get; } |
Methods
| Method | Summary |
|---|---|
LayerInputDomain GetInputDomain(int[] inputShape) |
|
ModelMetadata<T> GetModelMetadata() |
|
T EvaluateTrainingObjective(TtsTrainingSample<T> sample) |
|
Tensor<T> DecodeFromTokens(Tensor<T> tokens) |
|
Tensor<T> EncodeToTokens(Tensor<T> audio) |
|
Tensor<T> Synthesize(string text) |
|
void LoadCodecWeights(string path) |
Loads SNAC: hubertsiuzdak/snac_24khz’s pytorch_model.bin. |
void LoadReferenceWeights(string directory) |
Loads a released Orpheus model directory: its *.safetensors (the Hugging Face LlamaForCausalLM state dict) and, when present, its tokenizer.json. |
void LoadTokenizer(string path) |
Reads the language model’s tokenizer from a Hugging Face tokenizer.json (a fine-tune’s own). |