Zonos<T>
Models & Types · AiDotNet.TextToSpeech.CodecBased
Zonos, Zyphra’s text-to-speech model (2025): a prefix of conditioning tokens — the text’s eSpeak phonemes, a speaker embedding, and targets for emotion, pitch variation, speaking rate, bandwidth and language — followed by DAC’s nine codebooks, delayed one step each, predicted autoregressively by a Transformer or a hybrid of Mamba2 and attention layers, under classifier-free guidance; DAC decodes the codes to 44.1 kHz audio.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → Zonos<T>
Implements: ICodecTts<T>
For Beginners
Zonos reads text aloud in a voice you choose from a few seconds of someone speaking, and lets you steer the emotion, pitch variation and speed.
How It Works
Zyphra released code and weights with a blog post, not a paper (Zyphra/Zonos; Zyphra/Zonos-v0.1-transformer,
Zyphra/Zonos-v0.1-hybrid, Zyphra/Zonos-v0.1-speaker-embedding). The blog fixes the design: eSpeak
phonemization, a prefix of conditioning embeddings, DAC tokens in Parler-TTS’s delay pattern merged into one embedding per
step and predicted by nine heads, a simple autoregressive objective. The release gives the rest: the conditioners, the
backbones, guidance against a text-only prefix, min-p sampling with a repetition penalty, and the end-of-audio countdown
over the delayed codebooks.
Voice cloning uses the release’s speaker model (a ResNet-293 with attentive statistics pooling and an LDA to 128
dimensions) on the reference recording; see SpeakerEmbedding.
Example
var architecture = new NeuralNetworkArchitecture<float>(InputType.OneDimensional, NeuralNetworkTaskType.Regression, inputSize: 1, outputSize: 1);
var model = new Zonos<float>(architecture, new ZonosOptions());
model.LoadReferenceWeights("Zonos-v0.1-transformer/model.safetensors");
model.Codec.LoadPretrainedWeights("dac_44khz/model.safetensors");
model.LoadSpeakerModel("Zonos-v0.1-speaker-embedding");
var speaker = model.SpeakerEmbedding(recording, 44100);
var audio = model.Generate("Hello, world!", speaker);
Constructors
| Constructor | Summary |
|---|---|
Zonos(NeuralNetworkArchitecture<T> architecture) |
Calls the fully specified overload with the omitted parameters’ defaults. |
Zonos(NeuralNetworkArchitecture<T> architecture, ZonosOptions options) |
Creates a native, trainable Zonos. |
Zonos(NeuralNetworkArchitecture<T> architecture, string modelPath, ZonosOptions options) |
Creates an ONNX-backed Zonos for inference. |
Properties
| Property | Summary |
|---|---|
DAC<T> Codec { get; } |
The DAC codec whose codes Zonos predicts (load descript/dac_44khz with LoadPretrainedWeights). |
IReadOnlyList<string> Languages { get; } |
The languages Zonos accepts (eSpeak codes); a language’s id is its index. |
bool TrainsOnSampleObjective { get; } |
|
int CodebookSize { get; } |
|
int CodecFrameRate { get; } |
|
int CodecTokenCodebooks { get; } |
|
int CodecTokenVocabulary { get; } |
|
int MaxTextLength { get; } |
|
int NumCodebooks { get; } |
Methods
| Method | Summary |
|---|---|
LayerInputDomain GetInputDomain(int[] inputShape) |
|
ModelMetadata<T> GetModelMetadata() |
|
T EvaluateTrainingObjective(TtsTrainingSample<T> sample) |
|
Tensor<T> DecodeFromTokens(Tensor<T> tokens) |
|
Tensor<T> EncodeToTokens(Tensor<T> audio) |
|
Tensor<T> Generate(string text, double[] speaker, int[] audioPrefix) |
Speaks text (Zonos.generate) with the options’ conditioning, as speaker (a 128-value embedding from SpeakerEmbedding; null leaves it unconditional), continuing audioPrefix’s codes [frames, 9] when given; returns the audio of the… |
Tensor<T> Synthesize(string text) |
|
double[] SpeakerEmbedding(Tensor<T> audio, int sampleRate) |
The speaker embedding Zonos conditions on (make_speaker_embedding): the speaker model’s 128-value LDA projection of audio ([samples], or [channels, samples] averaged) at sampleRate. |
void LoadReferenceWeights(string path) |
Loads Zonos’s weights (Zyphra/Zonos-v0.1-transformer or -hybrid): its model.safetensors, or a directory holding it. |
void LoadSpeakerModel(string directory) |
Loads the release’s speaker model (Zyphra/Zonos-v0.1-speaker-embedding): the directory holding ResNet293_SimAM_ASP_base.pt and ResNet293_SimAM_ASP_base_LDA-128.pt. |