Zonos<T>

Models & Types · AiDotNet.TextToSpeech.CodecBased

Zonos, Zyphra’s text-to-speech model (2025): a prefix of conditioning tokens — the text’s eSpeak phonemes, a speaker embedding, and targets for emotion, pitch variation, speaking rate, bandwidth and language — followed by DAC’s nine codebooks, delayed one step each, predicted autoregressively by a Transformer or a hybrid of Mamba2 and attention layers, under classifier-free guidance; DAC decodes the codes to 44.1 kHz audio.

Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → Zonos<T>

Implements: ICodecTts<T>

For Beginners

Zonos reads text aloud in a voice you choose from a few seconds of someone speaking, and lets you steer the emotion, pitch variation and speed.

How It Works

Zyphra released code and weights with a blog post, not a paper (Zyphra/Zonos; Zyphra/Zonos-v0.1-transformer, Zyphra/Zonos-v0.1-hybrid, Zyphra/Zonos-v0.1-speaker-embedding). The blog fixes the design: eSpeak phonemization, a prefix of conditioning embeddings, DAC tokens in Parler-TTS’s delay pattern merged into one embedding per step and predicted by nine heads, a simple autoregressive objective. The release gives the rest: the conditioners, the backbones, guidance against a text-only prefix, min-p sampling with a repetition penalty, and the end-of-audio countdown over the delayed codebooks.

Voice cloning uses the release’s speaker model (a ResNet-293 with attentive statistics pooling and an LDA to 128 dimensions) on the reference recording; see SpeakerEmbedding.

Example

var architecture = new NeuralNetworkArchitecture<float>(InputType.OneDimensional, NeuralNetworkTaskType.Regression, inputSize: 1, outputSize: 1);
var model = new Zonos<float>(architecture, new ZonosOptions());
model.LoadReferenceWeights("Zonos-v0.1-transformer/model.safetensors");
model.Codec.LoadPretrainedWeights("dac_44khz/model.safetensors");
model.LoadSpeakerModel("Zonos-v0.1-speaker-embedding");
var speaker = model.SpeakerEmbedding(recording, 44100);
var audio = model.Generate("Hello, world!", speaker);

Constructors

Constructor Summary
Zonos(NeuralNetworkArchitecture<T> architecture) Calls the fully specified overload with the omitted parameters’ defaults.
Zonos(NeuralNetworkArchitecture<T> architecture, ZonosOptions options) Creates a native, trainable Zonos.
Zonos(NeuralNetworkArchitecture<T> architecture, string modelPath, ZonosOptions options) Creates an ONNX-backed Zonos for inference.

Properties

Property Summary
DAC<T> Codec { get; } The DAC codec whose codes Zonos predicts (load descript/dac_44khz with LoadPretrainedWeights).
IReadOnlyList<string> Languages { get; } The languages Zonos accepts (eSpeak codes); a language’s id is its index.
bool TrainsOnSampleObjective { get; }
int CodebookSize { get; }
int CodecFrameRate { get; }
int CodecTokenCodebooks { get; }
int CodecTokenVocabulary { get; }
int MaxTextLength { get; }
int NumCodebooks { get; }

Methods

Method Summary
LayerInputDomain GetInputDomain(int[] inputShape)
ModelMetadata<T> GetModelMetadata()
T EvaluateTrainingObjective(TtsTrainingSample<T> sample)
Tensor<T> DecodeFromTokens(Tensor<T> tokens)
Tensor<T> EncodeToTokens(Tensor<T> audio)
Tensor<T> Generate(string text, double[] speaker, int[] audioPrefix) Speaks text (Zonos.generate) with the options’ conditioning, as speaker (a 128-value embedding from SpeakerEmbedding; null leaves it unconditional), continuing audioPrefix’s codes [frames, 9] when given; returns the audio of the…
Tensor<T> Synthesize(string text)
double[] SpeakerEmbedding(Tensor<T> audio, int sampleRate) The speaker embedding Zonos conditions on (make_speaker_embedding): the speaker model’s 128-value LDA projection of audio ([samples], or [channels, samples] averaged) at sampleRate.
void LoadReferenceWeights(string path) Loads Zonos’s weights (Zyphra/Zonos-v0.1-transformer or -hybrid): its model.safetensors, or a directory holding it.
void LoadSpeakerModel(string directory) Loads the release’s speaker model (Zyphra/Zonos-v0.1-speaker-embedding): the directory holding ResNet293_SimAM_ASP_base.pt and ResNet293_SimAM_ASP_base_LDA-128.pt.