Chatterbox<T>

Models & Types · AiDotNet.TextToSpeech.VoiceCloning

Resemble AI’s Chatterbox: an emotion-controllable zero-shot T3 model whose speech tokens are rendered by S3Gen.

Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → CosyVoiceBase<T> → Chatterbox<T>

How It Works

This implementation follows the released English model: BPE text, voice-encoder and prompt-token conditioning, the 520M Llama-based T3 decoder, S3Tokenizer-v2, causal conditional flow matching, CAM++ and HiFT.

Like the release, every synthesized waveform carries Resemble’s Perth implicit watermark (a frozen PerthNet run natively; see Watermark and WatermarkModelPath).

Constructors

Constructor Summary
Chatterbox(NeuralNetworkArchitecture<T> architecture, ChatterboxOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer)
Chatterbox(NeuralNetworkArchitecture<T> architecture, string modelPath, ChatterboxOptions options)

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()
T EvaluateTrainingObjective(TtsTrainingSample<T> sample)
Tensor<T> Synthesize(string text)
TtsVoice<T> CreateVoice(Tensor<T> recording, int sampleRate, string transcript) Creates both sets of released conditionals: T3 reads the full voice-encoder recording and its first six seconds of S3 tokens; S3Gen reads at most ten seconds for its prompt mel, tokens and CAM++ x-vector.
int[] Tokenize(string text)
string Detokenize(IEnumerable<int> ids)
string NormalizeText(string text) Applies the release’s punctuation cleanup before BPE tokenization.
void LoadPretrainedWeights(string modelDirectory) Loads the unmodified release directory: for the English model t3_cfg.safetensors, s3gen.safetensors, ve.safetensors and tokenizer.json; for a multilingual one t3_mtl23ls_v2 (or _v3) .safetensors, s3gen.pt, ve.pt, `grapheme…
void SetTrainingMode(bool isTraining)