Chatterbox<T>
Models & Types · AiDotNet.TextToSpeech.VoiceCloning
Resemble AI’s Chatterbox: an emotion-controllable zero-shot T3 model whose speech tokens are rendered by S3Gen.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → CosyVoiceBase<T> → Chatterbox<T>
How It Works
This implementation follows the released English model: BPE text, voice-encoder and prompt-token conditioning, the 520M Llama-based T3 decoder, S3Tokenizer-v2, causal conditional flow matching, CAM++ and HiFT.
Like the release, every synthesized waveform carries Resemble’s Perth implicit watermark (a frozen PerthNet run
natively; see Watermark and WatermarkModelPath).
Constructors
| Constructor | Summary |
|---|---|
Chatterbox(NeuralNetworkArchitecture<T> architecture, ChatterboxOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
Chatterbox(NeuralNetworkArchitecture<T> architecture, string modelPath, ChatterboxOptions options) |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
T EvaluateTrainingObjective(TtsTrainingSample<T> sample) |
|
Tensor<T> Synthesize(string text) |
|
TtsVoice<T> CreateVoice(Tensor<T> recording, int sampleRate, string transcript) |
Creates both sets of released conditionals: T3 reads the full voice-encoder recording and its first six seconds of S3 tokens; S3Gen reads at most ten seconds for its prompt mel, tokens and CAM++ x-vector. |
int[] Tokenize(string text) |
|
string Detokenize(IEnumerable<int> ids) |
|
string NormalizeText(string text) |
Applies the release’s punctuation cleanup before BPE tokenization. |
void LoadPretrainedWeights(string modelDirectory) |
Loads the unmodified release directory: for the English model t3_cfg.safetensors, s3gen.safetensors, ve.safetensors and tokenizer.json; for a multilingual one t3_mtl23ls_v2 (or _v3) .safetensors, s3gen.pt, ve.pt, `grapheme… |
void SetTrainingMode(bool isTraining) |