ChatTTSOptions
Options & Configuration · AiDotNet.TextToSpeech.CodecBased
Options for ChatTTS (2noise/ChatTTS, 2024), as its released configuration (ChatTTS/config/config.py)
and inference defaults (RefineTextParams, InferCodeParams) set them.
Inheritance: ModelOptions → TtsModelOptions → ChatTTSOptions
For Beginners
ChatTTS turns text, with optional laughs and pauses, into conversational speech in a voice drawn at random or given by a speaker embedding or a short recording.
How It Works
GPT: a Llama decoder (Hugging Face LlamaModel: 20 layers, width 768, 12 heads, feed-forward 3072,
RMSNorm ε 1e-6) over ChatTTS’s own embeddings: a text table of 21,178 ids (bert-base-chinese’s 21,128 plus 50
control tokens) and 4 code tables of 626 ids (625 DVAE codes and the end). Weight-normalized heads read a text
logit or 4 code logits from each position.
DVAE: 24 kHz audio to 100 log-mel bands (n_fft 1024, hop 256), a 2× downsampling convolution pair, a 12-block ConvNeXt encoder to 1024 channels, and grouped residual FSQ (levels 5·5·5·5 = 625, 2 groups, 2 residual stages): 4 codes per frame at 46.875 Hz. Its decoder maps codes back to mels; the “Decoder” (a second stack, 384 wide) maps the GPT’s hidden states to mels, which synthesis uses by default. Vocos (8 blocks, width 512) renders the mels.
No training code or paper exists; LearningRate is the library’s default, not
ChatTTS’s.
Constructors
| Constructor | Summary |
|---|---|
ChatTTSOptions() |
Creates the released configuration. |
ChatTTSOptions(ChatTTSOptions other) |
Creates a copy of other. |
Properties
| Property | Summary |
|---|---|
bool RefineText { get; set; } |
Whether synthesis first refines the text (adds pauses, laughs and fillers with the text head); the release does by default (skip_refine_text=False). |
bool UseHiddenStateDecoder { get; set; } |
Whether mels come from the GPT’s hidden states through the “Decoder” (the default, use_decoder=True) or from the codes through the DVAE. |
double CodeRepetitionPenalty { get; set; } |
Code repetition penalty over the last 16 generated frames (1.05). |
double CodeTemperature { get; set; } |
Code sampling temperature (0.3). |
double CodeTopP { get; set; } |
Code nucleus threshold (0.7, keeping at least 3). |
double RefineRepetitionPenalty { get; set; } |
Refinement repetition penalty (1.0: none). |
double RefineTemperature { get; set; } |
Refinement temperature (0.7). |
double RefineTopP { get; set; } |
Refinement nucleus threshold (0.7). |
double RmsNormEpsilon { get; set; } |
RMSNorm ε (Hugging Face LlamaConfig’s default, 1e-6, which ChatTTS keeps). |
int CodeTopK { get; set; } |
Code top-k (20, keeping at least 3). |
int DecoderBottleneck { get; set; } |
Its bottleneck width (128). |
int DecoderDim { get; } |
The hidden-state decoder’s width (384): half the GPT’s HiddenDim, since the decoder reads the GPT’s hidden states as two halves. |
int DecoderHidden { get; set; } |
Its ConvNeXt width (512). |
int DecoderLayers { get; set; } |
Its ConvNeXt blocks (12). |
int DvaeBottleneck { get; set; } |
The DVAE stacks’ bottleneck width (128). |
int DvaeDecoderHidden { get; set; } |
The DVAE decoder’s ConvNeXt width (256). |
int DvaeDim { get; set; } |
The DVAE’s width (512); its encoder emits twice this (1024). |
int DvaeEncoderHidden { get; set; } |
The DVAE encoder’s ConvNeXt width (256). |
int DvaeLayers { get; set; } |
ConvNeXt blocks in each DVAE stack (12). |
int FsqGroups { get; set; } |
FSQ groups (2). |
int FsqResiduals { get; set; } |
Residual FSQ stages per group (2). |
int IntermediateSize { get; set; } |
The Llama feed-forward width (3072). |
int MaxNewCodeTokens { get; set; } |
The most code frames per utterance (2048). |
int MaxNewRefineTokens { get; set; } |
The most refined text tokens (384). |
int NumVq { get; } |
Codes per audio frame, one code table and head each (4): the DVAE’s FsqGroups groups × FsqResiduals stages, since the GPT reads every code its quantizer emits. |
int RefineTopK { get; set; } |
Refinement top-k (20). |
int SamplingSeed { get; set; } |
The seed of synthesis’s sampling (fixed, so synthesis is deterministic). |
int VocosDim { get; set; } |
Vocos’s width (512). |
int VocosIntermediate { get; set; } |
Vocos’s ConvNeXt intermediate width (1536). |
int VocosLayers { get; set; } |
Vocos’s ConvNeXt blocks (8). |
int[] FsqLevels { get; set; } |
FSQ levels per dimension ((5, 5, 5, 5): 625 codes). |