ParlerTTSOptions
Options & Configuration · AiDotNet.TextToSpeech.DescriptionBased
Options for Parler-TTS (Lyth & King 2024, “Natural language guidance of high-fidelity text-to-speech with synthetic annotations”).
Inheritance: ModelOptions → TtsModelOptions → ParlerTTSOptions
For Beginners
These options set the sizes of Parler-TTS’s networks, its codec and how it is trained and sampled. The defaults reproduce the released model.
How It Works
The paper fixes the design (§3): the transcript prepended to the codes of DAC at 44.1 kHz (nine codebooks, 86 frames a second) in MusicGen’s delay pattern, and the description read by cross-attention from a frozen Flan-T5 encoder. It leaves sizes and training settings open; the defaults take them from the open reproduction the paper’s code release points to (huggingface/parler-tts), as its released parler-tts-mini-v1 configures them: a 24-layer, 1024-wide decoder with 16 heads and feed-forward 4096, sinusoidal positions, dropout 0.1, Flan-T5-large as the description encoder, and training with AdamW (9.5e-4, betas 0.9 and 0.99, weight decay 0.01, gradients clipped to norm 1) held constant after 20,000 warm-up updates, the encoder frozen. Generation samples (temperature 1, top 50) for at most 2580 steps and at least 10.
Constructors
| Constructor | Summary |
|---|---|
ParlerTTSOptions() |
Creates the released parler-tts-mini-v1 configuration. |
ParlerTTSOptions(ParlerTTSOptions other) |
Creates a copy of other. |
Properties
| Property | Summary |
|---|---|
DACOptions Codec { get; set; } |
The codec (DAC 44.1 kHz at 8 kbps: nine codebooks). |
bool DoSample { get; set; } |
Whether generation samples (on) or takes the most likely code. |
bool FreezeTextEncoder { get; set; } |
Whether the description encoder is frozen in training (on, §3). |
bool PromptCrossAttention { get; set; } |
Whether the transcript is cross-attended next to the description instead of prepended to the codes (off, as the paper prepends it; the reproduction’s prompt_cross_attention). |
bool RotaryEmbeddings { get; set; } |
Rotary position embeddings in place of the added sinusoidal ones (off in the released model; the reproduction’s rope_embeddings). |
double ActivationDropout { get; set; } |
Dropout after the feed-forward activation (0). |
double AttentionDropout { get; set; } |
Dropout on the attention weights (0). |
double Beta1 { get; set; } |
AdamW β₁ (0.9). |
double Beta2 { get; set; } |
AdamW β₂ (0.99). |
double Epsilon { get; set; } |
AdamW ε (1e-8). |
double InitializerFactor { get; set; } |
Standard deviation of the decoder’s initial weights (0.02, initializer_factor). |
double LayerDrop { get; set; } |
Probability of skipping a decoder layer in training (0). |
double MaxGradientNorm { get; set; } |
Gradient-norm clipping (1; 0 turns it off). |
double RopeTheta { get; set; } |
The rotary embeddings’ base (10,000). |
double Temperature { get; set; } |
Sampling temperature (1). |
double TextEncoderDropout { get; set; } |
Dropout inside the description encoder (0.1, Flan-T5’s; active in training even when frozen, as the reproduction leaves the frozen encoder in training mode). |
double[] CodebookWeights { get; set; } |
Per-codebook loss weights (none: the mean over the codebooks). |
int AudioVocabulary { get; set; } |
Logits per codebook (1088): the codes, the end token (the codebook size) and the start token (one more), rounded up as released. |
int CodebookSize { get; set; } |
Codes per codebook (1024, the codec’s). |
int CrossAttentionKeyValueHeads { get; set; } |
Key/value heads of the decoder’s cross-attention (0: as many as KeyValueHeads). |
int FeedForwardDim { get; set; } |
Feed-forward width of the decoder (4096). |
int KeyValueHeads { get; set; } |
Key/value heads of the decoder’s self-attention (0: one per head, as released; fewer is grouped-query attention). |
int MaxLength { get; set; } |
Most decoder steps, the start column included (2580: 30 seconds of codes and the delay). |
int MinNewTokens { get; set; } |
Steps before the end token is allowed (10). |
int NumCodebooks { get; set; } |
Codebooks (9: DAC at 8 kbps). |
int PromptVocabulary { get; set; } |
Rows of the transcript embedding (32,128: Flan-T5’s vocabulary, which tokenizes the transcript). |
int SamplingSeed { get; set; } |
Seed of the training draws (dropout) and of sampling. |
int TextEncoderDim { get; set; } |
Width of the description encoder (1024); a width other than the decoder’s adds a projection. |
int TextEncoderFeedForwardDim { get; set; } |
Feed-forward width of the description encoder (2816, gated GELU). |
int TextEncoderHeads { get; set; } |
Attention heads of the description encoder (16). |
int TextEncoderKeyValueDim { get; set; } |
Width of each attention head of the description encoder (64). |
int TextEncoderLayers { get; set; } |
Layers of the description encoder (24). |
int TextEncoderVocabulary { get; set; } |
Vocabulary of the description encoder (32,128). |
int TopK { get; set; } |
Sampling top-k cut (50; 0 keeps every code). |
int WarmupSteps { get; set; } |
Linear warm-up of the learning rate, in updates (20,000), after which it stays constant. |