PromptTTSOptions
Options & Configuration · AiDotNet.TextToSpeech.DescriptionBased
Options for PromptTTS (Guo et al. 2023, “PromptTTS: Controllable Text-to-Speech with Text Descriptions”).
Inheritance: ModelOptions → TtsModelOptions → AcousticModelOptions → FastSpeech2Options → PromptTTSOptions
For Beginners
These options set the sizes of PromptTTS’s networks and how its style encoder is tuned. The defaults reproduce the paper.
How It Works
The defaults are the paper’s (§4.1): a 12-layer BERT-base style encoder fine-tuned with P-tuning v2 on the five style
factors of PromptSpeech (gender 2, pitch 3, speaking speed 3, volume 3 and emotion 5 classes, §3), style and content
representations of width 256, a content encoder of 4 Transformer blocks with FastSpeech 2’s variance adaptor, and a
speech decoder with the same blocks. Everything the paper takes from FastSpeech 2 (block sizes, the variance
adaptor, the loss and the optimizer) is FastSpeech2Options’. The P-tuning v2 settings the paper does
not give are the reference code’s argument defaults (THUDM/P-tuning-v2: 4 prefix positions, no prefix projection,
dropout 0.1, Hugging Face Trainer’s AdamW at 5e-5); its own task scripts use 16 to 128 positions at 5e-3 to 2e-2,
so set them for your prompts.
Constructors
| Constructor | Summary |
|---|---|
PromptTTSOptions() |
Creates the paper’s options. |
PromptTTSOptions(PromptTTSOptions other) |
Creates a copy of other. |
Properties
| Property | Summary |
|---|---|
PromptStyleFineTuning StyleFineTuning { get; set; } |
How the style encoder is fine-tuned (P-tuning v2, §4.1). |
bool PrefixProjection { get; set; } |
Whether P-tuning v2 reparameterizes the prefixes through a two-layer MLP (off, the reference’s default). |
bool TrainStyleEncoderWithSynthesis { get; set; } |
Whether the synthesis loss also updates the style encoder (off: the paper’s ablation, where an unfine-tuned BERT loses most of the style accuracy, shows that synthesis training does not adapt it). |
double StyleDropout { get; set; } |
BERT’s dropout, also on the prefixes and before the classifiers (0.1). |
double StyleLearningRate { get; set; } |
Learning rate of the style-classification stage (5e-5, Hugging Face Trainer’s default, which the P-tuning v2 code trains with). |
int PrefixHiddenSize { get; set; } |
Hidden width of that MLP (512, the reference’s default). |
int PrefixLength { get; set; } |
P-tuning v2’s prefix positions per layer (4, the reference code’s default; the paper gives none). |
int SamplingSeed { get; set; } |
Seed of the dropout draws. |
int StyleHeads { get; set; } |
BERT’s attention heads (12). |
int StyleHiddenDim { get; set; } |
BERT’s width (768). |
int StyleIntermediateDim { get; set; } |
BERT’s feed-forward width (3072). |
int StyleLayers { get; set; } |
BERT’s layers (12). |
int StyleMaxPositions { get; set; } |
BERT’s position table (512). |
int StyleTrainingSteps { get; set; } |
Updates over which the style stage’s learning rate decays linearly to zero, as the Trainer schedules it over the run (0 keeps it constant: the run’s length depends on your data). |
int StyleVocabularySize { get; set; } |
BERT’s vocabulary (30,522: bert-base-uncased’s WordPiece table). |
int[] StyleFactorClasses { get; set; } |
Classes of each style factor the auxiliary task predicts: gender 2, pitch 3, speaking speed 3, volume 3, emotion 5 (PromptSpeech’s synthesized version, §3; the real version has no emotion). |