VALLEOptions
Options & Configuration · AiDotNet.TextToSpeech.CodecBased
Options for VALL-E (Wang et al. 2023, “Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers”).
Inheritance: ModelOptions → VallEModelOptions → VALLEOptions
For Beginners
These options set the sizes of VALL-E’s two networks, its codec and how it is trained and sampled. The defaults reproduce the paper.
How It Works
The defaults are the paper’s (§5.1): EnCodec at 6 kbps (eight codebooks of 1024 codes, 75 frames a second, 24 kHz),
an AR and a NAR Transformer each of 12 layers, 16 heads, width 1024, feed-forward 4096 and dropout 0.1, trained with
AdamW warmed up over 32,000 updates to 5e-4 and decayed linearly over 800,000 updates; the NAR’s training prompt is
a random 3-second segment of the same utterance. The paper converts text with a Kaldi ASR model’s phoneme
alignments; this model reads espeak-style phonemes (EnglishG2P), as the reference
reproduction (lifeiteng/vall-e) does, and takes the rest of what the paper leaves open from it: the phoneme table
(512 embedding rows), sinusoidal positions with a learned scale in the AR model, pre-norm layers, the NAR heads
sharing the acoustic embeddings, and sampling with temperature 1 and no top-k cut.
Constructors
| Constructor | Summary |
|---|---|
VALLEOptions() |
Creates the paper’s options. |
VALLEOptions(VALLEOptions other) |
Creates a copy of other. |
Properties
| Property | Summary |
|---|---|
double PromptSeconds { get; set; } |
Length of the NAR’s training prompt, in seconds (3, §5.1). |
Methods
| Method | Summary |
|---|---|
EnCodecOptions CreatePaperCodec() |
EnCodec’s released 24 kHz model at 6 kbps: eight codebooks. |