VALLE2Options

Options & Configuration · AiDotNet.TextToSpeech.CodecBased

Options for VALL-E 2 (Chen et al. 2024, “VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers”).

Inheritance: ModelOptions → VallEModelOptions → VALLE2Options

For Beginners

These options set VALL-E 2’s sizes, how many codes it predicts per step, how it samples, and how it is trained. The defaults follow the paper.

How It Works

VALL-E 2 keeps VALL-E’s Transformers (§4.1.1: “both the AR model and the NAR models employ the same Transformer architecture in VALL-E”; VALLEOptions’s sizes), EnCodec at 6 kbps and AdamW warmed up over 32k updates with linear decay (§4.1.1), and adds grouped code modeling (§3.1) and repetition-aware sampling (§3.4.1, Algorithm 1: window K = 10, threshold 0.1). It decodes with Vocos’s released EnCodec model.

Where the paper searches or leaves a value open: the group size defaults to 2 (it trains 1, 2, 4 and 8; 1 and 2 score best, 2 at half the AR length); top-p defaults to 0 (it searches 0.0–0.8 and finds small values most robust); the NAR’s acoustic condition is at most half the utterance, otherwise a random 3–30 seconds (§4.1.1, “the maximum of half of the current utterance with a random value from 3s to 30s”); the text is espeak-style phonemes, as VALL-E’s reproduction reads them (the paper’s BPE model is not public); learned position tables hold MaxCodePositions code groups.

Constructors

Constructor Summary
VALLE2Options() Creates the paper’s options.
VALLE2Options(VALLE2Options other) Creates a copy of other.

Properties

Property Summary
bool PrependBos { get; set; } The community VALL-E-X beginning-token switch is unsupported by VALL-E 2.
bool StopOnMostLikelyEndToken { get; set; } VALL-E 2 stops only on a sampled end token, as Algorithm 1 specifies.
double MaxPromptSeconds { get; set; } Longest NAR acoustic condition in training, in seconds (30, §4.1.1).
double MinPromptSeconds { get; set; } Shortest NAR acoustic condition in training, in seconds (3, §4.1.1).
double RepetitionThreshold { get; set; } Repetition ratio above which a code is resampled at random (t_r = 0.1, §4.1.2).
double Temperature { get; set; } VALL-E 2 samples at unit temperature; other temperatures are rejected by the native constructor.
double TopP { get; set; } Nucleus-sampling mass (0, keeping the most likely code; the paper searches 0.0–0.8).
int DecoderDim { get; set; } Width of the Vocos decoder’s backbone (384, charactr/vocos-encodec-24khz).
int DecoderIntermediateDim { get; set; } The Vocos decoder’s ConvNeXt intermediate width (1152).
int DecoderLayers { get; set; } The Vocos decoder’s ConvNeXt blocks (8).
int GroupSize { get; set; } First-codebook codes per AR step (2; the paper trains 1, 2, 4 and 8, §4.1.1).
int MaxCodePositions { get; set; } Rows of the learned code-position tables (4096 frames, about 55 seconds at 75 Hz; the paper does not say).
int RepetitionWindow { get; set; } Codes over which the repetition ratio is counted (K = 10, §4.1.2).
int TopK { get; set; } VALL-E 2 uses nucleus sampling; nonzero top-k cuts are rejected.