VALLEXOptions

Options & Configuration · AiDotNet.TextToSpeech.CodecBased

Options for VALL-E X (Zhang et al. 2023, “Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modeling”).

Inheritance: ModelOptions → VallEModelOptions → VALLEXOptions

Known derived types: VALLEXCloneOptions

For Beginners

These options set VALL-E X’s sizes, codec and training. The defaults reproduce the paper; LanguagePlacement switches to the reference reproduction’s language embedding.

How It Works

VALL-E X is VALL-E (VALLEOptions) trained on two languages with a language ID. The paper states the Transformers’ sizes (§5.2: 12 layers, attention width 1024, feed-forward 4096), EnCodec with eight codebooks of 1024 codes at 75 Hz (§3.2) and the schedule (§5.2: a peak learning rate of 5e-4 after 8,000 warm-up steps, 800k steps); the rest is VALL-E’s, which the paper extends. Its phonemes come from an unreleased lexicon (BigCiDian); this model reads English through EnglishG2P and Mandarin through MandarinG2P (VALL-E X’s runnable reference, Plachtaa/VALL-E-X), in one table.

Constructors

Constructor Summary
VALLEXOptions() Creates the paper’s options.
VALLEXOptions(VALLEXOptions other) Creates a copy of other.

Properties

Property Summary
VallELanguagePlacement LanguagePlacement { get; set; } Where the language embedding is added: the AR model’s acoustic-token embeddings (the paper, §3.3, the default) or the phoneme embeddings of both models (Plachtaa/VALL-E-X).
VallEXLanguage? SynthesisLanguage { get; set; } Explicit target language for synthesis, or null to infer English/Mandarin from phonemes.
bool UseCommunityPhoneTokenizer { get; set; } Use the community’s 70-token character BPE and unframed phoneme sequence.
int LanguageCount { get; set; } Language embedding rows: two for the paper, three for the community checkpoint.

Methods

Method Summary
VALLEXOptions CommunityCheckpoint() Original Plachtaa/VALL-E-X checkpoint axes and decoding choices; this is a community reproduction.