VALLEXClone<T>
Models & Types · AiDotNet.TextToSpeech.VoiceCloning
Voice-cloning facade over VALL-E X’s cross-lingual neural-codec language model.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → VallEModelBase<T> → VALLEX<T> → VALLEXClone<T>
Implements: IVoiceCloner<T>
How It Works
Voice cloning is the central inference task of VALL-E X, not a second network. This class inherits the exact native
bilingual phoneme table, language embeddings, AR first-codebook Transformer, prompted NAR remaining-codebook
Transformer and 24 kHz EnCodec pipeline from VALLEX. It only adapts that prompt API to
IVoiceCloner. The reference transcript remains mandatory because Eq. 3 conditions the AR model on
both the enrolled phonemes and enrolled acoustic codes.
Constructors
| Constructor | Summary |
|---|---|
VALLEXClone(NeuralNetworkArchitecture<T> architecture, VALLEXCloneOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates the facade over the native VALL-E X paper architecture. |
VALLEXClone(NeuralNetworkArchitecture<T> architecture, string modelPath, VALLEXCloneOptions options) |
Creates the facade over the native VALL-E X paper architecture. |
Properties
| Property | Summary |
|---|---|
double MinReferenceDuration { get; } |
|
int SpeakerEmbeddingDim { get; } |
Width of each frame in the prompt representation. |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> ExtractSpeakerEmbedding(Tensor<T> referenceAudio) |
Returns VALL-E X’s actual variable-length in-context speaker representation: [frames, codebooks] EnCodec ids. |
Tensor<T> SynthesizeWithVoice(string text, Tensor<T> referenceAudio) |
|
Tensor<T> SynthesizeWithVoice(string text, string referenceText, Tensor<T> referenceAudio) |
Runs Eq. |