VALLEXClone<T>

Models & Types · AiDotNet.TextToSpeech.VoiceCloning

Voice-cloning facade over VALL-E X’s cross-lingual neural-codec language model.

Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → VallEModelBase<T> → VALLEX<T> → VALLEXClone<T>

Implements: IVoiceCloner<T>

How It Works

Voice cloning is the central inference task of VALL-E X, not a second network. This class inherits the exact native bilingual phoneme table, language embeddings, AR first-codebook Transformer, prompted NAR remaining-codebook Transformer and 24 kHz EnCodec pipeline from VALLEX. It only adapts that prompt API to IVoiceCloner. The reference transcript remains mandatory because Eq. 3 conditions the AR model on both the enrolled phonemes and enrolled acoustic codes.

Constructors

Constructor Summary
VALLEXClone(NeuralNetworkArchitecture<T> architecture, VALLEXCloneOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Creates the facade over the native VALL-E X paper architecture.
VALLEXClone(NeuralNetworkArchitecture<T> architecture, string modelPath, VALLEXCloneOptions options) Creates the facade over the native VALL-E X paper architecture.

Properties

Property Summary
double MinReferenceDuration { get; }
int SpeakerEmbeddingDim { get; } Width of each frame in the prompt representation.

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()
Tensor<T> ExtractSpeakerEmbedding(Tensor<T> referenceAudio) Returns VALL-E X’s actual variable-length in-context speaker representation: [frames, codebooks] EnCodec ids.
Tensor<T> SynthesizeWithVoice(string text, Tensor<T> referenceAudio)
Tensor<T> SynthesizeWithVoice(string text, string referenceText, Tensor<T> referenceAudio) Runs Eq.