MetaVoice1B<T>
Models & Types · AiDotNet.TextToSpeech.VoiceCloning
MetaVoice’s learned-position causal GPT, noncausal codebook GPT, speaker LSTM and EnCodec-conditioned four-band diffusion waveform generator.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → MetaVoice1B<T>
Implements: IEndToEndTts<T>, IVoiceCloner<T>
How It Works
The released output includes loudness normalization and DeepFilterNet3 enhancement at 48 kHz. RawDecoderOutput selects the original 24 kHz neural decoder waveform.
Constructors
| Constructor | Summary |
|---|---|
MetaVoice1B(NeuralNetworkArchitecture<T> architecture, MetaVoice1BOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Constructs the actual native transformer and speaker graphs. |
MetaVoice1B(NeuralNetworkArchitecture<T> architecture, string modelPath, MetaVoice1BOptions options) |
Constructs the actual native transformer and speaker graphs. |
Properties
| Property | Summary |
|---|---|
bool HasPaperPipeline { get; } |
Whether the actual native language and speaker graphs are present. |
bool TrainsOnSampleObjective { get; } |
|
double MinReferenceDuration { get; } |
|
int CodecTokenCodebooks { get; } |
|
int CodecTokenVocabulary { get; } |
|
int HiddenDim { get; } |
|
int MaxTextLength { get; } |
|
int NumFlowSteps { get; } |
|
int SpeakerEmbeddingDim { get; } |
Methods
| Method | Summary |
|---|---|
LayerInputDomain GetInputDomain(int[] inputShape) |
|
ModelMetadata<T> GetModelMetadata() |
|
T EvaluateTrainingObjective(TtsTrainingSample<T> sample) |
|
Tensor<T> ExtractSpeakerEmbedding(Tensor<T> referenceAudio) |
|
Tensor<T> ExtractSpeakerEmbedding(Tensor<T> referenceAudio, int sampleRate) |
Runs the original speaker frontend and LSTM, without imposing the synthesis UI’s duration threshold. |
Tensor<T> Synthesize(string text) |
|
Tensor<T> SynthesizeWithVoice(string text, Tensor<T> referenceAudio) |
|
Tensor<T> SynthesizeWithVoice(string text, Tensor<T> referenceAudio, int sampleRate) |
Clones a mono recording at an explicitly specified sample rate. |
int[] EncodeText(string text) |
Encodes normalized text using the actual first-stage checkpoint tokenizer. |
void LoadDecoderCheckpoints(string encodecSafetensors, string multibandCheckpoint) |
Loads actual HF EnCodec and all four original AudioCraft PTH bands. |
void LoadEnhancerCheckpoint(string checkpoint) |
Loads the unmodified DeepFilterNet3 model_120.ckpt.best archive. |
void LoadLanguageModelCheckpoints(string firstStage, string secondStage) |
Loads both unmodified actor PTH files, including their trained byte-BPE metadata. |
void LoadPretrainedWeights(string modelDirectory, string encodecSafetensors, string multibandCheckpoint) |
Loads the original MetaVoice actor directory and its original external decoder assets. |
void LoadPretrainedWeights(string modelDirectory, string encodecSafetensors, string multibandCheckpoint, string deepFilterCheckpoint) |
Loads all actors, the neural decoder and the release’s DeepFilterNet3 enhancer. |
void LoadSpeakerEncoderCheckpoint(string checkpoint) |
Loads all 14 active speaker tensors; the author’s two unused similarity scalars are explicitly recognized. |
void SetTrainingMode(bool isTraining) |