OpenVoiceV2<T>
Models & Types · AiDotNet.TextToSpeech.VoiceCloning
OpenVoiceV2: OpenVoice V2: Improved Instant Voice Cloning.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → OpenVoiceV2<T>
Implements: IEndToEndTts<T>, IVoiceCloner<T>
For Beginners
OpenVoiceV2: OpenVoice V2: Improved Instant Voice Cloning.. This model converts text input into speech audio output.
How It Works
References:
- Paper: “OpenVoice V2: Improved Instant Voice Cloning” (Qin et al., 2024)
Example
// Create an OpenVoice V2 model for improved instant voice cloning
// with enhanced tone color conversion and multilingual support
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.OneDimensional,
taskType: NeuralNetworkTaskType.Regression,
inputHeight: 200, inputWidth: 1, inputDepth: 1, outputSize: 80);
// ONNX inference mode with pre-trained model
var model = new OpenVoiceV2<double>(architecture, "openvoicev2.onnx");
// Training mode with native layers
var trainModel = new OpenVoiceV2<double>(architecture, new OpenVoiceV2Options());
Constructors
| Constructor | Summary |
|---|---|
OpenVoiceV2(NeuralNetworkArchitecture<T> architecture, OpenVoiceV2Options options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
OpenVoiceV2(NeuralNetworkArchitecture<T> architecture, string modelPath, OpenVoiceV2Options options) |
Properties
| Property | Summary |
|---|---|
ITtsModel<T> BaseSynthesizer { get; set; } |
The separate native MeloTTS base voice used by the V2 release. |
bool TrainsOnSampleObjective { get; } |
|
double MinReferenceDuration { get; } |
|
int HiddenDim { get; } |
|
int MaxTextLength { get; } |
|
int NumFlowSteps { get; } |
|
int SpeakerEmbeddingDim { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
T EvaluateTrainingObjective(TtsTrainingSample<T> sample) |
|
Tensor<T> ConvertVoice(Tensor<T> audio, Tensor<T> sourceEmbedding, Tensor<T> targetEmbedding, double tau, Tensor<T> noise) |
Converts mono 22050 Hz source audio using source and target speaker embeddings. |
Tensor<T> ExtractSpeakerEmbedding(Tensor<T> referenceAudio) |
|
Tensor<T> Synthesize(string text) |
Synthesizes speech from text. |
Tensor<T> SynthesizeWithVoice(string text, Tensor<T> referenceAudio) |
|
void LoadConverterCheckpoint(string path) |
Loads the original V2 converter checkpoint, including its reference encoder. |