OpenVoice<T>
Models & Types · AiDotNet.TextToSpeech.VoiceCloning
OpenVoice: versatile instant voice cloning with decoupled tone color conversion.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → OpenVoice<T>
Implements: IEndToEndTts<T>, IVoiceCloner<T>
For Beginners
OpenVoice: versatile instant voice cloning with decoupled tone color conversion.. This model converts text input into speech audio output.
How It Works
References:
- Paper: “OpenVoice: Versatile Instant Voice Cloning” (Qin et al., 2023)
Example
// Create an OpenVoice model for versatile instant voice cloning
// with decoupled tone color conversion for flexible synthesis
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.OneDimensional,
taskType: NeuralNetworkTaskType.Regression,
inputHeight: 200, inputWidth: 1, inputDepth: 1, outputSize: 80);
// ONNX inference mode with pre-trained model
var model = new OpenVoice<double>(architecture, "openvoice.onnx");
// Training mode with native layers
var trainModel = new OpenVoice<double>(architecture, new OpenVoiceOptions());
Constructors
| Constructor | Summary |
|---|---|
OpenVoice(NeuralNetworkArchitecture<T> architecture, OpenVoiceOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
OpenVoice(NeuralNetworkArchitecture<T> architecture, string modelPath, OpenVoiceOptions options) |
Properties
| Property | Summary |
|---|---|
ITtsModel<T> BaseSynthesizer { get; set; } |
The separate base TTS of paper section 2.2; defaults to the native VITS implementation. |
bool TrainsOnSampleObjective { get; } |
|
double MinReferenceDuration { get; } |
|
int HiddenDim { get; } |
|
int MaxTextLength { get; } |
|
int NumFlowSteps { get; } |
|
int SpeakerEmbeddingDim { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
T EvaluateTrainingObjective(TtsTrainingSample<T> sample) |
|
Tensor<T> ConvertVoice(Tensor<T> audio, Tensor<T> sourceEmbedding, Tensor<T> targetEmbedding, double tau, Tensor<T> noise) |
Converts mono 22050 Hz source audio using the source and target speaker embeddings. |
Tensor<T> ExtractSpeakerEmbedding(Tensor<T> referenceAudio) |
|
Tensor<T> Synthesize(string text) |
|
Tensor<T> SynthesizeWithVoice(string text, Tensor<T> referenceAudio) |
|
void LoadConverterCheckpoint(string path) |
Loads the original converter checkpoint, including its learned reference encoder. |