CosyVoice3<T>
Models & Types · AiDotNet.TextToSpeech.CodecBased
Fun-CosyVoice 3: Qwen2 speech-token generation, S3-v3 prompt tokens, causal DiT flow matching and CausalHiFT.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → CosyVoiceBase<T> → CosyVoice3<T>
How It Works
This implements the released Fun-CosyVoice3-0.5B-2512 topology. PyTorch state dictionaries are loaded after
conversion to safetensors; the frozen prompt tokenizer remains the release’s speech_tokenizer_v3.onnx.
Constructors
| Constructor | Summary |
|---|---|
CosyVoice3(NeuralNetworkArchitecture<T> architecture, CosyVoice3Options options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates the native released networks for training or checkpoint-backed inference. |
CosyVoice3(NeuralNetworkArchitecture<T> architecture, string modelPath, CosyVoice3Options options) |
Creates the native released networks for training or checkpoint-backed inference. |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
T EvaluateDifferentiableReward(TtsTrainingSample<T> sample, CosyVoice3<T> referencePolicy, ICosyVoiceTokenReward<T> reward, CosyVoiceDiffROOptions options) |
Evaluates DiffRO with reproducible noise without consuming the training noise stream. |
T TrainDifferentiableReward(TtsTrainingSample<T> sample, CosyVoice3<T> referencePolicy, ICosyVoiceTokenReward<T> reward, CosyVoiceDiffROOptions options, Random random) |
Runs one DiffRO policy step, equations 3-7 of the CosyVoice 3 paper. |
Tensor<T> TokensToMel(IReadOnlyList<int> tokens, Random random) |
|
int[] SpeechTokens(Tensor<T> recording, int sampleRate) |
|
int[] Tokenize(string text) |
|
string Detokenize(IEnumerable<int> ids) |
|
void LoadPretrainedWeights(string modelDirectory) |
Loads original llm/flow/hift.pt and frozen ONNX encoders, or converted safetensors components. |