SparkTTS<T>
Models & Types · AiDotNet.TextToSpeech.CodecBased
Spark-TTS: Qwen2.5 generates a single semantic speech-token stream which BiCodec renders with decoupled speaker tokens.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → SparkTTS<T>
Implements: ICodecTts<T>
How It Works
The native graph follows the official 0.5B release: XLSR hidden-state mixing, a Vocos semantic encoder and 8,192-way factorized VQ, an ECAPA/Perceiver/FSQ speaker tokenizer, Qwen2.5-0.5B, a d-vector-conditioned Vocos prenet, and the released four-stage DAC WaveGenerator. Voice cloning uses the official 32-global-token prompt grammar; controllable synthesis asks the LM to generate those global tokens from gender, pitch and speed labels.
Constructors
| Constructor | Summary |
|---|---|
SparkTTS(NeuralNetworkArchitecture<T> architecture, SparkTTSOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates the native official-release graph. |
SparkTTS(NeuralNetworkArchitecture<T> architecture, string modelPath, SparkTTSOptions options) |
Creates the native official-release graph. |
Properties
| Property | Summary |
|---|---|
bool HasPaperPipeline { get; } |
Whether this instance owns the official Qwen/BiCodec component graph. |
bool TrainsOnSampleObjective { get; } |
|
int CodebookSize { get; } |
|
int CodecFrameRate { get; } |
|
int CodecTokenCodebooks { get; } |
|
int CodecTokenVocabulary { get; } |
|
int MaxTextLength { get; } |
|
int NumCodebooks { get; } |
Methods
| Method | Summary |
|---|---|
LayerInputDomain GetInputDomain(int[] inputShape) |
|
ModelMetadata<T> GetModelMetadata() |
|
T EvaluateTrainingObjective(TtsTrainingSample<T> sample) |
|
T TrainAnnotatedSample(TtsTrainingSample<T> sample, bool female, SparkTTSControlLevel pitch, SparkTTSControlLevel speed, int pitchMel, int syllablesPerSecond) |
Trains both zero-shot and controllable samples for one annotated recording (paper Eq. |
Tensor<T> DecodeFromTokens(Tensor<T> semanticTokens, Tensor<T> globalTokens) |
Decodes semantic speech tokens with an explicit 32-token speaker prompt. |
Tensor<T> DecodeFromTokens(Tensor<T> tokens) |
|
Tensor<T> EncodeGlobalTokens(Tensor<T> referenceAudio) |
Extracts the 32 released BiCodec global speaker-token ids. |
Tensor<T> EncodeToTokens(Tensor<T> audio) |
|
Tensor<T> Synthesize(string text) |
Synthesizes with a configured voice prompt, or with the paper’s neutral controllable-speaker path. |
Tensor<T> SynthesizeControlled(string text, bool female, SparkTTSControlLevel pitch, SparkTTSControlLevel speed) |
Runs the official gender/pitch/speed controllable-speaker prompt and its global-token chain of thought. |
Tensor<T> SynthesizeWithPrompt(string text, string promptText, Tensor<T> referenceAudio) |
Runs the released zero-shot voice-cloning prompt format. |
void LoadAuthorWeights(string languageModelPath, string biCodecPath, string wav2VecPath) |
Loads all three original release components. |
void LoadConvertedWeights(string path) |
Copies {ConvertedParameterTensor} from source straight into a parameter vector. |
void LoadLanguageModelWeights(string path) |
Loads the original Hugging Face Qwen2.5 state dictionary, including its tied output embedding. |
void LoadSpeakerTokenizerWeights(string path, string prefix) |
Loads the speaker_encoder subtree of the released BiCodec checkpoint, including ECAPA running statistics. |