SeedTTS<T>
Models & Types · AiDotNet.TextToSpeech.CodecBased
Seed-TTS’s disclosed autoregressive pipeline: speech tokenizer, causal token LM, token diffusion, and acoustic vocoder.
Inheritance: NeuralNetworkBase<T> → TtsModelBase<T> → SeedTTS<T>
Implements: ICodecTts<T>
Known derived types: SeedTTSClone<T>
How It Works
This implementation follows Figure 1 and Section 2 of the technical report. The LM predicts speech tokens while masking text positions from its loss; diffusion restores continuous acoustic detail in a coarse-to-fine process; and a separately trainable neural vocoder produces the waveform. A reference recording supplies both the speech token prefix and the timbre condition used by the paper’s self-distillation extension.
ByteDance explicitly did not release model code or weights. The native graph therefore exposes all undisclosed dimensions as options and loads only clearly labelled AiDotNet-converted flat checkpoints; it does not claim that its configurable defaults are proprietary production settings.
Constructors
| Constructor | Summary |
|---|---|
SeedTTS(NeuralNetworkArchitecture<T> architecture, SeedTTSOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates the native four-block Seed-TTS graph. |
SeedTTS(NeuralNetworkArchitecture<T> architecture, string modelPath, SeedTTSOptions options) |
Creates the native four-block Seed-TTS graph. |
Properties
| Property | Summary |
|---|---|
SeedTTSTrainingComponent TrainingComponent { get; } |
|
bool HasPaperPipeline { get; } |
Whether the native graph contains the four paper blocks rather than caller-supplied sequential layers. |
bool TrainsOnSampleObjective { get; } |
|
int CodebookSize { get; } |
|
int CodecFrameRate { get; } |
|
int CodecTokenCodebooks { get; } |
|
int CodecTokenVocabulary { get; } |
|
int MaxTextLength { get; } |
|
int NumCodebooks { get; } |
Methods
| Method | Summary |
|---|---|
LayerInputDomain GetInputDomain(int[] inputShape) |
|
ModelMetadata<T> GetModelMetadata() |
|
T EvaluateTrainingObjective(TtsTrainingSample<T> sample) |
|
Tensor<T> DecodeFromTokens(Tensor<T> tokens) |
|
Tensor<T> EncodeToTokens(Tensor<T> audio) |
|
Tensor<T> ExtractTimbreEmbedding(Tensor<T> referenceAudio) |
Extracts the timbre reference integrated into the self-distilled diffusion model. |
Tensor<T> Synthesize(string text) |
Synthesizes with the configured voice prompt when one is present, otherwise from the learned prior. |
Tensor<T> SynthesizeWithPrompt(string text, string promptText, Tensor<T> referenceAudio) |
Performs the paper’s in-context continuation from a text/audio prompt. |
void LoadConvertedWeights(string path) |
Copies {ConvertedParameterTensor} from source straight into a parameter vector. |