TtsSupervision

Enums · AiDotNet.TextToSpeech

Training supervision a text-to-speech paper takes from outside the recording, which a token/mel pair does not carry and the model cannot derive itself.

Fields

Field Summary
TtsSupervision CodecTokens The recording’s discrete codec tokens, [frames, codebooks] (or the recording, which the model encodes with its codec), for models that generate codec tokens rather than a spectrogram (Pheme’s SpeechTokenizer codes).
TtsSupervision Description A natural-language description of how the utterance sounds, as the description encoder’s token ids (Parler-TTS cross-attends to the Flan-T5 encoding of it, Lyth & King 2024 §3).
TtsSupervision Durations Per-token durations from a forced aligner (e.g.
TtsSupervision LanguageId The language’s index in a multilingual model’s language table (YourTTS, Casanova et al.
TtsSupervision None The model trains on text and its target spectrogram alone.
TtsSupervision PromptCodecTokens The codec tokens of another utterance of the same speaker, [frames, codebooks], which the model reads as an acoustic prompt in training (VALL-E X’s NAR model reads the previous sentence of the same speaker, Zhang et al.
TtsSupervision Recording The recording itself (or its linear spectrogram), beyond the mel spectrogram: Tacotron’s post-processing network predicts the linear-frequency spectrogram (Wang et al.
TtsSupervision ReferenceRecording A reference recording as a waveform, for models whose speaker encoder reads audio rather than the model’s own spectrogram (YourTTS’s H/ASP speaker encoder).
TtsSupervision SpeakerId The speaker’s index in a multi-speaker model’s speaker table.
TtsSupervision SpeakerReference A reference recording of the target speaker or style.
TtsSupervision Text The utterance’s text (Text), for a model that runs its own front end and text encoders on the sentence in training (MeloTTS).
int value__