SpeechGptTrainingStage

Enums · AiDotNet.TextToSpeech.MultiModal

SpeechGPT’s three training stages (§4.2).

Fields

Field Summary
SpeechGptTrainingStage ChainOfModality Stage 3, chain-of-modality instruction fine-tuning: LoRA on the attention’s query and value projections, the same loss (eq.
SpeechGptTrainingStage CrossModalInstruction Stage 2, cross-modal instruction fine-tuning: ASR and TTS pairs, the loss on the response only (eq.
SpeechGptTrainingStage ModalityAdaptation Stage 1, modality-adaptation pre-training: next-token prediction on unlabelled speech’s units (eq.
int value__