SpeechGptTrainingStage
Enums · AiDotNet.TextToSpeech.MultiModal
SpeechGPT’s three training stages (§4.2).
Fields
| Field | Summary |
|---|---|
SpeechGptTrainingStage ChainOfModality |
Stage 3, chain-of-modality instruction fine-tuning: LoRA on the attention’s query and value projections, the same loss (eq. |
SpeechGptTrainingStage CrossModalInstruction |
Stage 2, cross-modal instruction fine-tuning: ASR and TTS pairs, the loss on the response only (eq. |
SpeechGptTrainingStage ModalityAdaptation |
Stage 1, modality-adaptation pre-training: next-token prediction on unlabelled speech’s units (eq. |
int value__ |