GraniteSpeechTrainingStage
Enums · AiDotNet.Enums
What a Granite-speech training step optimizes (Saon et al. 2025, §II-C and §II-E).
How It Works
The paper trains in two stages: the conformer encoder alone with self-conditioned CTC (EncoderCtc),
then the Q-Former projector and LoRA adapters on the frozen encoder and LLM with the transcript’s cross-entropy
(ProjectorAndLora). The second stage assumes a pretrained LLM; a model built here starts from random
weights, and a frozen random LLM cannot fit a transcript, so Joint is the default. It is NOT the
paper’s recipe.
Fields
| Field | Summary |
|---|---|
GraniteSpeechTrainingStage EncoderCtc |
§II-C: the encoder alone, with self-conditioned CTC (0.2 on the middle layer) over the CTC vocabulary; the training target holds CTC labels. |
GraniteSpeechTrainingStage Joint |
The transcript’s cross-entropy with every parameter trained. |
GraniteSpeechTrainingStage ProjectorAndLora |
§II-E: the transcript’s cross-entropy training the Q-Former projector and the LoRA adapters on the query and value projections; the encoder and the LLM’s own weights stay fixed. |
int value__ |