GraniteSpeechTrainingStage

Enums · AiDotNet.Enums

What a Granite-speech training step optimizes (Saon et al. 2025, §II-C and §II-E).

How It Works

The paper trains in two stages: the conformer encoder alone with self-conditioned CTC (EncoderCtc), then the Q-Former projector and LoRA adapters on the frozen encoder and LLM with the transcript’s cross-entropy (ProjectorAndLora). The second stage assumes a pretrained LLM; a model built here starts from random weights, and a frozen random LLM cannot fit a transcript, so Joint is the default. It is NOT the paper’s recipe.

Fields

Field Summary
GraniteSpeechTrainingStage EncoderCtc §II-C: the encoder alone, with self-conditioned CTC (0.2 on the middle layer) over the CTC vocabulary; the training target holds CTC labels.
GraniteSpeechTrainingStage Joint The transcript’s cross-entropy with every parameter trained.
GraniteSpeechTrainingStage ProjectorAndLora §II-E: the transcript’s cross-entropy training the Q-Former projector and the LoRA adapters on the query and value projections; the encoder and the LLM’s own weights stay fixed.
int value__