Phi4AudioTrainingStage
Enums · AiDotNet.Enums
What a Phi-4-multimodal speech training step updates (Abdin et al. 2025, §2.2.2).
How It Works
The paper trains speech in two stages on a frozen Phi-4-Mini: pre-training aligns the audio encoder and projector on
ASR data (EncoderAlignment), then post-training updates the projector and the speech LoRA adapters with
the encoder frozen (ProjectorAndLora). Both assume a pretrained language model; a model built here
starts from random weights, and a frozen random decoder cannot fit a transcript, so Joint is the
default. It is NOT the paper’s recipe.
Fields
| Field | Summary |
|---|---|
Phi4AudioTrainingStage EncoderAlignment |
§2.2.2 speech pre-training: the audio encoder and projector, with the language model frozen. |
Phi4AudioTrainingStage Joint |
Every parameter, the language model’s included. |
Phi4AudioTrainingStage ProjectorAndLora |
§2.2.2 speech post-training: the projector and the speech LoRA adapters, with the audio encoder and the language model frozen. |
int value__ |