Phi4AudioTrainingStage

Enums · AiDotNet.Enums

What a Phi-4-multimodal speech training step updates (Abdin et al. 2025, §2.2.2).

How It Works

The paper trains speech in two stages on a frozen Phi-4-Mini: pre-training aligns the audio encoder and projector on ASR data (EncoderAlignment), then post-training updates the projector and the speech LoRA adapters with the encoder frozen (ProjectorAndLora). Both assume a pretrained language model; a model built here starts from random weights, and a frozen random decoder cannot fit a transcript, so Joint is the default. It is NOT the paper’s recipe.

Fields

Field Summary
Phi4AudioTrainingStage EncoderAlignment §2.2.2 speech pre-training: the audio encoder and projector, with the language model frozen.
Phi4AudioTrainingStage Joint Every parameter, the language model’s included.
Phi4AudioTrainingStage ProjectorAndLora §2.2.2 speech post-training: the projector and the speech LoRA adapters, with the audio encoder and the language model frozen.
int value__