Phi4AudioOptions
Options & Configuration · AiDotNet.SpeechRecognition.LLMIntegrated
Options for Phi-4-multimodal’s speech path: a conformer audio encoder, a projector and Phi-4-Mini.
Inheritance: ModelOptions → Phi4AudioOptions
For Beginners
Defaults are the released Phi-4-multimodal-instruct: an audio encoder of three convolutions (8× subsampling, an 80 ms token rate) and 24 conformer blocks of width 1,024 (16 heads, feed-forward 1,536) over 80-band log filterbanks; a two-layer projector to 3,072; and Phi-4-Mini (32 layers of width 3,072, 24 query and 8 key-value heads, SwiGLU 8,192, a tied 200,064-row vocabulary, rotary embeddings on three quarters of each head with LongRoPE) with the speech LoRA of rank 320 on every projection.
Constructors
| Constructor | Summary |
|---|---|
Phi4AudioOptions() |
Initializes a new instance with default values. |
Phi4AudioOptions(Phi4AudioOptions other) |
Initializes a new instance by copying from another instance. |
Properties
| Property | Summary |
|---|---|
OnnxModelOptions OnnxOptions { get; set; } |
Gets or sets the ONNX Runtime settings used when the model is loaded from an ONNX file. |
Phi4AudioTrainingStage TrainingStage { get; set; } |
What training updates: every parameter by default; the paper’s stages are encoder alignment, then the projector and the speech LoRA. |
bool UseLongRope { get; set; } |
Whether the rotary embedding uses the release’s LongRoPE factors (its 128K context extension); they apply to a head width of 128 with the default rotary share. |
double LlmRmsEpsilon { get; set; } |
Gets or sets the RMS-normalization stability constant for the language-model. |
double LlmRopeTheta { get; set; } |
Gets or sets the rotary positional-embedding frequency base for the language-model. |
double LoraAlpha { get; set; } |
Gets or sets the LoRA adapter scaling factor. |
double LoraDropout { get; set; } |
Gets or sets the dropout probability of the LoRA adapter. |
double PartialRotaryFactor { get; set; } |
The share of each head the rotary embedding turns (0.75). |
int ConvKernel { get; set; } |
The causal depthwise convolution kernel (3). |
int EncoderDim { get; set; } |
The audio encoder’s width (1,024). |
int EncoderFeedForwardDim { get; set; } |
The audio encoder’s GLU feed-forward width (1,536). |
int LlmDim { get; set; } |
Phi-4-Mini’s width (3,072). |
int LlmFeedForwardDim { get; set; } |
Phi-4-Mini’s SwiGLU width (8,192). |
int LoraRank { get; set; } |
The speech LoRA’s rank on every attention and feed-forward projection (320). |
int MaxAudioLengthSeconds { get; set; } |
The longest recording transcribed (30 seconds, the post-training limit for ASR). |
int MaxTextLength { get; set; } |
The longest transcript generated, in tokens. |
int NumAttentionHeads { get; set; } |
The audio encoder’s attention heads (16). |
int NumEncoderLayers { get; set; } |
The audio encoder’s conformer blocks (24). |
int NumLlmAttentionHeads { get; set; } |
Phi-4-Mini’s query heads (24). |
int NumLlmKvHeads { get; set; } |
Phi-4-Mini’s key-value heads (8). |
int NumLlmLayers { get; set; } |
Phi-4-Mini’s layers (32). |
int NumMels { get; set; } |
Log filterbank bands (80). |
int RelativeBiasMaxDistance { get; set; } |
The T5 relative attention bias’s largest offset (500). |
int SampleRate { get; set; } |
Gets or sets the sample rate, in Hz, that input audio is expected at. |
int SubsamplingChannels { get; set; } |
Channels of the subsampling convolutions (1,024). |
int VocabSize { get; set; } |
The vocabulary rows (200,064). |
string Instruction { get; set; } |
The ASR instruction after the audio (“Transcribe the audio clip into text.”, Appendix of the paper). |
string Language { get; set; } |
Gets or sets the language code used for speech processing. |
string ModelPath { get; set; } |
Gets or sets the path of the pretrained ONNX model. |
string[] Vocabulary { get; set; } |
Optional vocabulary mapping token ids to text when the vocabulary is not Phi’s. |