Phi4AudioOptions

Options & Configuration · AiDotNet.SpeechRecognition.LLMIntegrated

Options for Phi-4-multimodal’s speech path: a conformer audio encoder, a projector and Phi-4-Mini.

Inheritance: ModelOptions → Phi4AudioOptions

For Beginners

Defaults are the released Phi-4-multimodal-instruct: an audio encoder of three convolutions (8× subsampling, an 80 ms token rate) and 24 conformer blocks of width 1,024 (16 heads, feed-forward 1,536) over 80-band log filterbanks; a two-layer projector to 3,072; and Phi-4-Mini (32 layers of width 3,072, 24 query and 8 key-value heads, SwiGLU 8,192, a tied 200,064-row vocabulary, rotary embeddings on three quarters of each head with LongRoPE) with the speech LoRA of rank 320 on every projection.

Constructors

Constructor Summary
Phi4AudioOptions() Initializes a new instance with default values.
Phi4AudioOptions(Phi4AudioOptions other) Initializes a new instance by copying from another instance.

Properties

Property Summary
OnnxModelOptions OnnxOptions { get; set; } Gets or sets the ONNX Runtime settings used when the model is loaded from an ONNX file.
Phi4AudioTrainingStage TrainingStage { get; set; } What training updates: every parameter by default; the paper’s stages are encoder alignment, then the projector and the speech LoRA.
bool UseLongRope { get; set; } Whether the rotary embedding uses the release’s LongRoPE factors (its 128K context extension); they apply to a head width of 128 with the default rotary share.
double LlmRmsEpsilon { get; set; } Gets or sets the RMS-normalization stability constant for the language-model.
double LlmRopeTheta { get; set; } Gets or sets the rotary positional-embedding frequency base for the language-model.
double LoraAlpha { get; set; } Gets or sets the LoRA adapter scaling factor.
double LoraDropout { get; set; } Gets or sets the dropout probability of the LoRA adapter.
double PartialRotaryFactor { get; set; } The share of each head the rotary embedding turns (0.75).
int ConvKernel { get; set; } The causal depthwise convolution kernel (3).
int EncoderDim { get; set; } The audio encoder’s width (1,024).
int EncoderFeedForwardDim { get; set; } The audio encoder’s GLU feed-forward width (1,536).
int LlmDim { get; set; } Phi-4-Mini’s width (3,072).
int LlmFeedForwardDim { get; set; } Phi-4-Mini’s SwiGLU width (8,192).
int LoraRank { get; set; } The speech LoRA’s rank on every attention and feed-forward projection (320).
int MaxAudioLengthSeconds { get; set; } The longest recording transcribed (30 seconds, the post-training limit for ASR).
int MaxTextLength { get; set; } The longest transcript generated, in tokens.
int NumAttentionHeads { get; set; } The audio encoder’s attention heads (16).
int NumEncoderLayers { get; set; } The audio encoder’s conformer blocks (24).
int NumLlmAttentionHeads { get; set; } Phi-4-Mini’s query heads (24).
int NumLlmKvHeads { get; set; } Phi-4-Mini’s key-value heads (8).
int NumLlmLayers { get; set; } Phi-4-Mini’s layers (32).
int NumMels { get; set; } Log filterbank bands (80).
int RelativeBiasMaxDistance { get; set; } The T5 relative attention bias’s largest offset (500).
int SampleRate { get; set; } Gets or sets the sample rate, in Hz, that input audio is expected at.
int SubsamplingChannels { get; set; } Channels of the subsampling convolutions (1,024).
int VocabSize { get; set; } The vocabulary rows (200,064).
string Instruction { get; set; } The ASR instruction after the audio (“Transcribe the audio clip into text.”, Appendix of the paper).
string Language { get; set; } Gets or sets the language code used for speech processing.
string ModelPath { get; set; } Gets or sets the path of the pretrained ONNX model.
string[] Vocabulary { get; set; } Optional vocabulary mapping token ids to text when the vocabulary is not Phi’s.