GraniteSpeechOptions

Options & Configuration · AiDotNet.SpeechRecognition.LLMIntegrated

Options for Granite-speech: a conformer CTC encoder, a window Q-Former projector and a Granite LLM.

Inheritance: ModelOptions → GraniteSpeechOptions

For Beginners

Defaults are the released granite-speech-3.3-8b: a 16-block conformer encoder of width 1,024 (8 heads of 128, attention in blocks of 200 frames, kernel 15) over pairs of stacked 80-band log-mel frames; a 2-layer Q-Former that reads each window of 15 encoder frames with 3 queries; and granite-3.3-8b-instruct (40 layers of width 4,096, 32 query and 8 key-value heads, SwiGLU 12,800, a tied 49,160-row vocabulary) with LoRA of rank 64 on the query and value projections. granite-speech-3.3-2b differs only in its LLM: width 2,048, SwiGLU 8,192, attention multiplier 1/64 and logits scaling 8.

Constructors

Constructor Summary
GraniteSpeechOptions() Initializes a new instance with default values.
GraniteSpeechOptions(GraniteSpeechOptions other) Initializes a new instance by copying from another instance.

Properties

Property Summary
GraniteSpeechTrainingStage TrainingStage { get; set; } What training optimizes: every parameter on the transcript by default; the paper’s stages are the encoder’s CTC and then the projector and LoRA adapters.
OnnxModelOptions OnnxOptions { get; set; } Gets or sets the ONNX Runtime settings used when the model is loaded from an ONNX file.
double AttentionMultiplier { get; set; } Granite’s attention score scale (1/128 for the 8B model, 1/64 for the 2B).
double EmbeddingMultiplier { get; set; } Granite’s scale on the input embeddings (12).
double LlmRmsEpsilon { get; set; } Gets or sets the RMS-normalization stability constant for the language-model.
double LlmRopeTheta { get; set; } Gets or sets the rotary positional-embedding frequency base for the language-model.
double LogitsScaling { get; set; } Granite’s divisor of the logits (16 for the 8B model, 8 for the 2B).
double LoraAlpha { get; set; } Gets or sets the LoRA adapter scaling factor.
double LoraDropout { get; set; } Gets or sets the dropout probability of the LoRA adapter.
double ResidualMultiplier { get; set; } Granite’s scale on each block’s output before the residual sum (0.22).
int AttentionContextSize { get; set; } Encoder frames per attention block (200, 4 seconds).
int ConvExpansionFactor { get; set; } The convolution module’s inner width as a multiple of the encoder width (2).
int ConvKernel { get; set; } The depthwise convolution kernel (15).
int CtcVocabSize { get; set; } The CTC outputs, blank included (256 in the release; the paper’s character set has 42).
int EncoderDim { get; set; } The encoder’s width (1,024).
int EncoderHeadDim { get; set; } The encoder’s head width (128).
int FeedForwardMultiplier { get; set; } The encoder feed-forward width as a multiple of its width (4).
int LlmDim { get; set; } granite-3.3-8b’s width (4,096).
int LlmFeedForwardDim { get; set; } granite-3.3-8b’s SwiGLU width (12,800).
int LoraRank { get; set; } The LoRA rank on the query and value projections (64, the release’s adapter).
int MaxAudioLengthSeconds { get; set; } The longest recording transcribed (30 seconds).
int MaxRelativePosition { get; set; } The largest relative position the Shaw embedding table holds (512).
int MaxTextLength { get; set; } The longest transcript generated, in tokens.
int NumAttentionHeads { get; set; } The encoder’s attention heads (8).
int NumEncoderLayers { get; set; } The encoder’s conformer blocks (16 in the release; Table II of the paper lists 10).
int NumLlmAttentionHeads { get; set; } granite-3.3-8b’s query heads (32).
int NumLlmKvHeads { get; set; } granite-3.3-8b’s key-value heads (8).
int NumLlmLayers { get; set; } granite-3.3-8b’s layers (40).
int NumMels { get; set; } Log-mel bands (80); the encoder reads two stacked frames, 160 values.
int NumProjectorHeads { get; set; } The Q-Former’s attention heads (16).
int NumProjectorLayers { get; set; } The Q-Former’s layers (2), each with self-attention, cross-attention and a feed-forward block.
int ProjectorDim { get; set; } The Q-Former’s width (1,024).
int ProjectorDownsampleRate { get; set; } The projector’s temporal downsampling K / N (5, so 3 queries per window).
int ProjectorFeedForwardDim { get; set; } The Q-Former’s feed-forward width (4,096).
int ProjectorWindowSize { get; set; } Encoder frames per Q-Former window, K (15).
int SampleRate { get; set; } Gets or sets the sample rate, in Hz, that input audio is expected at.
int VocabSize { get; set; } The vocabulary rows (49,160).
int WarmupSteps { get; set; } Learning-rate warmup steps of the transcript stages’ default optimizer (1,000, §4).
string Instruction { get; set; } The user’s request after the audio (the model card’s).
string Language { get; set; } Gets or sets the language code used for speech processing.
string ModelPath { get; set; } Gets or sets the path to the model weights.
string SystemPrompt { get; set; } The system turn (the model card’s).
string[] Vocabulary { get; set; } Optional vocabulary mapping token ids to text when the vocabulary is not Granite’s.