PengiOptions

Options & Configuration · AiDotNet.Audio.Multimodal

Configuration options for the Pengi model.

Inheritance: ModelOptions → PengiOptions

For Beginners

Pengi treats all audio understanding as a conversation. Instead of having separate models for “what sound is this?” and “describe this audio”, Pengi uses one model that can answer any question about audio by generating text responses.

How It Works

Pengi (Deshmukh et al., 2023, Microsoft) is an audio language model that frames all audio tasks as text-generation tasks. It uses a frozen audio encoder (Audio Spectrogram Transformer) paired with a pre-trained language model, enabling open-ended audio reasoning, captioning, and question answering without task-specific heads.

Constructors

Constructor Summary
PengiOptions()

Properties

Property Summary
OnnxModelOptions OnnxOptions { get; set; } Gets or sets the ONNX runtime options.
double DropoutRate { get; set; } Gets or sets the dropout rate.
double LearningRate { get; set; } Gets or sets the learning rate.
double MaxAudioDurationSeconds { get; set; } Gets or sets the maximum audio duration in seconds.
int AudioEncoderDim { get; set; } Gets or sets the audio encoder dimension.
int LLMHiddenDim { get; set; } Gets or sets the LLM hidden dimension.
int MaxResponseTokens { get; set; } Gets or sets the maximum response tokens.
int NumProjectionLayers { get; set; } Gets or sets the number of projection layers from audio to LLM space.
int SampleRate { get; set; } Gets or sets the audio sample rate in Hz.
string ModelPath { get; set; } Gets or sets the path to the ONNX model file.
string Variant { get; set; } Gets or sets the model variant.