Qwen2Audio<T>
Models & Types · AiDotNet.Audio.Multimodal
Qwen2-Audio multimodal audio-language model for audio understanding and reasoning.
Inheritance: NeuralNetworkBase<T> → AudioNeuralNetworkBase<T> → Qwen2Audio<T>
Implements: IAudioLanguageModel<T>
For Beginners
Qwen2-Audio can “listen” to audio and answer questions about it. Play it music and ask “What genre is this?”, play it a conversation and ask “What language are they speaking?”, or play environmental sounds and ask “Describe this scene.”
Usage:
How It Works
Qwen2-Audio (Chu et al., 2024, Alibaba) uses a Whisper-style audio encoder with a Qwen2 language model backbone, connected by a perceiver-style adapter. It supports audio captioning, question answering, sound event detection, and audio reasoning.
Constructors
| Constructor | Summary |
|---|---|
Qwen2Audio(NeuralNetworkArchitecture<T> architecture, Qwen2AudioOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a Qwen2-Audio model in native training mode. |
Qwen2Audio(NeuralNetworkArchitecture<T> architecture, string modelPath, Qwen2AudioOptions options) |
Creates a Qwen2-Audio model in native training mode. |
Properties
| Property | Summary |
|---|---|
double MaxAudioDurationSeconds { get; } |
|
int MaxResponseTokens { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<string> GetCapabilities() |
|
ModelMetadata<T> GetModelMetadata() |
|
Task<string> UnderstandAsync(Tensor<T> audio, string prompt, int maxTokens, double temperature, CancellationToken cancellationToken) |
|
Tensor<T> ExtractAudioEmbeddings(Tensor<T> audio) |
|
string Caption(Tensor<T> audio, int maxTokens) |
|
string Understand(Tensor<T> audio, string prompt, int maxTokens, double temperature) |