QwenVL<T>
Models & Types · AiDotNet.VisionLanguage.InstructionTuned
Qwen-VL: visual window attention, multi-resolution, bounding box output via cross-attention resampler.
Inheritance: NeuralNetworkBase<T> → VisionLanguageModelBase<T> → QwenVL<T>
Implements: IInstructionTunedVLM<T>
For Beginners
Qwen-VL from Alibaba is a versatile model that goes beyond just describing images — it can locate objects by outputting bounding box coordinates. It uses visual window attention (processing image regions locally for efficiency) and a cross-attention resampler that compresses the large number of visual tokens into a fixed smaller set before feeding them to the language model. It supports images at multiple resolutions and can handle tasks like visual question answering, text reading (OCR), and visual grounding (finding where objects are in an image). Default values follow the original paper settings.
How It Works
Qwen-VL (Bai et al., 2023) uses a ViT vision encoder with visual window attention and a cross-attention resampler to compress visual features before feeding into the Qwen language model. It supports multi-resolution input and can output bounding box coordinates for visual grounding tasks.
References:
- Paper: “Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond” (Bai et al., 2023)
Example
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.TwoDimensional,
taskType: NeuralNetworkTaskType.ImageClassification,
inputHeight: 224, inputWidth: 224, inputDepth: 3, outputSize: 512);
var trainModel = new QwenVL<double>(architecture, new QwenVLOptions());
Constructors
| Constructor | Summary |
|---|---|
QwenVL(NeuralNetworkArchitecture<T> architecture, QwenVLOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
QwenVL(NeuralNetworkArchitecture<T> architecture, string modelPath, QwenVLOptions options) |
Properties
| Property | Summary |
|---|---|
int DecoderEmbeddingDim { get; } |
|
int EmbeddingDimension { get; } |
|
int MaxGenerationLength { get; } |
|
string LanguageModelName { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> Chat(Tensor<T> image, IEnumerable<ValueTuple<string, string>> conversationHistory, string userMessage) |
|
Tensor<T> EncodeImage(Tensor<T> image) |
|
Tensor<T> GenerateFromImage(Tensor<T> image, string prompt) |
Generates text using Qwen-VL’s window-attention + cross-attention resampler architecture. |