Qwen2VL<T>

Models & Types · AiDotNet.VisionLanguage.InstructionTuned

Qwen2-VL: Naive Dynamic Resolution with M-RoPE for multimodal positional encoding.

Inheritance: NeuralNetworkBase<T> → VisionLanguageModelBase<T> → Qwen2VL<T>

Implements: IInstructionTunedVLM<T>

For Beginners

Qwen2-VL introduces two key innovations. First, “Naive Dynamic Resolution” lets it process images at any resolution without padding or cropping — the model adapts to each image’s natural size. Second, Multimodal RoPE (M-RoPE) extends positional encoding across three dimensions: text position, spatial location in images, and temporal position in videos. This means the model naturally understands not just what tokens say, but where they are in text, where they are in an image, and when they appear in a video. This unified positional encoding enables native video understanding alongside image comprehension. Default values follow the original paper settings.

How It Works

Qwen2-VL (Wang et al., 2024) introduces Naive Dynamic Resolution to process images at any resolution without padding or cropping. Multimodal RoPE (M-RoPE) extends positional encoding across text, spatial, and temporal dimensions, enabling native video understanding alongside image comprehension.

References:

  • Paper: “Qwen2-VL: Enhancing Vision-Language Model’s Perception of the World at Any Resolution” (Wang et al., 2024)

Example

var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.TwoDimensional,
    taskType: NeuralNetworkTaskType.ImageClassification,
    inputHeight: 224, inputWidth: 224, inputDepth: 3, outputSize: 512);
var trainModel = new Qwen2VL<double>(architecture, new Qwen2VLOptions());

Constructors

Constructor Summary
Qwen2VL(NeuralNetworkArchitecture<T> architecture, Qwen2VLOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer)
Qwen2VL(NeuralNetworkArchitecture<T> architecture, string modelPath, Qwen2VLOptions options)

Properties

Property Summary
int DecoderEmbeddingDim { get; }
int EmbeddingDimension { get; }
int MaxGenerationLength { get; }
string LanguageModelName { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()
Tensor<T> Chat(Tensor<T> image, IEnumerable<ValueTuple<string, string>> conversationHistory, string userMessage)
Tensor<T> EncodeImage(Tensor<T> image)
Tensor<T> GenerateFromImage(Tensor<T> image, string prompt) Generates text using Qwen2-VL’s dynamic-resolution + M-RoPE architecture.