AquilaVL<T>
Models & Types · AiDotNet.VisionLanguage.InstructionTuned
AquilaVL: bilingual VLM aligned with Chinese and English visual-language understanding.
Inheritance: NeuralNetworkBase<T> → VisionLanguageModelBase<T> → AquilaVL<T>
Implements: IInstructionTunedVLM<T>
For Beginners
AquilaVL is a bilingual (Chinese/English) model that can look at an image and answer questions about it in either language. It uses a SigLIP vision encoder to understand what is in the image, projects those visual features through an MLP into the Qwen2.5 language model, and generates text responses. It handles images at their native resolution by splitting them into tiles, so it can understand both the big picture and fine details. Default values follow the original paper settings.
How It Works
AquilaVL (BAAI, 2024) is a bilingual vision-language model designed for strong performance in both Chinese and English visual understanding. It follows a LLaVA-style architecture using a SigLIP vision encoder to extract dense visual features, a 2-layer MLP projector with SiLU activation to map visual tokens into the language model’s embedding space, and a Qwen2.5 decoder backbone for text generation. AquilaVL supports dynamic resolution by tiling images and encoding each tile independently, enabling high-fidelity understanding of detailed scenes.
References:
- Paper: “AquilaVL: Advanced Vision-Language Model” (BAAI, 2024)
Example
// Create an AquilaVL model for bilingual Chinese/English visual QA
// using SigLIP encoder + MLP projector + Qwen2.5 decoder
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.TwoDimensional,
taskType: NeuralNetworkTaskType.ImageClassification,
inputHeight: 224, inputWidth: 224, inputDepth: 3, outputSize: 512);
// ONNX inference mode with pre-trained model
var model = new AquilaVL<double>(architecture, "aquilavl.onnx");
// Training mode with native layers
var trainModel = new AquilaVL<double>(architecture, new AquilaVLOptions());
Constructors
| Constructor | Summary |
|---|---|
AquilaVL(NeuralNetworkArchitecture<T> architecture, AquilaVLOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
AquilaVL(NeuralNetworkArchitecture<T> architecture, string modelPath, AquilaVLOptions options) |
Properties
| Property | Summary |
|---|---|
int DecoderEmbeddingDim { get; } |
|
int EmbeddingDimension { get; } |
|
int MaxGenerationLength { get; } |
|
string LanguageModelName { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> Chat(Tensor<T> image, IEnumerable<ValueTuple<string, string>> conversationHistory, string userMessage) |
|
Tensor<T> EncodeImage(Tensor<T> image) |
|
Tensor<T> GenerateFromImage(Tensor<T> image, string prompt) |
Generates text using Aquila-VL’s SigLIP + Qwen2.5 architecture. |