Dragonfly<T>
Models & Types · AiDotNet.VisionLanguage.InstructionTuned
Dragonfly: multi-resolution visual encoding VLM for fine-grained understanding.
Inheritance: NeuralNetworkBase<T> → VisionLanguageModelBase<T> → Dragonfly<T>
Implements: IInstructionTunedVLM<T>
For Beginners
Dragonfly is a vision-language model that looks at images at multiple zoom levels — like how you might first glance at a whole photo, then zoom into interesting parts for more detail. Its multi-resolution zoom module automatically identifies important regions and processes them at higher resolution while keeping the overall scene context at lower resolution. This approach is more efficient than processing the entire image at high resolution, while still capturing fine details where they matter. Default values follow the original paper settings.
How It Works
Dragonfly (Together AI, 2024) introduces multi-resolution visual encoding that processes images at multiple zoom levels simultaneously. It uses a “zoom-in” module that identifies important regions of the image and processes them at higher resolution, while maintaining a global view at lower resolution. This multi-resolution approach gives the model both broad scene understanding and fine-grained detail recognition without requiring extremely high-resolution processing of the entire image.
References:
- Paper: “Dragonfly: Multi-Resolution Zoom Supercharges Large Visual-Language Model” (Together AI, 2024)
Example
// Create a Dragonfly model for multi-resolution visual understanding
// using zoom-in modules for fine-grained detail recognition
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.TwoDimensional,
taskType: NeuralNetworkTaskType.ImageClassification,
inputHeight: 224, inputWidth: 224, inputDepth: 3, outputSize: 512);
// ONNX inference mode with pre-trained model
var model = new Dragonfly<double>(architecture, "dragonfly.onnx");
// Training mode with native layers
var trainModel = new Dragonfly<double>(architecture, new DragonflyOptions());
Constructors
| Constructor | Summary |
|---|---|
Dragonfly(NeuralNetworkArchitecture<T> architecture, DragonflyOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
Dragonfly(NeuralNetworkArchitecture<T> architecture, string modelPath, DragonflyOptions options) |
Properties
| Property | Summary |
|---|---|
int DecoderEmbeddingDim { get; } |
|
int EmbeddingDimension { get; } |
|
int MaxGenerationLength { get; } |
|
string LanguageModelName { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> Chat(Tensor<T> image, IEnumerable<ValueTuple<string, string>> conversationHistory, string userMessage) |
|
Tensor<T> EncodeImage(Tensor<T> image) |
|
Tensor<T> GenerateFromImage(Tensor<T> image, string prompt) |
Generates text using Dragonfly’s multi-resolution visual encoding with zoom-and-select. |