VisionLanguageModelBase<T>
Base Classes · AiDotNet.VisionLanguage
Base class for vision-language neural networks that can operate in both ONNX inference and native training modes.
Inheritance: NeuralNetworkBase<T> → VisionLanguageModelBase<T>
Implements: IShapeContract
Known derived types: ALIGN<T>, AquilaVL<T>, Aria<T>, BASIC<T>, BLIP3<T>, BiomedCLIP<T>, BridgeTower<T>, CLIPA<T>, Cambrian1<T>, Chameleon<T>, ClaudeVision<T>, CoCa<T>, CogVLM2<T>, CogVLM<T>, DFNCLIP<T>, DINOX<T>, DINOv2<T>, DINOv3<T>, DeCLIP<T>, DeepSeekVL2<T>, DeepSeekVL<T>, DocPedia<T>, DragonflyMed<T>, Dragonfly<T>, EVACLIP<T>, Eagle25<T>, Eagle<T>, Emu2<T>, Emu3<T>, Emu<T>, FLIP<T>, FerretV2<T>, Ferret<T>, Florence2<T>, Fuyu<T>, GIT<T>, GLaMM<T>, GOTOCR2<T>, GPT4Point<T>, GR00TN1<T>, GeminiVision<T>, Gemma3<T>, GeoChat<T>, GrokVision<T>, Groma<T>, GroundedSAM2<T>, GroundingDINO15<T>, GroundingDINO<T>, Helix<T>, IDEFICS2<T>, IDEFICS3<T>, IDEFICS<T>, InstructBLIP<T>, InternVL25<T>, InternVL2<T>, InternVL3<T>, InternVL<T>, InternViT<T>, JanusPro<T>, Janus<T> (+103 more)
For Beginners
Vision-language models process both images and text together. This base class provides:
- Support for pre-trained ONNX models (fast inference with existing models)
- Full training capability from scratch (like other neural networks)
- Image preprocessing utilities (normalization, resizing)
- Dual-encoder architecture support (separate image and text encoders)
You can use this class in two ways:
- Load a pre-trained ONNX model for quick inference
- Build and train a new model from scratch
How It Works
This class extends NeuralNetworkBase to provide vision-language-specific functionality
while maintaining full integration with the AiDotNet neural network infrastructure.
Properties
| Property | Summary |
|---|---|
ILossFunction<T> DefaultLossFunction { get; } |
Gets the default loss function for this model. |
bool IsOnnxMode { get; } |
Gets whether this model is running in ONNX inference mode. |
bool SupportsTraining { get; } |
Gets whether this network supports training. |
int EmbeddingDim { get; set; } |
Gets the embedding dimensionality for this model. |
int ImageChannels { get; set; } |
Gets the number of image channels expected (typically 3 for RGB). |
int ImageSize { get; set; } |
Gets the expected input image size (height = width in pixels). |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
The family’s default output law: one pooled EmbeddingDim-wide embedding per sample. |