InternViT<T>
Models & Types · AiDotNet.VisionLanguage.Encoders
InternViT, a 6B-parameter ViT used as the vision encoder in the InternVL series.
Inheritance: NeuralNetworkBase<T> → VisionLanguageModelBase<T> → InternViT<T>
Implements: IVisualEncoder<T>
For Beginners
InternViT is a 6 billion parameter Vision Transformer designed as the vision backbone for the InternVL series of multimodal models. It handles high- resolution images by dynamically tiling them and using pixel shuffle to reduce the number of visual tokens while preserving spatial detail. It is progressively aligned with large language models for visual-linguistic tasks. Default values follow the original paper settings.
How It Works
InternViT (Chen et al., 2024) is a large-scale ViT designed for progressive alignment with LLMs. It uses dynamic resolution via image tiling and pixel shuffle downsampling to reduce token counts while preserving spatial detail, making it efficient for high-resolution images.
References:
- Paper: “InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks” (Chen et al., 2024)
Example
// Create an InternViT model for large-scale visual feature extraction
// with dynamic resolution and pixel shuffle for LLM alignment
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.TwoDimensional,
taskType: NeuralNetworkTaskType.ImageClassification,
inputHeight: 224, inputWidth: 224, inputDepth: 3, outputSize: 512);
// ONNX inference mode with pre-trained model
var model = new InternViT<double>(architecture, "internvit.onnx");
// Training mode with native layers
var trainModel = new InternViT<double>(architecture, new InternViTOptions());
Constructors
| Constructor | Summary |
|---|---|
InternViT(NeuralNetworkArchitecture<T> architecture, InternViTOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
InternViT(NeuralNetworkArchitecture<T> architecture, string modelPath, InternViTOptions options) |
Properties
| Property | Summary |
|---|---|
int EmbeddingDimension { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> EncodeImage(Tensor<T> image) |