Llama32Vision<T>
Models & Types · AiDotNet.VisionLanguage.InstructionTuned
Llama 3.2 Vision: Meta’s 11B/90B vision models for edge/mobile deployment.
Inheritance: NeuralNetworkBase<T> → VisionLanguageModelBase<T> → Llama32Vision<T>
Implements: IInstructionTunedVLM<T>
For Beginners
Llama 3.2 Vision adds image understanding to Meta’s popular Llama 3.2 language model. Available in 11B and 90B parameter sizes, it uses a ViT vision encoder connected through an MLP projection to the Llama decoder. The 11B variant is particularly notable for being optimized for edge and mobile deployment — it can run on devices like phones and laptops rather than requiring expensive cloud servers. The 90B variant provides stronger performance for server-side use. Both models can describe images, answer visual questions, and reason about visual content. Default values follow the original paper settings.
How It Works
Llama 3.2 Vision (Meta, 2024) extends the Llama 3.2 language model with vision capabilities, available in 11B and 90B parameter variants. It uses a ViT vision encoder with MLP projection to integrate visual features into the Llama decoder, optimized for edge and mobile deployment.
References:
- Paper: “The Llama 3 Herd of Models” (Meta, 2024)
Example
// Create a Llama 3.2 Vision model for edge/mobile visual AI
// using ViT encoder with Llama 3.2 decoder backbone
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.TwoDimensional,
taskType: NeuralNetworkTaskType.ImageClassification,
inputHeight: 224, inputWidth: 224, inputDepth: 3, outputSize: 512);
// ONNX inference mode with pre-trained model
var model = new Llama32Vision<double>(architecture, "llama32vision.onnx");
// Training mode with native layers
var trainModel = new Llama32Vision<double>(architecture, new Llama32VisionOptions());
Constructors
| Constructor | Summary |
|---|---|
Llama32Vision(NeuralNetworkArchitecture<T> architecture, Llama32VisionOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
Llama32Vision(NeuralNetworkArchitecture<T> architecture, string modelPath, Llama32VisionOptions options) |
Properties
| Property | Summary |
|---|---|
int DecoderEmbeddingDim { get; } |
|
int EmbeddingDimension { get; } |
|
int MaxGenerationLength { get; } |
|
string LanguageModelName { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> Chat(Tensor<T> image, IEnumerable<ValueTuple<string, string>> conversationHistory, string userMessage) |
|
Tensor<T> EncodeImage(Tensor<T> image) |
|
Tensor<T> GenerateFromImage(Tensor<T> image, string prompt) |
Generates text using Llama 3.2 Vision’s cross-attention adapter architecture. |