Llama32Vision<T>

Models & Types · AiDotNet.VisionLanguage.InstructionTuned

Llama 3.2 Vision: Meta’s 11B/90B vision models for edge/mobile deployment.

Inheritance: NeuralNetworkBase<T> → VisionLanguageModelBase<T> → Llama32Vision<T>

Implements: IInstructionTunedVLM<T>

For Beginners

Llama 3.2 Vision adds image understanding to Meta’s popular Llama 3.2 language model. Available in 11B and 90B parameter sizes, it uses a ViT vision encoder connected through an MLP projection to the Llama decoder. The 11B variant is particularly notable for being optimized for edge and mobile deployment — it can run on devices like phones and laptops rather than requiring expensive cloud servers. The 90B variant provides stronger performance for server-side use. Both models can describe images, answer visual questions, and reason about visual content. Default values follow the original paper settings.

How It Works

Llama 3.2 Vision (Meta, 2024) extends the Llama 3.2 language model with vision capabilities, available in 11B and 90B parameter variants. It uses a ViT vision encoder with MLP projection to integrate visual features into the Llama decoder, optimized for edge and mobile deployment.

References:

  • Paper: “The Llama 3 Herd of Models” (Meta, 2024)

Example

// Create a Llama 3.2 Vision model for edge/mobile visual AI
// using ViT encoder with Llama 3.2 decoder backbone
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.TwoDimensional,
    taskType: NeuralNetworkTaskType.ImageClassification,
    inputHeight: 224, inputWidth: 224, inputDepth: 3, outputSize: 512);
            
// ONNX inference mode with pre-trained model
var model = new Llama32Vision<double>(architecture, "llama32vision.onnx");
            
// Training mode with native layers
var trainModel = new Llama32Vision<double>(architecture, new Llama32VisionOptions());

Constructors

Constructor Summary
Llama32Vision(NeuralNetworkArchitecture<T> architecture, Llama32VisionOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer)
Llama32Vision(NeuralNetworkArchitecture<T> architecture, string modelPath, Llama32VisionOptions options)

Properties

Property Summary
int DecoderEmbeddingDim { get; }
int EmbeddingDimension { get; }
int MaxGenerationLength { get; }
string LanguageModelName { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()
Tensor<T> Chat(Tensor<T> image, IEnumerable<ValueTuple<string, string>> conversationHistory, string userMessage)
Tensor<T> EncodeImage(Tensor<T> image)
Tensor<T> GenerateFromImage(Tensor<T> image, string prompt) Generates text using Llama 3.2 Vision’s cross-attention adapter architecture.