Moondream<T>
Models & Types · AiDotNet.VisionLanguage.InstructionTuned
Moondream: tiny yet capable VLM for resource-constrained environments.
Inheritance: NeuralNetworkBase<T> → VisionLanguageModelBase<T> → Moondream<T>
Implements: IInstructionTunedVLM<T>
For Beginners
Moondream is one of the smallest usable vision-language models available — with under 2 billion parameters, it can run on devices with very limited memory and compute. Despite its tiny size, it can describe images, answer questions about photos, and perform basic visual reasoning. It uses a SigLIP vision encoder paired with a compact Phi-based language model. Think of it as the model you use when you need on-device visual AI but have very limited resources — like a Raspberry Pi or an embedded system. Default values follow the original paper settings.
How It Works
Moondream (Korrapati, 2024) is an extremely compact vision-language model designed for resource-constrained environments. Despite having fewer than 2 billion parameters, it achieves surprisingly capable visual understanding by using an efficient SigLIP vision encoder with a compact Phi-based language model. Its small footprint makes it suitable for edge deployment, embedded systems, and scenarios where larger models would be impractical due to memory or compute limitations.
References:
- Paper: “Moondream: A Tiny Vision Language Model” (Vikhyat Korrapati, 2024)
Example
// Create a Moondream model for lightweight on-device visual AI
// using compact SigLIP + Phi architecture under 2B parameters
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.TwoDimensional,
taskType: NeuralNetworkTaskType.ImageClassification,
inputHeight: 224, inputWidth: 224, inputDepth: 3, outputSize: 512);
// ONNX inference mode with pre-trained model
var model = new Moondream<double>(architecture, "moondream.onnx");
// Training mode with native layers
var trainModel = new Moondream<double>(architecture, new MoondreamOptions());
Constructors
| Constructor | Summary |
|---|---|
Moondream(NeuralNetworkArchitecture<T> architecture, MoondreamOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
Moondream(NeuralNetworkArchitecture<T> architecture, string modelPath, MoondreamOptions options) |
Properties
| Property | Summary |
|---|---|
int DecoderEmbeddingDim { get; } |
|
int EmbeddingDimension { get; } |
|
int MaxGenerationLength { get; } |
|
string LanguageModelName { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> Chat(Tensor<T> image, IEnumerable<ValueTuple<string, string>> conversationHistory, string userMessage) |
|
Tensor<T> EncodeImage(Tensor<T> image) |
|
Tensor<T> GenerateFromImage(Tensor<T> image, string prompt) |
Generates text using Moondream’s lightweight SigLIP + Phi-1.5 architecture. |