Aria<T>
Models & Types · AiDotNet.VisionLanguage.InstructionTuned
Aria: mixture-of-experts multimodal model with efficient expert routing.
Inheritance: NeuralNetworkBase<T> → VisionLanguageModelBase<T> → Aria<T>
Implements: IInstructionTunedVLM<T>
For Beginners
Aria is a multimodal model that uses a “mixture of experts” approach — instead of running every parameter for every token, it has 64 specialized expert modules and routes each token to only the 8 most relevant ones. This makes the model very efficient (only 3.9B of 25.3B parameters are active at once) while still being highly capable. It can process images and long documents with up to 64K tokens of combined visual and text context. Default values follow the original paper settings.
How It Works
Aria (Rhymes AI, 2024) is a multimodal model that uses a Mixture-of-Experts (MoE) architecture for efficient scaling. It has 25.3 billion total parameters but only activates 3.9 billion per token, using 64 total experts with 8 active per token selected by a learned router. Each visual and text token is routed to its top-scoring experts, allowing different experts to specialize in different aspects of visual understanding. Aria supports a 64K multimodal context window for processing long sequences of interleaved images and text.
References:
- Paper: “Aria: An Open Multimodal Native Mixture-of-Experts Model” (Rhymes AI, 2024)
Example
// Create an Aria model for efficient multimodal understanding
// using Mixture-of-Experts with 64 experts, 8 active per token
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.TwoDimensional,
taskType: NeuralNetworkTaskType.ImageClassification,
inputHeight: 224, inputWidth: 224, inputDepth: 3, outputSize: 512);
// ONNX inference mode with pre-trained model
var model = new Aria<double>(architecture, "aria.onnx");
// Training mode with native layers
var trainModel = new Aria<double>(architecture, new AriaOptions());
Constructors
| Constructor | Summary |
|---|---|
Aria(NeuralNetworkArchitecture<T> architecture, AriaOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
Aria(NeuralNetworkArchitecture<T> architecture, string modelPath, AriaOptions options) |
Properties
| Property | Summary |
|---|---|
int DecoderEmbeddingDim { get; } |
|
int EmbeddingDimension { get; } |
|
int MaxGenerationLength { get; } |
|
string LanguageModelName { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> Chat(Tensor<T> image, IEnumerable<ValueTuple<string, string>> conversationHistory, string userMessage) |
|
Tensor<T> EncodeImage(Tensor<T> image) |
|
Tensor<T> GenerateFromImage(Tensor<T> image, string prompt) |
Generates text using Aria’s MoE-based multimodal architecture. |