JanusPro<T>
Models & Types · AiDotNet.VisionLanguage.Unified
Janus-Pro: unified multimodal understanding and generation with decoupled vision encoders (Chen et al., DeepSeek 2025, arXiv:2501.17811).
Inheritance: NeuralNetworkBase<T> → VisionLanguageModelBase<T> → JanusPro<T>
Implements: IUnifiedVisionModel<T>
For Beginners
Janus-Pro is the first model that does BOTH “understand image, answer
in text” AND “describe in text, generate image” with one unified backbone — but it uses two
completely different vision encoders for each direction, which the paper shows is much better than
trying to share. Default values follow the published 1.5B configuration (scale up via
JanusProOptions for the 7B variant).
How It Works
Janus-Pro is the scaled-up successor to Janus (Wu et al. 2024, arXiv:2410.13848). Both models share Janus’s central design insight: vision encoding for understanding (an image-to-language path that feeds SigLIP-style continuous features into the LLM) and vision encoding for generation (a VQ-VAE codebook that turns the LLM’s output token stream back into pixels) are fully decoupled. The two paths converge only at the autoregressive transformer backbone in the middle. Janus-Pro adds: a 16384-entry VQ codebook (vs Janus’s 8192), curriculum-based training, expanded synthetic data, and a 7B-parameter DeepSeek-LLM backbone.
Paper-faithful pieces implemented here:
What is NOT verified in-session:
Example
var image = Tensor<double>.CreateRandom(1, 3, 32, 32);
var arch = new NeuralNetworkArchitecture<double>(
inputType: InputType.TwoDimensional,
taskType: NeuralNetworkTaskType.ImageClassification,
inputHeight: 384, inputWidth: 384, inputDepth: 3, outputSize: 4096);
var model = new JanusPro<double>(arch, new JanusProOptions());
// Understanding path
var hidden = model.GenerateFromImage(image, "what do you see?");
// Generation path
var generated = model.GenerateImage("a red apple on a wooden table");
Constructors
| Constructor | Summary |
|---|---|
JanusPro(NeuralNetworkArchitecture<T> architecture, JanusProOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
JanusPro(NeuralNetworkArchitecture<T> architecture, string modelPath, JanusProOptions options) |
Properties
| Property | Summary |
|---|---|
bool SupportsGeneration { get; } |
|
double CfgScale { get; } |
Classifier-free guidance scale used during autoregressive image generation. |
int DecoderEmbeddingDim { get; } |
|
int EmbeddingDimension { get; } |
|
int GenerationTokenCount { get; } |
Number of generation-side VQ tokens in the output grid (24×24 for 384px output, matching paper §3.3). |
int MaxGenerationLength { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> EncodeImage(Tensor<T> image) |
Janus-Pro understanding path: image → SigLIP-style continuous features → LLM hidden state. |
Tensor<T> GenerateFromImage(Tensor<T> image, string prompt) |
Image-to-text (understanding) forward pass. |
Tensor<T> GenerateImage(string textDescription) |
Text-to-image (generation) forward pass. |