KandinskyModel<T>
Models & Types · AiDotNet.Diffusion.TextToImage
Kandinsky 2.2/3.0 model for text-to-image generation.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → KandinskyModel<T>
For Beginners
Kandinsky works in two phases, like a translator:
How Kandinsky works:
- Prior stage: Translates text embeddings (CLIP) into image embeddings
- Decoder stage: A latent diffusion model generates images conditioned on the image embeddings
Key characteristics:
- Two-stage pipeline: Prior → Decoder
- Prior: Diffusion-based mapping from text to image embedding space
- Decoder: U-Net latent diffusion model (similar to SD architecture)
- Text encoder: CLIP ViT-G/14 (1280-dim) + multilingual XLM-RoBERTa
- Image encoder: CLIP ViT-G/14 (used to train the prior)
- Native resolution: 1024x1024 (Kandinsky 3.0) or 512x512 (Kandinsky 2.2)
- VAE: Movq (MoVQ-GAN), 4 latent channels
Advantages:
- Strong multilingual support through XLM-RoBERTa
- Two-stage architecture allows separate optimization of text→embedding and embedding→image
- Good prompt adherence through CLIP-space prior
Limitations:
- Two-stage pipeline adds latency
- Smaller ecosystem than Stable Diffusion
How It Works
Kandinsky is a two-stage text-to-image model developed by Sber AI and AI Forever. It uses a prior model (diffusion or transformer) to map text embeddings to image embeddings, then a latent diffusion decoder to generate images from those embeddings.
Technical specifications:
- Architecture: Prior + Latent Diffusion Decoder
- Prior: Diffusion transformer, maps CLIP text→image embeddings
- Decoder U-Net: ~1.2B parameters, 4-channel latent, channel multipliers [1, 2, 4, 4]
- Text encoder: CLIP ViT-G/14 (1280-dim)
- VAE: MoVQ-GAN, 4 latent channels, scale factor 0.18215
- Noise schedule: Linear beta schedule, 1000 training timesteps
Reference: Razzhigaev et al., “Kandinsky: an Improved Text-to-Image Synthesis with Image Prior and Latent Diffusion”, 2023
Example
// Create with industry-standard defaults
var kandinsky = new KandinskyModel<float>();
// Generate a 1024x1024 image from text
var image = kandinsky.GenerateFromText(
prompt: "An oil painting of a Russian winter landscape",
negativePrompt: "blurry, deformed",
width: 1024,
height: 1024,
numInferenceSteps: 50,
guidanceScale: 4.0,
seed: 42);
Constructors
| Constructor | Summary |
|---|---|
KandinskyModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> priorUnet, UNetNoisePredictor<T> decoderUnet, StandardVAE<T> vae, IConditioningModule<T> conditioner, KandinskyVersion version, int? seed) |
Initializes a new instance of KandinskyModel with full customization support. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
INoisePredictor<T> PriorModel { get; } |
Gets the prior model that maps text embeddings to image embeddings. |
IVAEModel<T> VAE { get; } |
|
KandinskyVersion Version { get; } |
Gets the model version. |
int CrossAttentionDim { get; } |
Gets the cross-attention dimension (1280 for CLIP ViT-G/14). |
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> GenerateFromText(string prompt, string negativePrompt, int width, int height, int numInferenceSteps, double? guidanceScale, int? seed) |
|
Tensor<T> ImageToImage(Tensor<T> inputImage, string prompt, string negativePrompt, double strength, int numInferenceSteps, double? guidanceScale, int? seed) |
Fields
| Field | Summary |
|---|---|
int DefaultHeight |
Default image height for Kandinsky 3.0 generation. |
int DefaultWidth |
Default image width for Kandinsky 3.0 generation. |