ImagenModel<T>
Models & Types · AiDotNet.Diffusion.TextToImage
Imagen model for cascaded text-to-image generation with T5 text encoding.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → ImagenModel<T>
For Beginners
Imagen generates images by starting small and upscaling:
How Imagen works:
- Text is encoded by a frozen T5-XXL language model (4096-dim embeddings)
- A base diffusion model generates a 64x64 image from the text embeddings
- A first super-resolution model upscales 64x64 → 256x256
- A second super-resolution model upscales 256x256 → 1024x1024
Key characteristics:
- Cascaded pixel-space diffusion: 64→256→1024
- Text encoder: Frozen T5-XXL (4.7B parameters, 4096-dim)
- Base model: ~2B parameters, 64x64 output
- Super-res 1: ~600M parameters, 256x256 output
- Super-res 2: ~400M parameters, 1024x1024 output
- Uses Efficient U-Net architecture
- Dynamic thresholding for improved image quality
Key innovations:
- Demonstrated that text encoder quality is most important
- Introduced dynamic thresholding (better high guidance scales)
- Efficient U-Net: memory-efficient attention, shifted convolutions
- Noise conditioning augmentation for super-resolution stages
Limitations:
- Not open-source (proprietary to Google)
- Very large compute requirements
- Three separate models needed for full pipeline
How It Works
Imagen is a cascaded text-to-image diffusion model developed by Google Brain. It demonstrates that large frozen language models (T5-XXL) are highly effective for text-to-image generation, and that scaling the text encoder matters more than scaling the image model.
Technical specifications:
- Architecture: Cascaded pixel-space diffusion with Efficient U-Net
- Base model: 64x64, ~2B parameters, 3 RGB channels
- Super-res 1: 64→256, ~600M parameters
- Super-res 2: 256→1024, ~400M parameters
- Text encoder: Frozen T5-XXL (4.7B params, 4096-dim, 256 max tokens)
- Noise schedule: Cosine schedule, 1000 training timesteps
- Prediction type: Epsilon prediction with dynamic thresholding
- Noise conditioning augmentation for super-res stages
Reference: Saharia et al., “Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding”, NeurIPS 2022
Example
// Create with industry-standard defaults
var imagen = new ImagenModel<float>();
// Generate a 64x64 base image
var image = imagen.GenerateFromText(
prompt: "A brain riding a rocketship heading towards the moon",
negativePrompt: null,
width: 64,
height: 64,
numInferenceSteps: 100,
guidanceScale: 7.5,
seed: 42);
Constructors
| Constructor | Summary |
|---|---|
ImagenModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, UNetNoisePredictor<T> baseUnet, UNetNoisePredictor<T> superRes1Unet, StandardVAE<T> vae, IConditioningModule<T> conditioner, double dynamicThresholdPercentile, int? seed) |
Initializes a new instance of ImagenModel with full customization support. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
INoisePredictor<T> SuperResolution1 { get; } |
Gets the super-resolution Stage 1 noise predictor (64→256). |
IVAEModel<T> VAE { get; } |
|
double DynamicThresholdPercentile { get; } |
Gets the dynamic thresholding percentile. |
int CrossAttentionDim { get; } |
Gets the cross-attention dimension (4096 for T5-XXL). |
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> GenerateFromText(string prompt, string negativePrompt, int width, int height, int numInferenceSteps, double? guidanceScale, int? seed) |
|
Tensor<T> ImageToImage(Tensor<T> inputImage, string prompt, string negativePrompt, double strength, int numInferenceSteps, double? guidanceScale, int? seed) |
Fields
| Field | Summary |
|---|---|
int DefaultHeight |
Default image height for Imagen base model (64x64). |
int DefaultWidth |
Default image width for Imagen base model (64x64). |