CogView4Model<T>
Models & Types · AiDotNet.Diffusion.TextToImage
CogView-4 model for bilingual text-to-image generation by Zhipu AI.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → CogView4Model<T>
For Beginners
CogView-4 is a model from Zhipu AI (Tsinghua University spinoff) that understands both Chinese and English prompts equally well.
How CogView-4 works:
- Text is encoded by GLM (General Language Model) with bilingual support
- A SiT (Scalable Interpolant Transformer) processes text and noise tokens
- Relay diffusion enables efficient high-resolution generation in two stages
- A 16-channel VAE decodes latents to full images
Key characteristics:
- SiT architecture with scalable interpolant formulation
- GLM text encoder for bilingual Chinese-English understanding
- Relay diffusion: low-res generation → high-res refinement
- 16 latent channels
- Strong understanding of Asian cultural contexts
- 30 inference steps typical
Advantages:
- Best-in-class bilingual prompt understanding
- Strong cultural context for Asian aesthetics
- Relay diffusion enables efficient high-res generation
- Open-weight model
Limitations:
- Smaller English-only community
- Less LoRA/adapter ecosystem
- Relay diffusion adds complexity
How It Works
CogView-4 uses a Scalable Interpolant Transformer (SiT) architecture with relay diffusion for high-resolution image generation. It features bilingual Chinese-English text understanding through a GLM-based text encoder, providing strong comprehension of both languages and Asian cultural contexts.
Technical specifications:
- Architecture: SiT with relay diffusion
- Text encoder: GLM (bilingual, 4096-dim)
- Hidden size: 2048, 24 layers, 32 heads
- Latent channels: 16
- Training: Continuous-time diffusion with interpolant formulation
- Default: 30 steps, guidance scale 7.5
- Resolution: 1024x1024 default
Reference: Zheng et al., “CogView-4: Bilingual Text-to-Image Generation with Relay Diffusion”, 2024
Example
// Create CogView-4
var cogView = new CogView4Model<float>();
// Generate with Chinese prompt
var image = cogView.GenerateFromText(
prompt: "A traditional Chinese ink painting of mountains in mist",
width: 1024,
height: 1024,
numInferenceSteps: 30,
guidanceScale: 7.5,
seed: 42);
Constructors
| Constructor | Summary |
|---|---|
CogView4Model(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, SiTPredictor<T> predictor, StandardVAE<T> vae, IConditioningModule<T> conditioner, int? seed) |
Initializes a new instance of CogView4Model. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> VAE { get; } |
|
bool SupportsBilingual { get; } |
Gets whether this model supports bilingual Chinese-English prompts. |
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> GenerateFromText(string prompt, string negativePrompt, int width, int height, int numInferenceSteps, double? guidanceScale, int? seed) |
|
Tensor<T> ImageToImage(Tensor<T> inputImage, string prompt, string negativePrompt, double strength, int numInferenceSteps, double? guidanceScale, int? seed) |
Fields
| Field | Summary |
|---|---|
int DefaultHeight |
Default image height for CogView-4 (1024x1024). |
int DefaultWidth |
Default image width for CogView-4 (1024x1024). |