OmniGenModel<T>
Models & Types · AiDotNet.Diffusion.TextToImage
OmniGen model — unified image generation model handling multiple tasks in one architecture.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → OmniGenModel<T>
For Beginners
OmniGen is one model that does everything:
Key characteristics:
- Unified model: text-to-image, editing, inpainting, subject-driven, etc.
- No task-specific adapters needed (unlike ControlNet, IP-Adapter)
- Single transformer backbone handles all tasks
- Interleaved image-text input for flexible conditioning
- In-context learning: understands task from examples
Tasks OmniGen can handle:
- Text-to-image generation
- Image editing (instruction-based)
- Subject-driven generation (given reference images)
- Visual conditional generation (depth, edge, pose)
- Style transfer
Use OmniGen when you need:
- Single model for multiple generation tasks
- Simplified pipeline (no adapter management)
- Flexible conditioning from images and text
How It Works
OmniGen is a unified image generation model that handles text-to-image, image editing, subject-driven generation, and visual conditional generation with a single model, without requiring task-specific adapters or fine-tuning.
Technical specifications:
- Architecture: Unified transformer with interleaved image-text tokens
- Parameters: ~3.8B
- Text encoder: integrated (not separate)
- Resolution: 512x512 to 1024x1024
- Latent channels: 4, 8x downsampling
Reference: Xiao et al., “OmniGen: Unified Image Generation”, 2024
Example
var existingImage = Tensor<float>.CreateRandom(1, 3, 32, 32);
// Create with industry-standard defaults
var omniGen = new OmniGenModel<float>();
// Create with custom architecture for pre-trained weight loading
var architecture = new NeuralNetworkArchitecture<float>(
InputType.ThreeDimensional, NeuralNetworkTaskType.Generative,
inputHeight: 32, inputWidth: 32, inputDepth: 3);
var customOmniGen = new OmniGenModel<float>(architecture: architecture);
// Generate a 1024x1024 image from text
var image = omniGen.GenerateFromText(
prompt: "A futuristic cityscape at dusk with neon lights reflecting off wet streets",
negativePrompt: "blurry, low quality",
width: 1024,
height: 1024,
numInferenceSteps: 50,
guidanceScale: 3.0,
seed: 42);
// Image-to-image transformation
var transformed = omniGen.ImageToImage(
inputImage: existingImage,
prompt: "Convert to watercolor painting style",
strength: 0.7);
Constructors
| Constructor | Summary |
|---|---|
OmniGenModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, DiTNoisePredictor<T> dit, StandardVAE<T> vae, IConditioningModule<T> conditioner, int? seed) |
Initializes a new instance of OmniGenModel with full customization support. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> VAE { get; } |
|
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> GenerateFromText(string prompt, string negativePrompt, int width, int height, int numInferenceSteps, double? guidanceScale, int? seed) |
Fields
| Field | Summary |
|---|---|
int DefaultHeight |
Default image height for OmniGen generation (1024 pixels). |
int DefaultWidth |
Default image width for OmniGen generation (1024 pixels). |