OmniGenModel<T>

Models & Types · AiDotNet.Diffusion.TextToImage

OmniGen model — unified image generation model handling multiple tasks in one architecture.

Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → OmniGenModel<T>

For Beginners

OmniGen is one model that does everything:

Key characteristics:

  • Unified model: text-to-image, editing, inpainting, subject-driven, etc.
  • No task-specific adapters needed (unlike ControlNet, IP-Adapter)
  • Single transformer backbone handles all tasks
  • Interleaved image-text input for flexible conditioning
  • In-context learning: understands task from examples

Tasks OmniGen can handle:

  • Text-to-image generation
  • Image editing (instruction-based)
  • Subject-driven generation (given reference images)
  • Visual conditional generation (depth, edge, pose)
  • Style transfer

Use OmniGen when you need:

  • Single model for multiple generation tasks
  • Simplified pipeline (no adapter management)
  • Flexible conditioning from images and text

How It Works

OmniGen is a unified image generation model that handles text-to-image, image editing, subject-driven generation, and visual conditional generation with a single model, without requiring task-specific adapters or fine-tuning.

Technical specifications:

  • Architecture: Unified transformer with interleaved image-text tokens
  • Parameters: ~3.8B
  • Text encoder: integrated (not separate)
  • Resolution: 512x512 to 1024x1024
  • Latent channels: 4, 8x downsampling

Reference: Xiao et al., “OmniGen: Unified Image Generation”, 2024

Example

var existingImage = Tensor<float>.CreateRandom(1, 3, 32, 32);
// Create with industry-standard defaults
var omniGen = new OmniGenModel<float>();
            
// Create with custom architecture for pre-trained weight loading
var architecture = new NeuralNetworkArchitecture<float>(
    InputType.ThreeDimensional, NeuralNetworkTaskType.Generative,
    inputHeight: 32, inputWidth: 32, inputDepth: 3);
var customOmniGen = new OmniGenModel<float>(architecture: architecture);
            
// Generate a 1024x1024 image from text
var image = omniGen.GenerateFromText(
    prompt: "A futuristic cityscape at dusk with neon lights reflecting off wet streets",
    negativePrompt: "blurry, low quality",
    width: 1024,
    height: 1024,
    numInferenceSteps: 50,
    guidanceScale: 3.0,
    seed: 42);
            
// Image-to-image transformation
var transformed = omniGen.ImageToImage(
    inputImage: existingImage,
    prompt: "Convert to watercolor painting style",
    strength: 0.7);

Constructors

Constructor Summary
OmniGenModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, DiTNoisePredictor<T> dit, StandardVAE<T> vae, IConditioningModule<T> conditioner, int? seed) Initializes a new instance of OmniGenModel with full customization support.

Properties

Property Summary
IConditioningModule<T> Conditioner { get; }
INoisePredictor<T> NoisePredictor { get; }
IVAEModel<T> VAE { get; }
int LatentChannels { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()
Tensor<T> GenerateFromText(string prompt, string negativePrompt, int width, int height, int numInferenceSteps, double? guidanceScale, int? seed)

Fields

Field Summary
int DefaultHeight Default image height for OmniGen generation (1024 pixels).
int DefaultWidth Default image width for OmniGen generation (1024 pixels).