PixArtModel<T>

Models & Types · AiDotNet.Diffusion.TextToImage

PixArt-α model for efficient high-quality text-to-image generation using DiT architecture.

Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → PixArtModel<T>

For Beginners

PixArt-α is like a sports car version of image generation:

Key advantages over traditional models:

  • 10x faster training than Stable Diffusion
  • Much more parameter-efficient
  • Uses transformer blocks instead of U-Net
  • T5-XXL text encoder for better prompt understanding

How PixArt-α works:

  1. Your prompt goes through a T5-XXL text encoder (larger = better understanding)
  2. The DiT (Diffusion Transformer) denoises using attention blocks
  3. Each block uses cross-attention to the text embedding
  4. The output is decoded by a VAE into an image

Example use cases:

  • Fast prototyping (quick iterations)
  • Resource-constrained environments (smaller models)
  • High-quality generation without massive GPU requirements
  • Applications requiring many generations

When to choose PixArt-α:

  • You need faster generation than SDXL
  • You want good quality without 70B+ model overhead
  • Your prompts are complex (T5 encoder helps)
  • You’re doing many generations in batch

How It Works

PixArt-α is an efficient text-to-image diffusion model that uses a Diffusion Transformer (DiT) architecture. It achieves comparable quality to larger models like Stable Diffusion XL while being significantly faster and more resource-efficient.

Technical specifications:

  • Architecture: Diffusion Transformer (DiT) with AdaLN-single
  • Text encoder: T5-XXL (4.3B parameters, optional smaller variants)
  • Native resolutions: 256x256 to 1024x1024
  • Latent space: 4 channels, 8x spatial downsampling
  • Training: Decomposed training strategy for efficiency

Architecture innovations:

  • Cross-attention in every DiT block
  • AdaLN-single for timestep conditioning (not AdaLN-Zero)
  • Efficient attention patterns
  • Multi-resolution training support

Example

// Create a PixArt-α model
var pixart = new PixArtModel<float>();
            
// Generate an image efficiently
var image = pixart.GenerateFromText(
    prompt: "A serene Japanese garden with cherry blossoms",
    negativePrompt: "blurry, low quality",
    width: 1024,
    height: 1024,
    numInferenceSteps: 20,
    guidanceScale: 4.5,
    seed: 42);
            
// Use different resolutions
var portrait = pixart.GenerateFromText(
    prompt: "Portrait of an astronaut",
    width: 768,
    height: 1024);
            
// Generate multiple images with different seeds
var variations = pixart.GenerateVariations(
    prompt: "Abstract art with vibrant colors",
    count: 4);

Constructors

Constructor Summary
PixArtModel(NeuralNetworkArchitecture<T> architecture, PixArtVariant modelSize, IConditioningModule<T> conditioner, INoiseScheduler<T> scheduler, DiTNoisePredictor<T> dit, StandardVAE<T> vae, int? seed) Initializes a new instance of PixArtModel with default parameters.

Properties

Property Summary
IConditioningModule<T> Conditioner { get; }
INoisePredictor<T> NoisePredictor { get; }
IVAEModel<T> VAE { get; }
PixArtVariant ModelSize { get; } Gets the model variant.
bool SupportsVariableAspectRatio { get; } Gets whether this model supports variable aspect ratios.
int DefaultResolution { get; } Gets the default resolution for this model.
int HiddenDimension { get; } Gets the hidden dimension of the transformer.
int LatentChannels { get; }
int NumAttentionHeads { get; } Gets the number of attention heads.
int NumLayers { get; } Gets the number of transformer layers.

Methods

Method Summary
List<Tensor<T>> GenerateVariations(string prompt, string negativePrompt, int count, int width, int height, int numInferenceSteps, double? guidanceScale, int? baseSeed) Generates multiple image variations with different seeds.
List<ValueTuple<int, int, string>> GetSupportedResolutions() Gets supported resolutions for this model variant.
ModelMetadata<T> GetModelMetadata()
Tensor<T> GenerateFromText(string prompt, string negativePrompt, int width, int height, int numInferenceSteps, double? guidanceScale, int? seed) Generates an image with PixArt-α’s efficient DiT architecture.
Tensor<T> GenerateWithAspectRatio(string prompt, string aspectRatio, string negativePrompt, int baseResolution, int numInferenceSteps, double? guidanceScale, int? seed) Generates an image with specified aspect ratio preset.
Tensor<T> ImageToImage(Tensor<T> inputImage, string prompt, string negativePrompt, double strength, int numInferenceSteps, double? guidanceScale, int? seed) Performs image-to-image transformation with PixArt-α.
ValueTuple<int, double, int> GetRecommendedSettings() Gets the recommended settings for this model variant.

Fields

Field Summary
PixArtVariant DefaultVariant Default model variant.