MeissonicModel<T>
Models & Types · AiDotNet.Diffusion.TextToImage
Meissonic model for non-autoregressive masked image modeling (MIM) generation.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → MeissonicModel<T>
For Beginners
Unlike diffusion models that gradually remove noise over many steps, Meissonic works by masking parts of an image and predicting all missing parts at once.
How Meissonic works:
- Text is encoded by a CLIP text encoder
- Image tokens are randomly masked according to a cosine schedule
- An E-MMDiT transformer predicts all masked tokens simultaneously
- Iterative refinement: re-mask low-confidence tokens and predict again
- VQ-VAE decodes discrete tokens to pixel space
Key characteristics:
- Non-autoregressive: predicts ALL masked tokens in parallel (not one by one)
- E-MMDiT backbone: only 304M parameters (very lightweight)
- Cosine masking schedule for iterative refinement
- CLIP text encoder for conditioning
- 16 latent channels via VQ-VAE tokenizer
- 10-20 refinement iterations (faster than 50-step diffusion)
Advantages:
- Very fast generation (10-20 iterations)
- Extremely lightweight (304M vs 2-12B for competitors)
- Runs on consumer GPUs with minimal VRAM
- High quality despite small size
Limitations:
- Quality below top diffusion models for complex scenes
- Discrete token space can show quantization artifacts
- Less fine-grained control than continuous latent diffusion
How It Works
Meissonic uses masked image modeling (MIM) with a non-autoregressive approach for fast, high-quality image generation. Instead of iterative denoising, it masks and predicts image tokens in parallel using a lightweight E-MMDiT backbone (304M parameters), achieving fast generation with quality competitive with much larger diffusion models.
Technical specifications:
- Architecture: Non-autoregressive masked generative transformer
- Backbone: E-MMDiT with 304M parameters
- Hidden size: 1024, 12 layers, 16 heads
- Text encoder: CLIP (768-dim)
- Tokenizer: VQ-VAE with 16 channels
- Masking: Cosine schedule with iterative refinement
- Default: 18 refinement iterations, guidance scale 9.0
- Resolution: 1024x1024
Reference: Bai et al., “Meissonic: Revitalizing Masked Generative Transformers for Efficient High-Resolution Text-to-Image Synthesis”, 2024
Example
// Create Meissonic model
var meissonic = new MeissonicModel<float>();
// Generate quickly with fewer iterations
var image = meissonic.GenerateFromText(
prompt: "A vibrant sunset over a calm ocean with sailing boats",
width: 1024,
height: 1024,
numInferenceSteps: 18,
guidanceScale: 9.0,
seed: 42);
Constructors
| Constructor | Summary |
|---|---|
MeissonicModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, EMMDiTPredictor<T> predictor, StandardVAE<T> vae, IConditioningModule<T> conditioner, int? seed) |
Initializes a new instance of MeissonicModel. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> VAE { get; } |
|
bool UsesMaskedModeling { get; } |
Gets whether this model uses masked image modeling (true) vs continuous diffusion. |
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> GenerateFromText(string prompt, string negativePrompt, int width, int height, int numInferenceSteps, double? guidanceScale, int? seed) |
|
Tensor<T> ImageToImage(Tensor<T> inputImage, string prompt, string negativePrompt, double strength, int numInferenceSteps, double? guidanceScale, int? seed) |
Fields
| Field | Summary |
|---|---|
int DefaultHeight |
Default image height for Meissonic (1024x1024). |
int DefaultWidth |
Default image width for Meissonic (1024x1024). |