Mochi1Model<T>

Models & Types · AiDotNet.Diffusion.Video

Mochi 1 model for asymmetric DiT video generation with joint text-video attention.

Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → VideoDiffusionModelBase<T> → Mochi1Model<T>

For Beginners

Mochi 1 is a state-of-the-art open-source video generation model.

How Mochi 1 works:

  1. Text prompt is encoded by T5-XXL into 4096-dim embeddings
  2. Video is compressed by the asymmetric VAE into 12-channel latent space
  3. The AsymmDiT processes text and video tokens jointly (not via cross-attention)
  4. Joint attention allows deep text-video interaction in every layer
  5. The heavy VAE decoder reconstructs high-quality video

Key characteristics:

  • ~10B parameters (one of the largest open-source video models)
  • Joint text-video attention (deeper integration than cross-attention)
  • Asymmetric VAE: fast encoding, high-quality decoding
  • 480p at 30 FPS, 84 frames (~2.8 seconds)
  • Open-source weights under Apache 2.0 license

When to use Mochi 1:

  • High-quality open-source video generation
  • Research on joint attention mechanisms
  • Text-to-video with strong motion understanding
  • Commercial use (Apache 2.0 license)

Limitations:

  • Very large model (10B parameters, high VRAM requirements)
  • Limited to 480p resolution
  • Shorter duration than some competitors
  • Slower inference due to model size

How It Works

Mochi 1 by Genmo uses an Asymmetric Diffusion Transformer (AsymmDiT) architecture with joint text-video attention and an asymmetric encoder-decoder VAE. The asymmetric design uses lightweight encoding and heavy decoding for quality.

Architecture components:

  • AsymmDiT with 48 transformer layers and 3072 hidden dimension
  • 24 attention heads with joint text-video attention (not cross-attention)
  • Asymmetric VAE: lightweight encoder, heavy decoder for quality
  • 12 latent channels with 3D causal temporal compression
  • T5-XXL text encoder for 4096-dim context embeddings
  • Flow matching training objective

Technical specifications:

  • Architecture: AsymmDiT (Asymmetric Diffusion Transformer)
  • Hidden dimension: 3072
  • Transformer layers: 48
  • Attention heads: 24
  • Patch size: 2
  • Latent channels: 12 (asymmetric VAE)
  • Context dimension: 4096 (T5-XXL)
  • Attention: Joint text-video (not cross-attention)
  • Default: 84 frames at 30 FPS (~2.8 seconds)
  • Training objective: Flow matching
  • Total parameters: ~10B
  • License: Apache 2.0

Reference: Genmo, “Mochi 1: A New SOTA in Open-Source Video Generation”, 2024

Example

// Create with industry-standard defaults
var mochi = new Mochi1Model<float>();
            
// Generate video from text
var video = mochi.GenerateFromText(
    prompt: "A butterfly emerging from its chrysalis in slow motion",
    width: 848,
    height: 480,
    numFrames: 84,
    fps: 30,
    numInferenceSteps: 64,
    guidanceScale: 4.5);

Constructors

Constructor Summary
Mochi1Model(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, DiTNoisePredictor<T> dit, TemporalVAE<T> temporalVAE, IConditioningModule<T> conditioner, int defaultNumFrames, int defaultFPS, int? seed) Initializes a new instance of Mochi1Model with full customization support.

Properties

Property Summary
IConditioningModule<T> Conditioner { get; }
INoisePredictor<T> NoisePredictor { get; }
IVAEModel<T> TemporalVAE { get; }
IVAEModel<T> VAE { get; }
bool SupportsImageToVideo { get; }
bool SupportsTextToVideo { get; }
bool SupportsVideoToVideo { get; }
int LatentChannels { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()