OpenSora13Model<T>

Models & Types · AiDotNet.Diffusion.Video

Open-Sora 1.3 with upgraded 3D-VAE and rectified flow.

Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → VideoDiffusionModelBase<T> → OpenSora13Model<T>

For Beginners

Open-Sora 1.3 is a fully open-source video generation model that supports any resolution and aspect ratio. It uses alternating spatial and temporal attention (STDiT) for efficiency and rectified flow training for stable, high-quality generation from text prompts.

How It Works

References:

  • Paper: “Open-Sora: Democratizing Efficient Video Production for All” (HPC-AI Tech, 2024)

Open-Sora 1.3 features an upgraded 3D-VAE with improved temporal compression and rectified flow training objective. The STDiT backbone alternates spatial and temporal attention for efficient video generation. Supports any resolution and aspect ratio through dynamic patching.

Technical specifications:

  • Architecture: STDiT + Rectified Flow + Upgraded 3D-VAE
  • Latent channels: 4
  • Default: 51 frames at 24 FPS
  • Supports I2V: Yes | T2V: Yes | V2V: No

Example

var options = new DiffusionModelOptions<float> { LatentChannels = 4, DefaultInferenceSteps = 30 };
var noise = Tensor<float>.CreateRandom(new[] { 1, 4, 51, 60, 106 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
    .ConfigureModel(new OpenSora13Model<float>(options: options))
    .Build(trainX, trainY);
var video = result.Predict(noise);

Constructors

Constructor Summary
OpenSora13Model(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, DiTNoisePredictor<T> predictor, TemporalVAE<T> temporalVAE, IConditioningModule<T> conditioner, int defaultNumFrames, int defaultFPS, int? seed) Initializes a new instance of OpenSora13Model with full customization support.

Properties

Property Summary
IConditioningModule<T> Conditioner { get; }
INoisePredictor<T> NoisePredictor { get; }
IVAEModel<T> TemporalVAE { get; }
IVAEModel<T> VAE { get; }
bool SupportsImageToVideo { get; }
bool SupportsTextToVideo { get; }
bool SupportsVideoToVideo { get; }
int LatentChannels { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()