RunwayGen4Model<T>
Models & Types · AiDotNet.Diffusion.Video
Runway Gen-4 multi-modal understanding and generation model.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → VideoDiffusionModelBase<T> → RunwayGen4Model<T>
For Beginners
Runway Gen-4 combines scene understanding with video generation, enabling multi-modal control over the output. It can generate from text, images, or existing video with fine-grained control over style, motion, and composition.
How It Works
References:
- Reference: Runway Gen-4 (2025)
Runway Gen-4 combines multi-modal understanding with video generation, enabling the model to understand and generate video from diverse inputs including text, images, video references, and style descriptions. Features advanced camera control and consistent character generation.
Technical specifications:
- Architecture: Multi-Modal DiT + Understanding Engine
- Latent channels: 16
- Default: 120 frames at 24 FPS
- Supports I2V: Yes | T2V: Yes | V2V: No
Example
var options = new DiffusionModelOptions<float> { LatentChannels = 16, DefaultInferenceSteps = 30 };
var noise = Tensor<float>.CreateRandom(new[] { 1, 16, 120, 135, 240 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
.ConfigureModel(new RunwayGen4Model<float>(options: options))
.Build(trainX, trainY);
var video = result.Predict(noise);
Constructors
| Constructor | Summary |
|---|---|
RunwayGen4Model(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, DiTNoisePredictor<T> predictor, TemporalVAE<T> temporalVAE, IConditioningModule<T> conditioner, int defaultNumFrames, int defaultFPS) |
Initializes a new instance of RunwayGen4Model with full customization support. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> TemporalVAE { get; } |
|
IVAEModel<T> VAE { get; } |
|
bool SupportsImageToVideo { get; } |
|
bool SupportsTextToVideo { get; } |
|
bool SupportsVideoToVideo { get; } |
|
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |