StreamingT2VModel<T>
Models & Types · AiDotNet.Diffusion.Video.LongVideo
StreamingT2V autoregressive long video generation up to 1200 frames.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → VideoDiffusionModelBase<T> → StreamingT2VModel<T>
For Beginners
Streaming T2V generates long videos (up to 2 minutes / 1200 frames) by producing segments autoregressively - each new segment continues seamlessly from the previous one. This overcomes the memory limitations of generating all frames at once.
How It Works
References:
- Paper: “StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text” (2024)
StreamingT2V enables creation of long videos (up to 1200 frames / 2 minutes) through an advanced autoregressive technique. It generates short clips and extends them using temporal overlap and conditioning on previous frames. Ensures rich motion dynamics and temporal consistency throughout the entire video sequence.
Technical specifications:
- Architecture: Autoregressive UNet + Temporal Overlap + Long Video
- Latent channels: 4
- Default: 1200 frames at 10 FPS
- Supports I2V: Yes | T2V: Yes | V2V: No
Example
var options = new DiffusionModelOptions<float> { LatentChannels = 4, DefaultInferenceSteps = 50 };
var noise = Tensor<float>.CreateRandom(new[] { 1, 4, 24, 32, 32 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
.ConfigureModel(new StreamingT2VModel<float>(options: options))
.Build(trainX, trainY);
var video = result.Predict(noise);
Constructors
| Constructor | Summary |
|---|---|
StreamingT2VModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, VideoUNetPredictor<T> predictor, TemporalVAE<T> temporalVAE, IConditioningModule<T> conditioner, int defaultNumFrames, int defaultFPS, int? seed) |
Initializes a new instance of StreamingT2VModel with full customization support. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> TemporalVAE { get; } |
|
IVAEModel<T> VAE { get; } |
|
bool SupportsImageToVideo { get; } |
|
bool SupportsTextToVideo { get; } |
|
bool SupportsVideoToVideo { get; } |
|
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |