FreeNoiseVideoModel<T>
Models & Types · AiDotNet.Diffusion.Video.LongVideo
FreeNoise extended video generation through noise rescheduling.
Inheritance: DiffusionModelBase<T> → LatentDiffusionModelBase<T> → VideoDiffusionModelBase<T> → FreeNoiseVideoModel<T>
For Beginners
FreeNoise extends short video generators to produce longer sequences by intelligently rescheduling the noise patterns. This simple technique enables existing models to generate longer videos without retraining.
How It Works
References:
- Paper: “FreeNoise: Tuning-Free Longer Video Diffusion via Noise Rescheduling” (2024)
FreeNoise enables longer video generation without additional training by rescheduling the noise initialization. Instead of independent noise for each frame, it uses temporally correlated noise with a sliding window scheme. This produces longer, more coherent videos from any short-video diffusion model.
Technical specifications:
- Architecture: Noise Rescheduling + Sliding Window + Temporal Correlation
- Latent channels: 4
- Default: 64 frames at 8 FPS
- Supports I2V: Yes | T2V: Yes | V2V: No
Example
var options = new DiffusionModelOptions<float> { LatentChannels = 4, DefaultInferenceSteps = 50 };
var noise = Tensor<float>.CreateRandom(new[] { 1, 4, 64, 64, 64 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
.ConfigureModel(new FreeNoiseVideoModel<float>(options: options))
.Build(trainX, trainY);
var video = result.Predict(noise);
Constructors
| Constructor | Summary |
|---|---|
FreeNoiseVideoModel(NeuralNetworkArchitecture<T> architecture, DiffusionModelOptions<T> options, INoiseScheduler<T> scheduler, VideoUNetPredictor<T> predictor, TemporalVAE<T> temporalVAE, IConditioningModule<T> conditioner, int defaultNumFrames, int defaultFPS) |
Initializes a new instance of FreeNoiseVideoModel with full customization support. |
Properties
| Property | Summary |
|---|---|
IConditioningModule<T> Conditioner { get; } |
|
INoisePredictor<T> NoisePredictor { get; } |
|
IVAEModel<T> TemporalVAE { get; } |
|
IVAEModel<T> VAE { get; } |
|
bool SupportsImageToVideo { get; } |
|
bool SupportsTextToVideo { get; } |
|
bool SupportsVideoToVideo { get; } |
|
int LatentChannels { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |