FuSta<T>

Models & Types · AiDotNet.Video.Stabilization

FuSta hybrid full-frame video stabilization with warping and outpainting.

Inheritance: NeuralNetworkBase<T> → VideoNeuralNetworkBase<T> → VideoStabilizationBase<T> → FuSta<T>

For Beginners

FuSta (Fusion Stabilization) stabilizes video by fusing multiple stabilization strategies including trajectory smoothing and homography warping for robust results.

How It Works

References:

  • Paper: “FuSta: Hybrid Approach for Full-frame Video Stabilization” (Liu et al., 2021)

FuSta achieves full-frame stabilization through a two-stage approach: first warping frames using optical-flow-based motion compensation, then using a neural outpainting network to fill missing border regions, avoiding the field-of-view loss of traditional cropping.

Example

// Create a FuSta model for full-frame video stabilization
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.ThreeDimensional,
    taskType: NeuralNetworkTaskType.Generative,
    inputHeight: 256, inputWidth: 256, inputDepth: 3);
var options = new FuStaOptions();
var fuSta = new FuSta<double>(architecture, options);
            
// Or load a pre-trained ONNX model for inference
var fuStaOnnx = new FuSta<double>(architecture, "fusta_model.onnx");

Constructors

Constructor Summary
FuSta(NeuralNetworkArchitecture<T> architecture, FuStaOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Creates a FuSta model for native training and inference.
FuSta(NeuralNetworkArchitecture<T> architecture, string modelPath, FuStaOptions options) Creates a FuSta model for native training and inference.

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()
Tensor<T> Stabilize(Tensor<T> unstableFrames)