FlowLens<T>
Models & Types · AiDotNet.Video.Inpainting
FlowLens optical-flow-guided video inpainting with flow completion.
Inheritance: NeuralNetworkBase<T> → VideoNeuralNetworkBase<T> → VideoInpaintingBase<T> → FlowLens<T>
For Beginners
FlowLens performs video inpainting by using optical flow as a lens to guide content from visible regions into masked areas. It produces temporally consistent fills for removed objects.
How It Works
References:
- Paper: “FlowLens: Seeing Beyond the FoV via Optical Flow Completion” (Xu et al., ECCV 2022)
FlowLens decouples motion estimation from pixel synthesis by first completing optical flow in masked regions, then using the completed flow for temporal propagation of known pixels, followed by a refinement network for remaining holes, achieving sharp and temporally consistent inpainting.
Example
// Create a FlowLens model for optical-flow-guided video inpainting
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.ThreeDimensional,
taskType: NeuralNetworkTaskType.Generative,
inputHeight: 256, inputWidth: 256, inputDepth: 3);
var options = new FlowLensOptions();
var flowLens = new FlowLens<double>(architecture, options);
// Or load a pre-trained ONNX model for inference
var flowLensOnnx = new FlowLens<double>(architecture, "flowlens_model.onnx");
Constructors
| Constructor | Summary |
|---|---|
FlowLens(NeuralNetworkArchitecture<T> architecture, FlowLensOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a FlowLens model for native training and inference. |
FlowLens(NeuralNetworkArchitecture<T> architecture, string modelPath, FlowLensOptions options) |
Creates a FlowLens model for native training and inference. |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> ForwardForTraining(Tensor<T> input) |
|
Tensor<T> Inpaint(Tensor<T> frames, Tensor<T> masks) |