ProPainter<T>

Models & Types · AiDotNet.Video.Inpainting

ProPainter for video inpainting - removes unwanted objects and fills regions in video.

Inheritance: NeuralNetworkBase<T> → VideoNeuralNetworkBase<T> → VideoInpaintingBase<T> → ProPainter<T>

For Beginners

ProPainter is a state-of-the-art model for video inpainting, which means removing unwanted objects from video and filling the resulting holes with realistic content. This is useful for:

  • Removing watermarks or logos from video
  • Removing unwanted people or objects
  • Repairing damaged video footage
  • Creating special effects

Unlike image inpainting, video inpainting needs to maintain temporal consistency across frames to avoid flickering and artifacts.

How It Works

Technical Details:

  • Dual-domain propagation (image + flow domains)
  • Recurrent flow completion for temporal consistency
  • Mask-guided sparse convolution
  • Transformer-based global feature aggregation

Reference: Zhou et al., “ProPainter: Improving Propagation and Transformer for Video Inpainting” ICCV 2023.

Example

// Create a ProPainter model for video object removal and inpainting
var proPainter = new ProPainter<double>();
            
// Or configure with custom transformer and feature parameters
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.ThreeDimensional,
    taskType: NeuralNetworkTaskType.Regression,
    inputHeight: 256, inputWidth: 256, inputDepth: 3, outputSize: 2);
var model = new ProPainter<double>(architecture, options: new ProPainterOptions { NumFeatures = 128, NumTransformerBlocks = 6 });

Constructors

Constructor Summary
ProPainter() Initializes a new instance of the ProPainter class.
ProPainter(NeuralNetworkArchitecture<T> architecture, ProPainterOptions options)

Properties

Property Summary
bool SupportsTraining { get; } Gets whether training is supported.

Methods

Method Summary
List<Tensor<T>> Inpaint(List<Tensor<T>> frames, List<Tensor<T>> masks)
List<Tensor<T>> RemoveObject(List<Tensor<T>> frames, Tensor<T> objectMask) Removes an object specified by mask from video frames.
ModelMetadata<T> GetModelMetadata()
Tensor<T> ForwardForTraining(Tensor<T> input)
Tensor<T> Inpaint(Tensor<T> frames, Tensor<T> masks)