Figan<T>
Models & Types · AiDotNet.Video.FrameInterpolation
FIGAN: frame interpolation with multi-scale deep loss functions and a generative adversarial network.
Inheritance: NeuralNetworkBase<T> → VideoNeuralNetworkBase<T> → FrameInterpolationBase<T> → Figan<T>
For Beginners
Given two consecutive video frames, this invents the frame in between. It first guesses how things moved — roughly at low resolution, then sharpening — then drags both frames towards the middle and decides pixel by pixel which one to trust. A second network acts as a critic, judging whether the invented frame looks real, which pushes the result to be sharp rather than blurry.
How It Works
van Amersfoort, Shi, Acosta, Massa, Totz, Wang and Caballero, “Frame Interpolation with Multi-Scale Deep Loss Functions and Generative Adversarial Networks” (arXiv:1711.06045).
REPLACES TDPNet. That class cited arXiv:2404.05765 as “TDPNet: Temporal Difference Prediction
Network for Video Frame Interpolation” by Pengcheng Lei and Fei Gao. That identifier is “A Novel
Bi-LSTM And Transformer Architecture For Generating Tabla Music” — Indian percussion music
generation. The title, authors, year and subject were all invented, and no TDPNet paper appears to
exist. Its documented mechanisms (“temporal difference prediction”, “difference-aware attention”,
a DifferenceThreshold) had no published basis either.
The paper’s three contributions, and where each lives:
The generator is built from the paper’s module template (Table 1): six 3x3 convolutions,
N_i -> 32, four 32 -> 32 ReLU layers, then 32 -> N_o. The discriminator is
eight blocks of convolution + batch normalisation + leaky ReLU with strides alternating 2 and 1 and
the filter count doubling at each stride-2 block.
Example
var arch = new NeuralNetworkArchitecture<double>(
inputType: InputType.ThreeDimensional,
taskType: NeuralNetworkTaskType.Regression,
inputHeight: 128, inputWidth: 128, inputDepth: 3);
var model = new Figan<double>(arch, new FiganOptions { NumScales = 3 });
var frame0 = Tensor<double>.CreateRandom(1, 3, 128, 128);
var frame1 = Tensor<double>.CreateRandom(1, 3, 128, 128);
var middle = model.Interpolate(frame0, frame1);
Constructors
| Constructor | Summary |
|---|---|
Figan(NeuralNetworkArchitecture<T> architecture, FiganOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a FIGAN model in native training mode. |
Figan(NeuralNetworkArchitecture<T> architecture, string modelPath, FiganOptions options) |
Creates a FIGAN model in native training mode. |
Properties
| Property | Summary |
|---|---|
FiganMultiScaleFlow<T> Flow { get; } |
Gets the multi-scale flow estimator and synthesiser. |
IReadOnlyList<ILayer<T>> DiscriminatorLayers { get; } |
Gets the discriminator layers, or an empty list in ONNX mode. |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> Interpolate(Tensor<T> frame0, Tensor<T> frame1, double t) |