FloRNN<T>
Models & Types · AiDotNet.Video.Denoising
FloRNN: unidirectional video denoising with a look-ahead recurrent module that mimics a backward one (Li et al., ECCV 2022).
Inheritance: NeuralNetworkBase<T> → VideoNeuralNetworkBase<T> → VideoDenoisingBase<T> → FloRNN<T>
For Beginners
FloRNN denoises video frame by frame, but it also looks a few frames into the future and warps what it saw there back to the current frame, getting most of the benefit of a two-way model while still running in a single pass.
How It Works
The network follows the authors’ release (github.com/nagejacob/FloRNN, models/flornn.py). Two recurrent modules, each a 3x3 convolution followed by residual blocks (conv-ReLU-conv plus identity, no biases), read the noisy frame, its noise-level map and the previous hidden state:
The sequence is reflected past its end so the last frames have frames to look ahead to. The paper’s flow is a pretrained PWC-Net that is never trained; here a SPyNet estimator plays that role and is likewise excluded from training (load pretrained weights into it for meaningful alignment). Weights are Kaiming-normal scaled by 0.2, as in the release. Training feeds the noise level the noise was drawn with (the RMS of noisy minus clean) and scores frames 1 to T-ForwardCount-1, as the authors’ script does; their summed MSE over 2N is a constant multiple of the mean used here.
Example
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.ThreeDimensional,
taskType: NeuralNetworkTaskType.Generative,
inputHeight: 96, inputWidth: 96, inputDepth: 3);
var flornn = new FloRNN<double>(architecture, new FloRNNOptions());
var clean = flornn.Denoise(noisyFrames, noiseLevel: 25.0 / 255.0); // [T, C, H, W] in [0, 1]
Constructors
| Constructor | Summary |
|---|---|
FloRNN(NeuralNetworkArchitecture<T> architecture, FloRNNOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a FloRNN model for native training and inference. |
FloRNN(NeuralNetworkArchitecture<T> architecture, string modelPath, FloRNNOptions options) |
Creates a FloRNN model for native training and inference. |
Methods
| Method | Summary |
|---|---|
LayerInputDomain GetInputDomain(int[] inputShape) |
The released FloRNN model consumes and emits normalized image tensors. |
LayerInputDomain GetOutputDomain(int[] outputShape) |
|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> Denoise(Tensor<T> noisyFrames) |
|
Tensor<T> Denoise(Tensor<T> noisyFrames, double noiseLevel) |
Denoises frames [T, C, H, W] (or [N, T, C, H, W]) corrupted by noise of the given standard deviation, the value the noise-level map carries. |
Tensor<T> ForwardForTraining(Tensor<T> input) |