RVM<T>
Models & Types · AiDotNet.Video.Matting
RVM: Robust Video Matting for real-time human segmentation.
Inheritance: NeuralNetworkBase<T> → RVM<T>
For Beginners
RVM extracts people from video backgrounds in real-time. Unlike simple background removal that creates hard edges, matting produces a soft alpha matte that preserves hair details and semi-transparent regions.
Key capabilities:
- Real-time video matting without green screen
- High-quality alpha matte output
- Temporal consistency across frames
- Works with any background
Outputs:
- Alpha matte: Transparency at each pixel (0=background, 1=foreground)
- Foreground: RGB colors of the person with pre-multiplied alpha
Example usage:
How It Works
Technical Details:
- A four-stage stride-2 encoder (a ReLU6 conv stack standing in for MobileNetV3) with an LR-ASPP head
- The paper’s recurrent decoder: a ConvGRU at the bottleneck and in each of the three upsampling blocks, on half
of the channels, carrying temporal state from frame to frame (reset with ResetState)
- Output: alpha and a foreground residual added to the source frame, both clamped to [0, 1]
- The deep guided filter for high-resolution refinement is not included; frames are matted at input resolution
Reference: “Robust High-Resolution Video Matting with Temporal Guidance”
https://arxiv.org/abs/2108.11515
Constructors
| Constructor | Summary |
|---|---|
RVM() |
Initializes a new instance with default architecture settings. |
RVM(NeuralNetworkArchitecture<T> architecture, RVMOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
|
RVM(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, RVMOptions options) |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
Methods
| Method | Summary |
|---|---|
List<ValueTuple<Tensor<T>, Tensor<T>>> MatteVideo(List<Tensor<T>> frames) |
Processes a video to extract alpha mattes and foregrounds. |
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> CompositeWithBackground(Tensor<T> foreground, Tensor<T> alpha, Tensor<T> background) |
Composites the foreground onto a new background. |
Tensor<T> ForwardForTraining(Tensor<T> input) |
|
Tensor<T> GetAlpha(Tensor<T> frame) |
Extracts just the alpha matte. |
Tensor<T> GreenScreenExtract(Tensor<T> frame) |
Creates a green screen effect (extracts foreground). |
ValueTuple<Tensor<T>, Tensor<T>> MatteSingleFrame(Tensor<T> frame) |
Mattes a single frame, maintaining temporal consistency with previous frames. |
void ResetState() |
Resets the recurrent state for a new video. |