SpyNetLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
SPyNet (Spatial Pyramid Network) layer for optical flow estimation.
Inheritance: LayerBase<T> → SpyNetLayer<T>
Implements: IShapeContract
For Beginners
Optical flow tells us how pixels move between two frames. SPyNet is a lightweight network that estimates this motion efficiently by processing the images at multiple scales (pyramid levels).
The network works by:
- Building image pyramids at different resolutions
- Estimating flow at the coarsest level first
- Refining the flow at each finer level
- Combining all levels for the final flow
How It Works
SPyNet uses a coarse-to-fine spatial pyramid approach to estimate optical flow between two consecutive video frames. It’s widely used in video super-resolution and frame interpolation models.
Reference: Ranjan and Black, “Optical Flow Estimation using a Spatial Pyramid Network”,
CVPR 2017. https://arxiv.org/abs/1611.00850
Constructors
| Constructor | Summary |
|---|---|
SpyNetLayer(int numLevels) |
Creates a new SPyNet layer for optical flow estimation. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
|
int NumLevels { get; } |
The number of pyramid levels; inputs must be at least 2^(levels-1) pixels on each side. |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
Tensor<T> EstimateFlow(Tensor<T> frame1, Tensor<T> frame2) |
Estimates optical flow between two frames using separate tensors. |
int[] GetInputShape() |
|
int[] GetOutputShape() |
Gets the output shape for this layer (2 channels for optical flow: dx, dy). |
void ClearGradients() |
|
void ResetState() |
|
void UpdateParameters(T learningRate) |