TemporalShiftLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Temporal Shift Module: moves a fraction of the channels one step forward in time and another fraction one step backward, leaving the rest in place.
Inheritance: LayerBase<T> → TemporalShiftLayer<T>
Implements: IShapeContract
How It Works
Introduced by Lin et al. (TSM) and used as BSVD’s training-time temporal fusion in Qi et al., 2022, “Real-time Streaming Video Denoising with Bidirectional Buffers” (arXiv:2207.06937). BSVD swaps these blocks for Bidirectional Buffer Blocks at inference to obtain streaming operation; the two are equivalent in what they compute, differing only in whether the neighbouring frames come from a buffer or from the batch.
Per the paper, floor(C / r) channels shift per direction, with r = 8 the
default shifted-channel ratio. Shifting a minority of channels lets information cross frame
boundaries at zero parameter and near-zero FLOP cost, which is what makes temporal fusion
affordable for real-time denoising — without it the network sees each frame in isolation and
the model is a per-frame denoiser wearing a video name.
Shapes: rank-4 [T, C, H, W] or rank-5 [B, T, C, H, W]. Frames that
would shift in from outside the clip are zero-filled, matching the standard TSM boundary.
Gradient tracking: built from Engine.TensorNarrow and
Engine.TensorConcatenate, both of which stay on the autodiff tape, so the shift is
differentiable and carries gradients back to the frames it moved.
Constructors
| Constructor | Summary |
|---|---|
TemporalShiftLayer(int shiftedChannelRatio) |
Initializes a new temporal shift module. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
Derived from this type’s [TensorLayout] declarations. |
void ResetState() |