TemporalShiftLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

Temporal Shift Module: moves a fraction of the channels one step forward in time and another fraction one step backward, leaving the rest in place.

Inheritance: LayerBase<T> → TemporalShiftLayer<T>

Implements: IShapeContract

How It Works

Introduced by Lin et al. (TSM) and used as BSVD’s training-time temporal fusion in Qi et al., 2022, “Real-time Streaming Video Denoising with Bidirectional Buffers” (arXiv:2207.06937). BSVD swaps these blocks for Bidirectional Buffer Blocks at inference to obtain streaming operation; the two are equivalent in what they compute, differing only in whether the neighbouring frames come from a buffer or from the batch.

Per the paper, floor(C / r) channels shift per direction, with r = 8 the default shifted-channel ratio. Shifting a minority of channels lets information cross frame boundaries at zero parameter and near-zero FLOP cost, which is what makes temporal fusion affordable for real-time denoising — without it the network sees each frame in isolation and the model is a per-frame denoiser wearing a video name.

Shapes: rank-4 [T, C, H, W] or rank-5 [B, T, C, H, W]. Frames that would shift in from outside the clip are zero-filled, matching the standard TSM boundary.

Gradient tracking: built from Engine.TensorNarrow and Engine.TensorConcatenate, both of which stay on the autodiff tape, so the shift is differentiable and carries gradients back to the frames it moved.

Constructors

Constructor Summary
TemporalShiftLayer(int shiftedChannelRatio) Initializes a new temporal shift module.

Properties

Property Summary
bool SupportsTraining { get; }

Methods

Method Summary
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) Derived from this type’s [TensorLayout] declarations.
void ResetState()