FrameSamplingVideoModerator<T>

Models & Types · AiDotNet.Safety.Video

Video content moderator built on Temporal Segment Networks: sparse segment-wise sampling with a segmental consensus over snippet scores, producing one video-level verdict per category.

Inheritance: ModelBase<T, Vector<T>, Vector<T>> → SafetyModuleBase<T> → VideoSafetyModuleBase<T> → FrameSamplingVideoModerator<T>

Implements: IGeneratedParameterRegistrar<T>

For Beginners

Instead of checking a frame every second — which examines 10 frames of a 10-second clip and 600 of a 10-minute one — this splits any video into a few equal chunks and looks at one frame from each. The separate opinions are then averaged into a single score for the whole video before deciding. That makes the decision stable: one odd-looking frame no longer condemns an entire video, and a genuinely problematic video is caught because several chunks agree.

How It Works

Wang et al., “Temporal Segment Networks: Towards Good Practices for Deep Action Recognition” (ECCV 2016, arXiv:1608.00859). The video is divided into K segments of EQUAL DURATION and one snippet is sampled from each; a shared classifier scores every snippet; a segmental consensus combines those scores into a single video-level score, which is only then thresholded. In the paper’s notation (Eq. 1):

TSN(T_1, ..., T_K) = H(G(F(T_1; W), ..., F(T_K; W)))

where F is the shared-weight classifier applied to snippet T_k, G is the segmental consensus and H is the prediction. The ordering is the substance of the method: consensus is applied BEFORE the prediction function, so the model is supervised at video level rather than frame level.

Why this differs from fixed-rate sampling. Sampling at a fixed FPS makes the number of examined frames scale with duration, so a long video is judged on many more samples than a short one, and each frame is decided independently — one twitchy frame flags the whole video. Segment -wise sampling examines a CONSTANT K snippets spread evenly across the video however long it is, which is what “sparse temporal sampling” means, and consensus-before-decision means the verdict reflects the video as a whole. The paper’s motivation is exactly this: dense frame-level sampling is both redundant and unable to model long-range structure.

Sampling at inference. The paper samples snippets randomly within each segment during training and uses fixed sampling at test time. This module is inference-only and must be reproducible — the same video has to yield the same verdict every time it is scanned — so it takes the CENTRE frame of each segment.

Constructors

Constructor Summary
FrameSamplingVideoModerator(int segmentCount, SegmentalConsensus consensus, double nsfwThreshold, double violenceThreshold) Initializes a TSN-style video moderator.

Properties

Property Summary
SegmentalConsensus Consensus { get; } Gets the segmental consensus function used to combine snippet scores.
int LastSampledFrameCount { get; set; } Number of frames the last EvaluateVideo call actually classified.
int SegmentCount { get; } Gets the number of equal-duration segments the video is divided into.
string ModuleName { get; }

Methods

Method Summary
IReadOnlyList<SafetyFinding> EvaluateVideo(IReadOnlyList<Tensor<T>> frames, double frameRate)

Fields

Field Summary
int DefaultSegmentCount Number of segments K used when the caller does not specify one.