FrameSamplingVideoModerator<T>
Models & Types · AiDotNet.Safety.Video
Video content moderator built on Temporal Segment Networks: sparse segment-wise sampling with a segmental consensus over snippet scores, producing one video-level verdict per category.
Inheritance: ModelBase<T, Vector<T>, Vector<T>> → SafetyModuleBase<T> → VideoSafetyModuleBase<T> → FrameSamplingVideoModerator<T>
Implements: IGeneratedParameterRegistrar<T>
For Beginners
Instead of checking a frame every second — which examines 10 frames of a 10-second clip and 600 of a 10-minute one — this splits any video into a few equal chunks and looks at one frame from each. The separate opinions are then averaged into a single score for the whole video before deciding. That makes the decision stable: one odd-looking frame no longer condemns an entire video, and a genuinely problematic video is caught because several chunks agree.
How It Works
Wang et al., “Temporal Segment Networks: Towards Good Practices for Deep Action Recognition”
(ECCV 2016, arXiv:1608.00859). The video is divided into K segments of EQUAL DURATION and
one snippet is sampled from each; a shared classifier scores every snippet; a segmental consensus
combines those scores into a single video-level score, which is only then thresholded. In the
paper’s notation (Eq. 1):
TSN(T_1, ..., T_K) = H(G(F(T_1; W), ..., F(T_K; W)))
where F is the shared-weight classifier applied to snippet T_k, G is the
segmental consensus and H is the prediction. The ordering is the substance of the method:
consensus is applied BEFORE the prediction function, so the model is supervised at video level
rather than frame level.
Why this differs from fixed-rate sampling. Sampling at a fixed FPS makes the number of examined frames scale with duration, so a long video is judged on many more samples than a short one, and each frame is decided independently — one twitchy frame flags the whole video. Segment -wise sampling examines a CONSTANT K snippets spread evenly across the video however long it is, which is what “sparse temporal sampling” means, and consensus-before-decision means the verdict reflects the video as a whole. The paper’s motivation is exactly this: dense frame-level sampling is both redundant and unable to model long-range structure.
Sampling at inference. The paper samples snippets randomly within each segment during training and uses fixed sampling at test time. This module is inference-only and must be reproducible — the same video has to yield the same verdict every time it is scanned — so it takes the CENTRE frame of each segment.
Constructors
| Constructor | Summary |
|---|---|
FrameSamplingVideoModerator(int segmentCount, SegmentalConsensus consensus, double nsfwThreshold, double violenceThreshold) |
Initializes a TSN-style video moderator. |
Properties
| Property | Summary |
|---|---|
SegmentalConsensus Consensus { get; } |
Gets the segmental consensus function used to combine snippet scores. |
int LastSampledFrameCount { get; set; } |
Number of frames the last EvaluateVideo call actually classified. |
int SegmentCount { get; } |
Gets the number of equal-duration segments the video is divided into. |
string ModuleName { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<SafetyFinding> EvaluateVideo(IReadOnlyList<Tensor<T>> frames, double frameRate) |
Fields
| Field | Summary |
|---|---|
int DefaultSegmentCount |
Number of segments K used when the caller does not specify one. |