FrechetVideoDistance<T>

Models & Types · AiDotNet.Metrics

Fréchet Video Distance (FVD) - A metric for evaluating the quality of generated videos.

How It Works

FVD extends Fréchet Inception Distance (FID) to videos by using a 3D video feature extractor (typically Inflated 3D ConvNet, I3D) to compare the distribution of generated videos against real videos.

The algorithm:

  1. Extract spatiotemporal features from video clips using a 3D CNN
  2. Compute statistics (mean and covariance) for real and generated video features
  3. Compute the Fréchet distance between the two Gaussian distributions

Formula: FVD = ||mu_1 - mu_2||^2 + Tr(Sigma_1 + Sigma_2 - 2 * sqrt(Sigma_1 * Sigma_2))

Typical FVD scores:

  • FVD < 50: Excellent quality (hard to distinguish from real)
  • FVD 50-100: Good quality
  • FVD 100-300: Moderate quality
  • FVD > 300: Poor quality

Based on “Towards Accurate Generative Models of Video: A New Metric and Challenges” by Unterthiner et al. (2018)

Constructors

Constructor Summary
FrechetVideoDistance(ConvolutionalNeuralNetwork<T> featureNetwork, int featureDimension, int framesPerClip) Initializes a new instance of FVD calculator.

Properties

Property Summary
ConvolutionalNeuralNetwork<T> FeatureNetwork { get; } Gets the 3D feature extraction network (e.g., I3D) used for video representations.
FrameSamplingStrategy SamplingStrategy { get; set; } Gets or sets the frame sampling strategy.
int FeatureDimension { get; } Gets the dimensionality of extracted video features.
int FramesPerClip { get; set; } Gets or sets the number of frames per video clip for feature extraction.

Methods

Method Summary
T ComputeFVD(Tensor<T> realVideos, Tensor<T> generatedVideos) Computes the FVD score between real and generated videos.
T ComputeFVDWithStats(Vector<T> realMean, Matrix<T> realCov, Tensor<T> generatedVideos) Computes FVD using pre-computed statistics for the real distribution.
ValueTuple<Vector<T>, Matrix<T>> PrecomputeStatistics(Tensor<T> videos) Pre-computes statistics for a set of videos.