FrechetVideoDistance<T>
Models & Types · AiDotNet.Metrics
Fréchet Video Distance (FVD) - A metric for evaluating the quality of generated videos.
How It Works
FVD extends Fréchet Inception Distance (FID) to videos by using a 3D video feature extractor (typically Inflated 3D ConvNet, I3D) to compare the distribution of generated videos against real videos.
The algorithm:
- Extract spatiotemporal features from video clips using a 3D CNN
- Compute statistics (mean and covariance) for real and generated video features
- Compute the Fréchet distance between the two Gaussian distributions
Formula: FVD = ||mu_1 - mu_2||^2 + Tr(Sigma_1 + Sigma_2 - 2 * sqrt(Sigma_1 * Sigma_2))
Typical FVD scores:
- FVD < 50: Excellent quality (hard to distinguish from real)
- FVD 50-100: Good quality
- FVD 100-300: Moderate quality
- FVD > 300: Poor quality
Based on “Towards Accurate Generative Models of Video: A New Metric and Challenges” by Unterthiner et al. (2018)
Constructors
| Constructor | Summary |
|---|---|
FrechetVideoDistance(ConvolutionalNeuralNetwork<T> featureNetwork, int featureDimension, int framesPerClip) |
Initializes a new instance of FVD calculator. |
Properties
| Property | Summary |
|---|---|
ConvolutionalNeuralNetwork<T> FeatureNetwork { get; } |
Gets the 3D feature extraction network (e.g., I3D) used for video representations. |
FrameSamplingStrategy SamplingStrategy { get; set; } |
Gets or sets the frame sampling strategy. |
int FeatureDimension { get; } |
Gets the dimensionality of extracted video features. |
int FramesPerClip { get; set; } |
Gets or sets the number of frames per video clip for feature extraction. |
Methods
| Method | Summary |
|---|---|
T ComputeFVD(Tensor<T> realVideos, Tensor<T> generatedVideos) |
Computes the FVD score between real and generated videos. |
T ComputeFVDWithStats(Vector<T> realMean, Matrix<T> realCov, Tensor<T> generatedVideos) |
Computes FVD using pre-computed statistics for the real distribution. |
ValueTuple<Vector<T>, Matrix<T>> PrecomputeStatistics(Tensor<T> videos) |
Pre-computes statistics for a set of videos. |