RFFKernel<T>

Models & Types · AiDotNet.Kernels

Random Fourier Features (RFF) kernel for scalable approximation of shift-invariant kernels.

Implements: IKernelFunction<T>

For Beginners

Random Fourier Features is a clever technique for making Gaussian Processes scale to large datasets. Instead of computing the full N×N kernel matrix (which has O(N²) memory and O(N³) computation), RFF approximates the kernel with D random features.

The key insight (Bochner’s theorem): Any shift-invariant kernel can be written as the expectation of random cosine features:

k(x, x’) ≈ (1/D) Σᵢ cos(ωᵢ·x + bᵢ) × cos(ωᵢ·x’ + bᵢ) = φ(x)ᵀ × φ(x’)

Where:

  • ωᵢ are random frequencies drawn from the kernel’s spectral density
  • bᵢ are random phases drawn uniformly from [0, 2π]
  • φ(x) is a D-dimensional feature map

This transforms the GP kernel computation into a linear model:

  • Instead of O(N³) for exact GP, we get O(ND² + D³)
  • Memory goes from O(N²) to O(ND)

For D ≈ 1000-10000, the approximation is usually very good.

How It Works

Applications:

  • Large-scale GP regression (N > 10,000 points)
  • Online/streaming GP updates
  • Deep kernel learning (combining with neural networks)

Constructors

Constructor Summary
RFFKernel(int inputDim, int numFeatures, RFFKernelType<T> kernelType, double lengthScale, double outputScale, int? seed) Initializes a new RFF kernel.

Properties

Property Summary
RFFKernelType<T> KernelType { get; } Gets the kernel type being approximated.
double LengthScale { get; } Gets the length scale.
double OutputScale { get; } Gets the output scale.
int InputDim { get; } Gets the input dimensionality.
int NumFeatures { get; } Gets the number of random features.

Methods

Method Summary
Matrix<T> GetFeatureMatrix(Matrix<T> X) Computes the feature matrix for multiple input points.
T Calculate(Vector<T> x1, Vector<T> x2) Calculates the approximate kernel value between two vectors.
T EstimateApproximationError(Matrix<T> X, IKernelFunction<T> exactKernel) Estimates the approximation quality by comparing to exact kernel on test points.
Vector<T> GetFeatures(Vector<T> x) Computes the random Fourier feature map for an input vector.