RFFKernel<T>
Models & Types · AiDotNet.Kernels
Random Fourier Features (RFF) kernel for scalable approximation of shift-invariant kernels.
Implements: IKernelFunction<T>
For Beginners
Random Fourier Features is a clever technique for making Gaussian Processes scale to large datasets. Instead of computing the full N×N kernel matrix (which has O(N²) memory and O(N³) computation), RFF approximates the kernel with D random features.
The key insight (Bochner’s theorem): Any shift-invariant kernel can be written as the expectation of random cosine features:
k(x, x’) ≈ (1/D) Σᵢ cos(ωᵢ·x + bᵢ) × cos(ωᵢ·x’ + bᵢ) = φ(x)ᵀ × φ(x’)
Where:
- ωᵢ are random frequencies drawn from the kernel’s spectral density
- bᵢ are random phases drawn uniformly from [0, 2π]
- φ(x) is a D-dimensional feature map
This transforms the GP kernel computation into a linear model:
- Instead of O(N³) for exact GP, we get O(ND² + D³)
- Memory goes from O(N²) to O(ND)
For D ≈ 1000-10000, the approximation is usually very good.
How It Works
Applications:
- Large-scale GP regression (N > 10,000 points)
- Online/streaming GP updates
- Deep kernel learning (combining with neural networks)
Constructors
| Constructor | Summary |
|---|---|
RFFKernel(int inputDim, int numFeatures, RFFKernelType<T> kernelType, double lengthScale, double outputScale, int? seed) |
Initializes a new RFF kernel. |
Properties
| Property | Summary |
|---|---|
RFFKernelType<T> KernelType { get; } |
Gets the kernel type being approximated. |
double LengthScale { get; } |
Gets the length scale. |
double OutputScale { get; } |
Gets the output scale. |
int InputDim { get; } |
Gets the input dimensionality. |
int NumFeatures { get; } |
Gets the number of random features. |
Methods
| Method | Summary |
|---|---|
Matrix<T> GetFeatureMatrix(Matrix<T> X) |
Computes the feature matrix for multiple input points. |
T Calculate(Vector<T> x1, Vector<T> x2) |
Calculates the approximate kernel value between two vectors. |
T EstimateApproximationError(Matrix<T> X, IKernelFunction<T> exactKernel) |
Estimates the approximation quality by comparing to exact kernel on test points. |
Vector<T> GetFeatures(Vector<T> x) |
Computes the random Fourier feature map for an input vector. |