SparseVariationalGaussianProcess<T>

Models & Types · AiDotNet.GaussianProcesses

Implements a Sparse Variational Gaussian Process (SVGP) for scalable GP regression.

Inheritance: ModelBase<T, Matrix<T>, Vector<T>> → GaussianProcessBase<T> → SparseVariationalGaussianProcess<T>

Implements: IGeneratedParameterRegistrar<T>

For Beginners

Standard Gaussian Processes are powerful but slow - they require O(n³) computation time, making them impractical for datasets larger than a few thousand points.

SVGP solves this problem using two key ideas:

  1. Inducing Points: Instead of using all training points, we use a smaller set of

“representative” points (inducing points) that summarize the data. If we have n training points but only m inducing points (where m << n), computation becomes O(nm²) instead of O(n³).

  1. Variational Inference: We approximate the true posterior distribution with a simpler

distribution (variational distribution) that’s easier to work with. We optimize this approximation to be as close as possible to the true posterior.

The result is a GP that can handle millions of data points while still providing uncertainty estimates and probabilistic predictions.

How It Works

When to use SVGP:

  • Large datasets (thousands to millions of points)
  • When you need mini-batch training (can’t fit all data in memory)
  • When you want uncertainty estimates but standard GP is too slow
  • When inducing points can reasonably summarize your data

Limitations:

  • The approximation quality depends on the number and placement of inducing points
  • More hyperparameters to tune compared to standard GP
  • May require more iterations to converge

Constructors

Constructor Summary
SparseVariationalGaussianProcess(IKernelFunction<T> kernel, int numInducingPoints, double noiseVariance, double learningRate, int maxIterations, MatrixDecompositionType decompositionType) Initializes a new instance of the SparseVariationalGaussianProcess class.

Methods

Method Summary
T ComputeELBO() Computes the Evidence Lower Bound (ELBO) for the current variational approximation.
ValueTuple<T, T> Predict(Vector<T> x)
void Fit(Matrix<T> X, Vector<T> y) Trains the SVGP model using variational inference and gradient-based optimization.
void UpdateKernel(IKernelFunction<T> kernel)