VariationalStrategies<T>

Models & Types · AiDotNet.GaussianProcesses

Provides variational inference strategies for scalable Gaussian Process inference.

For Beginners

Standard GP inference scales as O(n³), which becomes impractical for large datasets. Variational inference provides approximations that scale better:

Key ideas:

  1. Introduce “inducing points” that summarize the data
  2. Approximate the true posterior with a simpler variational distribution
  3. Optimize the variational parameters to minimize KL divergence

Common strategies:

  • SVGP (Sparse Variational GP): Full variational approximation
  • FITC: Fully Independent Training Conditional
  • VFE: Variational Free Energy
  • KISS-GP: Kernel Interpolation for Scalable Structured GPs

Trade-offs:

  • More inducing points = better approximation but slower
  • Fewer inducing points = faster but less accurate

Constructors

Constructor Summary
VariationalStrategies() Initializes a new variational strategies helper.

Methods

Method Summary
Matrix<T> SelectInducingPointsGreedyVariance(Matrix<T> X, IKernelFunction<T> kernel, int numInducingPoints) Selects inducing points using greedy variance reduction.
Matrix<T> SelectInducingPointsKMeans(Matrix<T> X, int numInducingPoints, int maxIterations, int seed) Selects inducing points using k-means clustering on training data.
T ComputeELBO(Matrix<T> X, Vector<T> y, Matrix<T> Z, IKernelFunction<T> kernel, double noiseVariance, Vector<T> variationalMean, Matrix<T> variationalCovChol) Computes the ELBO (Evidence Lower BOund) for sparse GP.
ValueTuple<Vector<T>, Matrix<T>> InitializeVariationalParameters(Matrix<T> Z, Vector<T> y, Matrix<T> Kuf, double noiseVariance) Initializes variational parameters for SVGP.