HyperparameterOptimizer<T>
Models & Types · AiDotNet.GaussianProcesses
Provides hyperparameter optimization for Gaussian Processes.
For Beginners
Gaussian Processes have hyperparameters (like kernel length scale, signal variance, and noise variance) that greatly affect performance. This class helps find good values for these hyperparameters automatically.
Methods available:
- Grid Search: Try all combinations from a predefined grid
- Random Search: Try random combinations (often more efficient than grid)
- Gradient Descent: Follow gradients of log marginal likelihood
- Bayesian Optimization: Use a GP to model the objective (meta!)
The optimization target is typically the log marginal likelihood (LML):
- Higher LML = better fit to data (accounting for model complexity)
- LML naturally balances fit quality with model simplicity
Constructors
| Constructor | Summary |
|---|---|
HyperparameterOptimizer(int maxIterations, double tolerance, int randomSeed) |
Initializes a new hyperparameter optimizer. |
Methods
| Method | Summary |
|---|---|
HyperparameterResult<T> GradientDescent(Dictionary<string, double> initialParameters, Func<Dictionary<string, double>, ValueTuple<double, Dictionary<string, double>>> evaluateWithGradient, double learningRate, Dictionary<string, ValueTuple<double, double>> parameterBounds) |
Performs gradient-based optimization of hyperparameters. |
HyperparameterResult<T> GridSearch(Dictionary<string, double[]> parameterGrid, Func<Dictionary<string, double>, double> evaluateFunction) |
Performs grid search over hyperparameter values. |
HyperparameterResult<T> MultiStartOptimization(Dictionary<string, ValueTuple<double, double>> parameterRanges, Func<Dictionary<string, double>, ValueTuple<double, Dictionary<string, double>>> evaluateWithGradient, int numRestarts, double learningRate) |
Performs multi-start optimization with random initializations. |
HyperparameterResult<T> RandomSearch(Dictionary<string, ValueTuple<double, double>> parameterRanges, Func<Dictionary<string, double>, double> evaluateFunction, int numSamples, string[] logScale) |
Performs random search over hyperparameter ranges. |
ValueTuple<double, double> CrossValidate(Matrix<T> X, Vector<T> y, Func<Dictionary<string, double>, ValueTuple<Func<Matrix<T>, Vector<T>, ValueTuple<Vector<T>, Vector<T>>>, Action<Matrix<T>, Vector<T>>>> createModel, Dictionary<string, double> parameters, int numFolds) |
Performs cross-validation to estimate generalization performance. |