BFGSOptimizer<T, TInput, TOutput>
Models & Types · AiDotNet.Optimizers
Implements the Broyden-Fletcher-Goldfarb-Shanno (BFGS) optimization algorithm.
Inheritance: OptimizerBase<T, TInput, TOutput> → GradientBasedOptimizerBase<T, TInput, TOutput> → BFGSOptimizer<T, TInput, TOutput>
For Beginners
BFGS is an advanced optimization algorithm that tries to find the best solution by making smart steps based on the function’s behavior. It’s particularly good at handling complex problems where the function being optimized is smooth but potentially has many variables.
How It Works
BFGS is a quasi-Newton method for solving unconstrained nonlinear optimization problems. It approximates the Hessian matrix of second derivatives of the function to be minimized.
Example
using AiDotNet;
using AiDotNet.Data.Loaders;
using AiDotNet.Enums;
using AiDotNet.NeuralNetworks;
using AiDotNet.Optimizers;
using AiDotNet.Tensors.LinearAlgebra;
var rng = new Random(0);
var trainX = new Tensor<double>(new[] { 32, 8 });
var trainY = new Tensor<double>(new[] { 32, 2 });
for (int i = 0; i < 32; i++)
{
for (int j = 0; j < 8; j++) trainX[new[] { i, j }] = rng.NextDouble();
trainY[new[] { i, i % 2 }] = 1.0;
}
var model = new NeuralNetwork<double>(new NeuralNetworkArchitecture<double>(
inputFeatures: 8, numClasses: 2, complexity: NetworkComplexity.Simple));
var result = await new AiModelBuilder<double, Tensor<double>, Tensor<double>>()
.ConfigureModel(model)
.ConfigureOptimizer(new BFGSOptimizer<double, Tensor<double>, Tensor<double>>(model))
.ConfigureDataLoader(DataLoaders.FromTensors(trainX, trainY))
.BuildAsync();
Console.WriteLine("Trained with BFGSOptimizer.");
Constructors
| Constructor | Summary |
|---|---|
BFGSOptimizer(IFullModel<T, TInput, TOutput> model, BFGSOptimizerOptions<T, TInput, TOutput> options, IEngine engine) |
Initializes a new instance of the BFGSOptimizer class. |
Methods
| Method | Summary |
|---|---|
BFGSOptimizer<T, TInput, TOutput> CreateForFunction(BFGSOptimizerOptions<T, TInput, TOutput> options) |
Creates a BFGS optimizer for minimizing a plain function, with no model attached. |
OptimizationAlgorithmOptions<T, TInput, TOutput> GetOptions() |
Gets the current options for the BFGS optimizer. |
OptimizationResult<T, TInput, TOutput> Optimize(OptimizationInputData<T, TInput, TOutput> inputData) |
Performs the main optimization process using the BFGS algorithm. |
Vector<T> Minimize(Vector<T> initialParameters, Func<Vector<T>, ValueTuple<T, Vector<T>>> objectiveAndGradient, int maxIterations, T tolerance, Func<Vector<T>, Vector<T>> projection) |
|
Vector<T> UpdateParameters(Vector<T> parameters, Vector<T> gradient) |
Updates parameters using the BFGS algorithm with inverse Hessian approximation. |
void UpdateParametersGpu(IGpuBuffer parameters, IGpuBuffer gradients, int parameterCount, IDirectGpuBackend backend) |
Updates parameters using GPU-accelerated BFGS. |