LBFGSOptimizer<T, TInput, TOutput>

Models & Types · AiDotNet.Optimizers

Implements the Limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) optimization algorithm.

Inheritance: OptimizerBase<T, TInput, TOutput> → GradientBasedOptimizerBase<T, TInput, TOutput> → LBFGSOptimizer<T, TInput, TOutput>

For Beginners

L-BFGS is an advanced optimization algorithm that efficiently finds the minimum of a function, especially useful for problems with many variables. It uses information from previous iterations to make intelligent decisions about where to search next, while keeping memory usage low.

How It Works

L-BFGS is a quasi-Newton method for solving unconstrained nonlinear optimization problems. It approximates the Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm using a limited amount of computer memory, making it suitable for optimization problems with many variables.

Example

using AiDotNet;
using AiDotNet.Data.Loaders;
using AiDotNet.Enums;
using AiDotNet.NeuralNetworks;
using AiDotNet.Optimizers;
using AiDotNet.Tensors.LinearAlgebra;

var rng = new Random(0);
var trainX = new Tensor<double>(new[] { 32, 8 });
var trainY = new Tensor<double>(new[] { 32, 2 });
for (int i = 0; i < 32; i++)
{
    for (int j = 0; j < 8; j++) trainX[new[] { i, j }] = rng.NextDouble();
    trainY[new[] { i, i % 2 }] = 1.0;
}

var model = new NeuralNetwork<double>(new NeuralNetworkArchitecture<double>(
    inputFeatures: 8, numClasses: 2, complexity: NetworkComplexity.Simple));

var result = await new AiModelBuilder<double, Tensor<double>, Tensor<double>>()
    .ConfigureModel(model)
    .ConfigureOptimizer(new LBFGSOptimizer<double, Tensor<double>, Tensor<double>>(model))
    .ConfigureDataLoader(DataLoaders.FromTensors(trainX, trainY))
    .BuildAsync();

Console.WriteLine("Trained with LBFGSOptimizer.");

Constructors

Constructor Summary
LBFGSOptimizer(IFullModel<T, TInput, TOutput> model, LBFGSOptimizerOptions<T, TInput, TOutput> options, IEngine engine) Initializes a new instance of the LBFGSOptimizer class.

Methods

Method Summary
LBFGSOptimizer<T, TInput, TOutput> CreateForFunction(LBFGSOptimizerOptions<T, TInput, TOutput> options) Creates an L-BFGS optimizer for minimizing a plain function, with no model attached.
OptimizationAlgorithmOptions<T, TInput, TOutput> GetOptions() Retrieves the current options of the optimizer.
OptimizationResult<T, TInput, TOutput> Optimize(OptimizationInputData<T, TInput, TOutput> inputData) Performs the main optimization process using the L-BFGS algorithm.
Vector<T> Minimize(Vector<T> initialParameters, Func<Vector<T>, ValueTuple<T, Vector<T>>> objectiveAndGradient, int maxIterations, T tolerance, Func<Vector<T>, Vector<T>> projection) Calculates the search direction using the L-BFGS algorithm.
Vector<T> UpdateParameters(Vector<T> parameters, Vector<T> gradient) Updates parameters using the L-BFGS algorithm.
void UpdateParametersGpu(IGpuBuffer parameters, IGpuBuffer gradients, int parameterCount, IDirectGpuBackend backend) Updates parameters using GPU-accelerated L-BFGS.