DFPOptimizer<T, TInput, TOutput>
Models & Types · AiDotNet.Optimizers
Implements the Davidon-Fletcher-Powell (DFP) optimization algorithm for numerical optimization problems.
Inheritance: OptimizerBase<T, TInput, TOutput> → GradientBasedOptimizerBase<T, TInput, TOutput> → DFPOptimizer<T, TInput, TOutput>
For Beginners
This optimizer is like a smart navigator that learns from its past steps to make better decisions about which direction to move in the future. It’s particularly good at handling complex optimization problems where the landscape of possible solutions is intricate.
How It Works
The DFP algorithm is a quasi-Newton method for solving unconstrained nonlinear optimization problems. It approximates the inverse Hessian matrix to determine the search direction, combining the efficiency of Newton’s method with the stability of gradient descent.
Example
using AiDotNet;
using AiDotNet.Data.Loaders;
using AiDotNet.Enums;
using AiDotNet.NeuralNetworks;
using AiDotNet.Optimizers;
using AiDotNet.Tensors.LinearAlgebra;
var rng = new Random(0);
var trainX = new Tensor<double>(new[] { 32, 8 });
var trainY = new Tensor<double>(new[] { 32, 2 });
for (int i = 0; i < 32; i++)
{
for (int j = 0; j < 8; j++) trainX[new[] { i, j }] = rng.NextDouble();
trainY[new[] { i, i % 2 }] = 1.0;
}
var model = new NeuralNetwork<double>(new NeuralNetworkArchitecture<double>(
inputFeatures: 8, numClasses: 2, complexity: NetworkComplexity.Simple));
var result = await new AiModelBuilder<double, Tensor<double>, Tensor<double>>()
.ConfigureModel(model)
.ConfigureOptimizer(new DFPOptimizer<double, Tensor<double>, Tensor<double>>(model))
.ConfigureDataLoader(DataLoaders.FromTensors(trainX, trainY))
.BuildAsync();
Console.WriteLine("Trained with DFPOptimizer.");
Constructors
| Constructor | Summary |
|---|---|
DFPOptimizer(IFullModel<T, TInput, TOutput> model, DFPOptimizerOptions<T, TInput, TOutput> options, IEngine engine) |
Initializes a new instance of the DFPOptimizer class. |
Methods
| Method | Summary |
|---|---|
DFPOptimizer<T, TInput, TOutput> CreateForFunction(DFPOptimizerOptions<T, TInput, TOutput> options) |
Creates a DFP optimizer for minimizing a plain function, with no model attached. |
OptimizationAlgorithmOptions<T, TInput, TOutput> GetOptions() |
Retrieves the current options of the DFP optimizer. |
OptimizationResult<T, TInput, TOutput> Optimize(OptimizationInputData<T, TInput, TOutput> inputData) |
Performs the main optimization process using the DFP algorithm. |
Vector<T> Minimize(Vector<T> initialParameters, Func<Vector<T>, ValueTuple<T, Vector<T>>> objectiveAndGradient, int maxIterations, T tolerance, Func<Vector<T>, Vector<T>> projection) |
|
Vector<T> UpdateParameters(Vector<T> parameters, Vector<T> gradient) |
Updates parameters using the DFP algorithm with inverse Hessian approximation. |
void UpdateParametersGpu(IGpuBuffer parameters, IGpuBuffer gradients, int parameterCount, IDirectGpuBackend backend) |
Updates parameters using GPU-accelerated DFP. |