NewtonMethodOptimizer<T, TInput, TOutput>
Models & Types · AiDotNet.Optimizers
Implements the Newton’s Method optimization algorithm.
Inheritance: OptimizerBase<T, TInput, TOutput> → GradientBasedOptimizerBase<T, TInput, TOutput> → NewtonMethodOptimizer<T, TInput, TOutput>
For Beginners
Imagine you’re trying to find the lowest point in a valley. Gradient descent is like rolling a ball and letting it follow the slope. Newton’s Method is like using a telescope to look at the whole valley, predicting where the lowest point is, and jumping directly there. It’s often faster but requires more complex calculations at each step.
How It Works
Newton’s Method is a powerful optimization algorithm that uses both first and second derivatives of the objective function. It often converges faster than first-order methods, especially near the optimum, but can be computationally expensive due to the need to compute and invert the Hessian matrix.
Example
using AiDotNet;
using AiDotNet.Data.Loaders;
using AiDotNet.Enums;
using AiDotNet.NeuralNetworks;
using AiDotNet.Optimizers;
using AiDotNet.Tensors.LinearAlgebra;
var rng = new Random(0);
var trainX = new Tensor<double>(new[] { 32, 8 });
var trainY = new Tensor<double>(new[] { 32, 2 });
for (int i = 0; i < 32; i++)
{
for (int j = 0; j < 8; j++) trainX[new[] { i, j }] = rng.NextDouble();
trainY[new[] { i, i % 2 }] = 1.0;
}
var model = new NeuralNetwork<double>(new NeuralNetworkArchitecture<double>(
inputFeatures: 8, numClasses: 2, complexity: NetworkComplexity.Simple));
var result = await new AiModelBuilder<double, Tensor<double>, Tensor<double>>()
.ConfigureModel(model)
.ConfigureOptimizer(new NewtonMethodOptimizer<double, Tensor<double>, Tensor<double>>(model))
.ConfigureDataLoader(DataLoaders.FromTensors(trainX, trainY))
.BuildAsync();
Console.WriteLine("Trained with NewtonMethodOptimizer.");
Constructors
| Constructor | Summary |
|---|---|
NewtonMethodOptimizer(IFullModel<T, TInput, TOutput> model, NewtonMethodOptimizerOptions<T, TInput, TOutput> options) |
Initializes a new instance of the NewtonMethodOptimizer class. |
Methods
| Method | Summary |
|---|---|
NewtonMethodOptimizer<T, TInput, TOutput> CreateForFunction(NewtonMethodOptimizerOptions<T, TInput, TOutput> options) |
Creates a Newton’s method optimizer for minimizing a plain function, with no model attached. |
OptimizationAlgorithmOptions<T, TInput, TOutput> GetOptions() |
Gets the current options of the optimizer. |
OptimizationResult<T, TInput, TOutput> Optimize(OptimizationInputData<T, TInput, TOutput> inputData) |
Performs the optimization process using Newton’s Method algorithm. |
void UpdateParametersGpu(IGpuBuffer parameters, IGpuBuffer gradients, int parameterCount, IDirectGpuBackend backend) |
Updates parameters using GPU-accelerated Newton’s Method. |