GradientDescentOptimizer<T, TInput, TOutput>
Models & Types · AiDotNet.Optimizers
Represents a Gradient Descent optimizer for machine learning models.
Inheritance: OptimizerBase<T, TInput, TOutput> → GradientBasedOptimizerBase<T, TInput, TOutput> → GradientDescentOptimizer<T, TInput, TOutput>
For Beginners
Imagine you’re trying to find the lowest point in a valley:
- You start at a random point (initial model parameters)
- You look around to see which way is steepest downhill (calculate the gradient)
- You take a step in that direction (update the parameters)
- You repeat this process until you reach the bottom of the valley (optimize the model)
This optimizer helps the model learn by gradually adjusting its parameters to minimize errors.
How It Works
Gradient Descent is a first-order iterative optimization algorithm for finding a local minimum of a differentiable function. It takes steps proportional to the negative of the gradient of the function at the current point.
Example
using AiDotNet;
using AiDotNet.Data.Loaders;
using AiDotNet.Enums;
using AiDotNet.NeuralNetworks;
using AiDotNet.Optimizers;
using AiDotNet.Tensors.LinearAlgebra;
var rng = new Random(0);
var trainX = new Tensor<double>(new[] { 32, 8 });
var trainY = new Tensor<double>(new[] { 32, 2 });
for (int i = 0; i < 32; i++)
{
for (int j = 0; j < 8; j++) trainX[new[] { i, j }] = rng.NextDouble();
trainY[new[] { i, i % 2 }] = 1.0;
}
var model = new NeuralNetwork<double>(new NeuralNetworkArchitecture<double>(
inputFeatures: 8, numClasses: 2, complexity: NetworkComplexity.Simple));
var result = await new AiModelBuilder<double, Tensor<double>, Tensor<double>>()
.ConfigureModel(model)
.ConfigureOptimizer(new GradientDescentOptimizer<double, Tensor<double>, Tensor<double>>(model))
.ConfigureDataLoader(DataLoaders.FromTensors(trainX, trainY))
.BuildAsync();
Console.WriteLine("Trained with GradientDescentOptimizer.");
Constructors
| Constructor | Summary |
|---|---|
GradientDescentOptimizer(IFullModel<T, TInput, TOutput> model, GradientDescentOptimizerOptions<T, TInput, TOutput> options, IEngine engine) |
Initializes a new instance of the GradientDescentOptimizer class. |
Methods
| Method | Summary |
|---|---|
GradientDescentOptimizer<T, TInput, TOutput> CreateForFunction(GradientDescentOptimizerOptions<T, TInput, TOutput> options) |
Creates a gradient descent optimizer for minimizing a plain function, with no model attached. |
OptimizationAlgorithmOptions<T, TInput, TOutput> GetOptions() |
Retrieves the current options for the Gradient Descent optimizer. |
OptimizationResult<T, TInput, TOutput> Optimize(OptimizationInputData<T, TInput, TOutput> inputData) |
Performs the main optimization process using the Gradient Descent algorithm. |
Vector<T> ReverseUpdate(Vector<T> updatedParameters, Vector<T> appliedGradients) |
|
void UpdateParametersGpu(IGpuBuffer parameters, IGpuBuffer gradients, int parameterCount, IDirectGpuBackend backend) |
Updates parameters on the GPU using vanilla SGD. |