AdagradOptimizer<T, TInput, TOutput>

Models & Types · AiDotNet.Optimizers

Represents an Adagrad (Adaptive Gradient) optimizer for gradient-based optimization.

Inheritance: OptimizerBase<T, TInput, TOutput> → GradientBasedOptimizerBase<T, TInput, TOutput> → AdagradOptimizer<T, TInput, TOutput>

For Beginners

Adagrad is like a smart learning assistant that adjusts how much it learns for each piece of information based on how often it has seen similar information before.

  • It learns more from new or rare information
  • It learns less from common or frequently seen information
  • This helps it focus on the most important parts of what it’s learning

This can be especially useful when some parts of your data are more important or occur less frequently.

How It Works

The Adagrad optimizer adapts the learning rate for each parameter based on the historical gradients. It performs larger updates for infrequent parameters and smaller updates for frequent ones.

Example

using AiDotNet;
using AiDotNet.Data.Loaders;
using AiDotNet.Enums;
using AiDotNet.NeuralNetworks;
using AiDotNet.Optimizers;
using AiDotNet.Tensors.LinearAlgebra;

var rng = new Random(0);
var trainX = new Tensor<double>(new[] { 32, 8 });
var trainY = new Tensor<double>(new[] { 32, 2 });
for (int i = 0; i < 32; i++)
{
    for (int j = 0; j < 8; j++) trainX[new[] { i, j }] = rng.NextDouble();
    trainY[new[] { i, i % 2 }] = 1.0;
}

var model = new NeuralNetwork<double>(new NeuralNetworkArchitecture<double>(
    inputFeatures: 8, numClasses: 2, complexity: NetworkComplexity.Simple));

var result = await new AiModelBuilder<double, Tensor<double>, Tensor<double>>()
    .ConfigureModel(model)
    .ConfigureOptimizer(new AdagradOptimizer<double, Tensor<double>, Tensor<double>>(model))
    .ConfigureDataLoader(DataLoaders.FromTensors(trainX, trainY))
    .BuildAsync();

Console.WriteLine("Trained with AdagradOptimizer.");

Constructors

Constructor Summary
AdagradOptimizer(IFullModel<T, TInput, TOutput> model, AdagradOptimizerOptions<T, TInput, TOutput> options) Initializes a new instance of the AdagradOptimizer class.

Properties

Property Summary
bool SupportsGpuUpdate { get; } Gets whether this optimizer supports GPU-accelerated parameter updates.

Methods

Method Summary
AdagradOptimizer<T, TInput, TOutput> CreateForFunction(AdagradOptimizerOptions<T, TInput, TOutput> options) Creates an Adagrad optimizer for minimizing a plain function, with no model attached.
OptimizationAlgorithmOptions<T, TInput, TOutput> GetOptions() Retrieves the current options of the Adagrad optimizer.
OptimizationResult<T, TInput, TOutput> Optimize(OptimizationInputData<T, TInput, TOutput> inputData) Performs the optimization process using the Adagrad algorithm.
Vector<T> ReverseUpdate(Vector<T> updatedParameters, Vector<T> appliedGradients) Reverses an Adagrad gradient update to recover original parameters.
Vector<T> UpdateParameters(Vector<T> parameters, Vector<T> gradient) Updates a vector of parameters using the Adagrad optimization algorithm.
void DisposeGpuState() Disposes GPU-allocated optimizer state.
void InitializeGpuState(int parameterCount, IDirectGpuBackend backend) Initializes Adagrad optimizer state on the GPU.
void UpdateParametersGpu(IGpuBuffer parameters, IGpuBuffer gradients, int parameterCount, IDirectGpuBackend backend) Updates parameters on the GPU using the Adagrad kernel.