MiniBatchGradientDescentOptimizer<T, TInput, TOutput>

Models & Types · AiDotNet.Optimizers

Implements the Mini-Batch Gradient Descent optimization algorithm.

Inheritance: OptimizerBase<T, TInput, TOutput> → GradientBasedOptimizerBase<T, TInput, TOutput> → MiniBatchGradientDescentOptimizer<T, TInput, TOutput>

For Beginners

Imagine you’re trying to find the bottom of a valley while blindfolded. Mini-Batch Gradient Descent is like taking a few steps, checking your position, adjusting your direction, and repeating. It’s faster than checking after every single step (Stochastic Gradient Descent) but more precise than taking a lot of steps before checking (Batch Gradient Descent).

How It Works

Mini-Batch Gradient Descent is a variation of gradient descent that splits the training data into small batches to calculate model error and update model coefficients. This approach strikes a balance between the efficiency of stochastic gradient descent and the stability of batch gradient descent.

Example

using AiDotNet;
using AiDotNet.Data.Loaders;
using AiDotNet.Enums;
using AiDotNet.NeuralNetworks;
using AiDotNet.Optimizers;
using AiDotNet.Tensors.LinearAlgebra;

var rng = new Random(0);
var trainX = new Tensor<double>(new[] { 32, 8 });
var trainY = new Tensor<double>(new[] { 32, 2 });
for (int i = 0; i < 32; i++)
{
    for (int j = 0; j < 8; j++) trainX[new[] { i, j }] = rng.NextDouble();
    trainY[new[] { i, i % 2 }] = 1.0;
}

var model = new NeuralNetwork<double>(new NeuralNetworkArchitecture<double>(
    inputFeatures: 8, numClasses: 2, complexity: NetworkComplexity.Simple));

var result = await new AiModelBuilder<double, Tensor<double>, Tensor<double>>()
    .ConfigureModel(model)
    .ConfigureOptimizer(new MiniBatchGradientDescentOptimizer<double, Tensor<double>, Tensor<double>>(model))
    .ConfigureDataLoader(DataLoaders.FromTensors(trainX, trainY))
    .BuildAsync();

Console.WriteLine("Trained with MiniBatchGradientDescentOptimizer.");

Constructors

Constructor Summary
MiniBatchGradientDescentOptimizer(IFullModel<T, TInput, TOutput> model, MiniBatchGradientDescentOptions<T, TInput, TOutput> options, IEngine engine) Initializes a new instance of the MiniBatchGradientDescentOptimizer class.

Methods

Method Summary
MiniBatchGradientDescentOptimizer<T, TInput, TOutput> CreateForFunction(MiniBatchGradientDescentOptions<T, TInput, TOutput> options) Creates a mini-batch gradient descent optimizer for minimizing a plain function, with no model attached.
OptimizationAlgorithmOptions<T, TInput, TOutput> GetOptions() Gets the current optimization algorithm options.
OptimizationResult<T, TInput, TOutput> Optimize(OptimizationInputData<T, TInput, TOutput> inputData) Performs the optimization process using Mini-Batch Gradient Descent.
Vector<T> ReverseUpdate(Vector<T> updatedParameters, Vector<T> appliedGradients) Reverses a Mini-Batch Gradient Descent update to recover original parameters.
void UpdateParametersGpu(IGpuBuffer parameters, IGpuBuffer gradients, int parameterCount, IDirectGpuBackend backend) Updates parameters on the GPU using vanilla SGD (same as SGD for parameter updates).