TrustRegionOptimizer<T, TInput, TOutput>
Models & Types · AiDotNet.Optimizers
Implements the Trust Region optimization algorithm for machine learning models.
Inheritance: OptimizerBase<T, TInput, TOutput> → GradientBasedOptimizerBase<T, TInput, TOutput> → TrustRegionOptimizer<T, TInput, TOutput>
For Beginners
Think of this optimizer as an explorer with a map:
- The “trust region” is like the area on the map the explorer trusts to be accurate.
- In each step, the explorer looks at this trusted area to decide where to go next.
- If the predictions (map) match reality well, the explorer might expand the trusted area.
- If the predictions are off, the explorer shrinks the trusted area and becomes more cautious.
This approach helps the optimizer make good decisions even in complex landscapes, balancing between making progress and staying reliable.
Example
using AiDotNet;
using AiDotNet.Data.Loaders;
using AiDotNet.Enums;
using AiDotNet.NeuralNetworks;
using AiDotNet.Optimizers;
using AiDotNet.Tensors.LinearAlgebra;
var rng = new Random(0);
var trainX = new Tensor<double>(new[] { 32, 8 });
var trainY = new Tensor<double>(new[] { 32, 2 });
for (int i = 0; i < 32; i++)
{
for (int j = 0; j < 8; j++) trainX[new[] { i, j }] = rng.NextDouble();
trainY[new[] { i, i % 2 }] = 1.0;
}
var model = new NeuralNetwork<double>(new NeuralNetworkArchitecture<double>(
inputFeatures: 8, numClasses: 2, complexity: NetworkComplexity.Simple));
var result = await new AiModelBuilder<double, Tensor<double>, Tensor<double>>()
.ConfigureModel(model)
.ConfigureOptimizer(new TrustRegionOptimizer<double, Tensor<double>, Tensor<double>>(model))
.ConfigureDataLoader(DataLoaders.FromTensors(trainX, trainY))
.BuildAsync();
Console.WriteLine("Trained with TrustRegionOptimizer.");
Constructors
| Constructor | Summary |
|---|---|
TrustRegionOptimizer(IFullModel<T, TInput, TOutput> model, TrustRegionOptimizerOptions<T, TInput, TOutput> options, IEngine engine) |
Initializes a new instance of the TrustRegionOptimizer class. |
Methods
| Method | Summary |
|---|---|
OptimizationAlgorithmOptions<T, TInput, TOutput> GetOptions() |
Retrieves the current options of the optimizer. |
OptimizationResult<T, TInput, TOutput> Optimize(OptimizationInputData<T, TInput, TOutput> inputData) |
Performs the optimization process using the Trust Region algorithm. |
TrustRegionOptimizer<T, TInput, TOutput> CreateForFunction(TrustRegionOptimizerOptions<T, TInput, TOutput> options) |
Creates a trust region optimizer for minimizing a plain function, with no model attached. |
Vector<T> UpdateParameters(Vector<T> parameters, Vector<T> gradient) |
Updates parameters using a simplified Cauchy-point Trust Region step. |
void UpdateParametersGpu(IGpuBuffer parameters, IGpuBuffer gradients, int parameterCount, IDirectGpuBackend backend) |
GPU parameter update is handled by the tape-based Step(TapeStepContext) method. |