FTRLOptimizer<T, TInput, TOutput>
Models & Types · AiDotNet.Optimizers
Represents a Follow The Regularized Leader (FTRL) optimizer for machine learning models.
Inheritance: OptimizerBase<T, TInput, TOutput> → GradientBasedOptimizerBase<T, TInput, TOutput> → FTRLOptimizer<T, TInput, TOutput>
For Beginners
FTRL is an advanced optimization technique that’s good at handling large-scale, sparse data. It’s often used in online advertising and recommendation systems.
Think of FTRL like a smart learning system that:
- Adjusts its learning speed for each feature independently
- Can handle situations where most features are zero (sparse data)
- Is good at balancing between finding a good solution and not overfitting
For example, in an online advertising system, FTRL might:
- Quickly learn which ad categories are important for a user
- Ignore or learn slowly from features that rarely appear
- Automatically adjust how much it learns from each new piece of data
This makes FTRL particularly good for systems that need to learn and predict in real-time with lots of data.
How It Works
The FTRL optimizer is an online learning algorithm that adapts regularization in a per-coordinate fashion. It’s particularly effective for sparse datasets and is widely used in click-through rate (CTR) prediction and other online learning scenarios.
Example
using AiDotNet;
using AiDotNet.Data.Loaders;
using AiDotNet.Enums;
using AiDotNet.NeuralNetworks;
using AiDotNet.Optimizers;
using AiDotNet.Tensors.LinearAlgebra;
var rng = new Random(0);
var trainX = new Tensor<double>(new[] { 32, 8 });
var trainY = new Tensor<double>(new[] { 32, 2 });
for (int i = 0; i < 32; i++)
{
for (int j = 0; j < 8; j++) trainX[new[] { i, j }] = rng.NextDouble();
trainY[new[] { i, i % 2 }] = 1.0;
}
var model = new NeuralNetwork<double>(new NeuralNetworkArchitecture<double>(
inputFeatures: 8, numClasses: 2, complexity: NetworkComplexity.Simple));
var result = await new AiModelBuilder<double, Tensor<double>, Tensor<double>>()
.ConfigureModel(model)
.ConfigureOptimizer(new FTRLOptimizer<double, Tensor<double>, Tensor<double>>(model))
.ConfigureDataLoader(DataLoaders.FromTensors(trainX, trainY))
.BuildAsync();
Console.WriteLine("Trained with FTRLOptimizer.");
Constructors
| Constructor | Summary |
|---|---|
FTRLOptimizer(IFullModel<T, TInput, TOutput> model, FTRLOptimizerOptions<T, TInput, TOutput> options, IEngine engine) |
Initializes a new instance of the FTRLOptimizer class. |
Methods
| Method | Summary |
|---|---|
FTRLOptimizer<T, TInput, TOutput> CreateForFunction(FTRLOptimizerOptions<T, TInput, TOutput> options) |
Creates an FTRL optimizer for minimizing a plain function, with no model attached. |
OptimizationAlgorithmOptions<T, TInput, TOutput> GetOptions() |
Retrieves the current options of the FTRL optimizer. |
OptimizationResult<T, TInput, TOutput> Optimize(OptimizationInputData<T, TInput, TOutput> inputData) |
Performs the main optimization process using the FTRL algorithm. |
Vector<T> ReverseUpdate(Vector<T> updatedParameters, Vector<T> appliedGradients) |
Reverses an FTRL gradient update to recover original parameters. |
Vector<T> UpdateParameters(Vector<T> parameters, Vector<T> gradient) |
Updates parameters using the FTRL (Follow The Regularized Leader) algorithm with L1/L2 regularization. |
void DisposeGpuState() |
Disposes GPU-allocated optimizer state. |
void InitializeGpuState(int parameterCount, IDirectGpuBackend backend) |
Initializes FTRL optimizer state on the GPU. |
void UpdateParametersGpu(IGpuBuffer parameters, IGpuBuffer gradients, int parameterCount, IDirectGpuBackend backend) |
Updates parameters on GPU using FTRL optimization. |