PerParameterOptimizer<T, TInput, TOutput>

Models & Types · AiDotNet.MetaLearning.Algorithms

Per-parameter optimizer for Meta-SGD that learns individual optimization coefficients.

For Beginners

This is a special optimizer where each weight in the network gets its own set of optimization settings that are learned during meta-training.

How It Works

This optimizer maintains learned coefficients for each parameter:

  • Learning rates: α_i for each parameter
  • Momentum: β_i for each parameter (optional)
  • Direction: d_i for each parameter (optional)
  • Adam parameters: beta1, beta2, epsilon (if using Adam)

Constructors

Constructor Summary
PerParameterOptimizer(int numParameters, MetaSGDOptions<T, TInput, TOutput> options, IEngine engine) Initializes a new instance of the PerParameterOptimizer.

Properties

Property Summary
int NumParameters { get; } Gets the number of model parameters this optimizer manages.

Methods

Method Summary
PerParameterOptimizer<T, TInput, TOutput> Clone() Creates a deep copy of this per-parameter optimizer.
T GetLearningRate(int parameterIndex) Gets the learning rate for a specific parameter.
T UpdateParameter(int parameterIndex, T parameter, T gradient) Updates a single parameter using its learned optimization coefficients.
int GetMetaParameterCount() Gets the total number of meta-parameters being learned.
void SetAdamBeta1(int parameterIndex, T beta1) Sets Adam beta1 for a specific parameter.
void SetAdamBeta2(int parameterIndex, T beta2) Sets Adam beta2 for a specific parameter.
void SetAdamEpsilon(int parameterIndex, T epsilon) Sets Adam epsilon for a specific parameter.
void SetDirection(int parameterIndex, T direction) Sets the direction for a specific parameter.
void SetLearningRate(int parameterIndex, T learningRate) Sets the learning rate for a specific parameter.
void SetMomentum(int parameterIndex, T momentum) Sets the momentum for a specific parameter.
void UpdateMetaParameters(Vector<T> metaGradients) Updates the meta-parameters (learned coefficients) of the optimizer.