IGradientComputable<T, TInput, TOutput>

Interfaces · AiDotNet.Interfaces

Base interface for models that can compute gradients explicitly without updating parameters.

Known implementations: A2CAgent<T>, A3CAgent<T>, CQLAgent<T>, ClusteringBase<T>, DDPGAgent<T>, DQNAgent<T>, DecisionTransformerAgent<T>, DoubleDQNAgent<T>, DoubleQLearningAgent<T>, EveryVisitMonteCarloAgent<T>, ExpectedSARSAAgent<T>, FinancialA2CAgent<T>, FinancialDQNAgent<T>, FinancialPPOAgent<T>, FinancialSACAgent<T>, FirstVisitMonteCarloAgent<T>, IAudioEventDetector<T>, IAudioGenerator<T>, ICausalModel<T>, IDiffusionModel<T>, IGenreClassifier<T>, IMultiLabelClassifier<T>, IMusicSourceSeparator<T>, INeuralNetwork<T>, INoisePredictor<T>, IOnlineLearningModel<T>, IQLAgent<T>, ISceneClassifier<T>, ISecondOrderGradientComputable<T, TInput, TOutput>, ISegmentationModel<T>, ISpeakerDiarizer<T>, ISpeakerVerifier<T>, ISpeechRecognizer<T>, ITextToSpeech<T>, IVAEModel<T>, LinearClassifierBase<T>, MarketMakingAgent<T>, ModelBase<T, TInput, TOutput>, ModelWrapperBase<T, TInput, TOutput>, NStepQLearningAgent<T>, NStepSARSAAgent<T>, NonLinearRegressionBase<T>, OrdinalLogisticRegression<T>, OrdinalRegression<T>, OrdinalRidgeRegression<T>, PPOAgent<T>, QMIXAgent<T>, REINFORCEAgent<T>, RainbowDQNAgent<T>, RegressionBase<T>, SACAgent<T>, SARSAAgent<T>, SelfTrainingClassifier<T>, ShardedModelBase<T, TInput, TOutput>, TD3Agent<T>, TRPOAgent<T>, TabularQLearningAgent<T>

For Beginners

Regular training computes gradients and immediately updates the model in one step. This interface separates those two operations:

  1. ComputeGradients - Calculate which direction improves the model (WITHOUT changing it)
  2. ApplyGradients - Actually update the model using those directions

This separation is crucial when you need to process gradients before applying them, such as averaging gradients across multiple GPUs in distributed training.

How It Works

This interface enables models to compute gradients without immediately applying parameter updates. This is essential for:

  • Distributed Training: Compute local gradients, synchronize across workers, then apply averaged gradients
  • Meta-Learning: Compute gradients on query sets after adaptation (see ISecondOrderGradientComputable)
  • Custom Optimization: Manually control when and how to apply gradients
  • Gradient Analysis: Inspect gradient values for debugging or monitoring

Distributed Training Use Case: In Data Parallel training (DDP), each GPU:

  1. Computes gradients on its local data batch
  2. Communicates gradients with other GPUs to compute the average
  3. Applies the averaged gradients to update parameters

Without this interface, step 2 would be impossible because gradients would already be applied in step 1.

Methods

Method Summary
Vector<T> ComputeGradients(TInput input, TOutput target, ILossFunction<T> lossFunction) Computes gradients of the loss function with respect to model parameters for the given data, WITHOUT updating the model parameters.
void ApplyGradients(Vector<T> gradients, T learningRate) Applies pre-computed gradients to update the model parameters.