IGradientComputable<T, TInput, TOutput>
Interfaces · AiDotNet.Interfaces
Base interface for models that can compute gradients explicitly without updating parameters.
Known implementations: A2CAgent<T>, A3CAgent<T>, CQLAgent<T>, ClusteringBase<T>, DDPGAgent<T>, DQNAgent<T>, DecisionTransformerAgent<T>, DoubleDQNAgent<T>, DoubleQLearningAgent<T>, EveryVisitMonteCarloAgent<T>, ExpectedSARSAAgent<T>, FinancialA2CAgent<T>, FinancialDQNAgent<T>, FinancialPPOAgent<T>, FinancialSACAgent<T>, FirstVisitMonteCarloAgent<T>, IAudioEventDetector<T>, IAudioGenerator<T>, ICausalModel<T>, IDiffusionModel<T>, IGenreClassifier<T>, IMultiLabelClassifier<T>, IMusicSourceSeparator<T>, INeuralNetwork<T>, INoisePredictor<T>, IOnlineLearningModel<T>, IQLAgent<T>, ISceneClassifier<T>, ISecondOrderGradientComputable<T, TInput, TOutput>, ISegmentationModel<T>, ISpeakerDiarizer<T>, ISpeakerVerifier<T>, ISpeechRecognizer<T>, ITextToSpeech<T>, IVAEModel<T>, LinearClassifierBase<T>, MarketMakingAgent<T>, ModelBase<T, TInput, TOutput>, ModelWrapperBase<T, TInput, TOutput>, NStepQLearningAgent<T>, NStepSARSAAgent<T>, NonLinearRegressionBase<T>, OrdinalLogisticRegression<T>, OrdinalRegression<T>, OrdinalRidgeRegression<T>, PPOAgent<T>, QMIXAgent<T>, REINFORCEAgent<T>, RainbowDQNAgent<T>, RegressionBase<T>, SACAgent<T>, SARSAAgent<T>, SelfTrainingClassifier<T>, ShardedModelBase<T, TInput, TOutput>, TD3Agent<T>, TRPOAgent<T>, TabularQLearningAgent<T>
For Beginners
Regular training computes gradients and immediately updates the model in one step. This interface separates those two operations:
ComputeGradients- Calculate which direction improves the model (WITHOUT changing it)ApplyGradients- Actually update the model using those directions
This separation is crucial when you need to process gradients before applying them, such as averaging gradients across multiple GPUs in distributed training.
How It Works
This interface enables models to compute gradients without immediately applying parameter updates. This is essential for:
- Distributed Training: Compute local gradients, synchronize across workers, then apply averaged gradients
- Meta-Learning: Compute gradients on query sets after adaptation (see
ISecondOrderGradientComputable) - Custom Optimization: Manually control when and how to apply gradients
- Gradient Analysis: Inspect gradient values for debugging or monitoring
Distributed Training Use Case: In Data Parallel training (DDP), each GPU:
- Computes gradients on its local data batch
- Communicates gradients with other GPUs to compute the average
- Applies the averaged gradients to update parameters
Without this interface, step 2 would be impossible because gradients would already be applied in step 1.
Methods
| Method | Summary |
|---|---|
Vector<T> ComputeGradients(TInput input, TOutput target, ILossFunction<T> lossFunction) |
Computes gradients of the loss function with respect to model parameters for the given data, WITHOUT updating the model parameters. |
void ApplyGradients(Vector<T> gradients, T learningRate) |
Applies pre-computed gradients to update the model parameters. |