RobustDirectPreferenceOptimization<T, TInput, TOutput>

Models & Types ยท AiDotNet.FineTuning

Implements Robust Direct Preference Optimization (RDPO) for fine-tuning.

Inheritance: FineTuningBase<T, TInput, TOutput> โ†’ RobustDirectPreferenceOptimization<T, TInput, TOutput>

For Beginners

Real-world preference data often contains mistakes - annotators sometimes disagree or make errors. RDPO handles this by being more cautious about uncertain preferences and focusing on high-confidence examples.

How It Works

RDPO extends DPO to handle noisy or inconsistent preference labels by incorporating label confidence and noise-aware training.

Key features:

  • Confidence-weighted preference learning
  • Robust to label noise and annotator disagreement
  • Uses sample weights to down-weight uncertain examples

Constructors

Constructor Summary
RobustDirectPreferenceOptimization(FineTuningOptions<T> options) Initializes a new instance of RDPO fine-tuning.

Properties

Property Summary
FineTuningCategory Category { get; }
bool RequiresReferenceModel { get; }
bool RequiresRewardModel { get; }
string MethodName { get; }

Methods

Method Summary
Task<FineTuningMetrics<T>> EvaluateAsync(IFullModel<T, TInput, TOutput> model, FineTuningData<T, TInput, TOutput> evaluationData, CancellationToken cancellationToken)
Task<IFullModel<T, TInput, TOutput>> FineTuneAsync(IFullModel<T, TInput, TOutput> baseModel, FineTuningData<T, TInput, TOutput> trainingData, CancellationToken cancellationToken)