RobustDirectPreferenceOptimization<T, TInput, TOutput>
Models & Types ยท AiDotNet.FineTuning
Implements Robust Direct Preference Optimization (RDPO) for fine-tuning.
Inheritance: FineTuningBase<T, TInput, TOutput> โ RobustDirectPreferenceOptimization<T, TInput, TOutput>
For Beginners
Real-world preference data often contains mistakes - annotators sometimes disagree or make errors. RDPO handles this by being more cautious about uncertain preferences and focusing on high-confidence examples.
How It Works
RDPO extends DPO to handle noisy or inconsistent preference labels by incorporating label confidence and noise-aware training.
Key features:
- Confidence-weighted preference learning
- Robust to label noise and annotator disagreement
- Uses sample weights to down-weight uncertain examples
Constructors
| Constructor | Summary |
|---|---|
RobustDirectPreferenceOptimization(FineTuningOptions<T> options) |
Initializes a new instance of RDPO fine-tuning. |
Properties
| Property | Summary |
|---|---|
FineTuningCategory Category { get; } |
|
bool RequiresReferenceModel { get; } |
|
bool RequiresRewardModel { get; } |
|
string MethodName { get; } |
Methods
| Method | Summary |
|---|---|
Task<FineTuningMetrics<T>> EvaluateAsync(IFullModel<T, TInput, TOutput> model, FineTuningData<T, TInput, TOutput> evaluationData, CancellationToken cancellationToken) |
|
Task<IFullModel<T, TInput, TOutput>> FineTuneAsync(IFullModel<T, TInput, TOutput> baseModel, FineTuningData<T, TInput, TOutput> trainingData, CancellationToken cancellationToken) |