PairwiseRankingOptimization<T, TInput, TOutput>
Models & Types ยท AiDotNet.FineTuning
Implements Pairwise Ranking Optimization (PRO) for fine-tuning.
Inheritance: FineTuningBase<T, TInput, TOutput> โ PairwiseRankingOptimization<T, TInput, TOutput>
For Beginners
PRO treats model alignment as a ranking problem. Given multiple responses, it learns to assign higher scores to better responses, using techniques from information retrieval and recommendation systems.
How It Works
PRO optimizes models using pairwise comparisons from ranked lists, similar to learning-to-rank algorithms used in search engines.
Key features:
- Uses pairwise ranking loss (similar to RankNet)
- Can leverage ranking information from multiple annotators
- Robust to noise through margin-based learning
Constructors
| Constructor | Summary |
|---|---|
PairwiseRankingOptimization(FineTuningOptions<T> options) |
Initializes a new instance of PRO fine-tuning. |
Properties
| Property | Summary |
|---|---|
FineTuningCategory Category { get; } |
|
bool RequiresReferenceModel { get; } |
|
bool RequiresRewardModel { get; } |
|
string MethodName { get; } |
Methods
| Method | Summary |
|---|---|
Task<FineTuningMetrics<T>> EvaluateAsync(IFullModel<T, TInput, TOutput> model, FineTuningData<T, TInput, TOutput> evaluationData, CancellationToken cancellationToken) |
|
Task<IFullModel<T, TInput, TOutput>> FineTuneAsync(IFullModel<T, TInput, TOutput> baseModel, FineTuningData<T, TInput, TOutput> trainingData, CancellationToken cancellationToken) |