PairwiseRankingOptimization<T, TInput, TOutput>

Models & Types ยท AiDotNet.FineTuning

Implements Pairwise Ranking Optimization (PRO) for fine-tuning.

Inheritance: FineTuningBase<T, TInput, TOutput> โ†’ PairwiseRankingOptimization<T, TInput, TOutput>

For Beginners

PRO treats model alignment as a ranking problem. Given multiple responses, it learns to assign higher scores to better responses, using techniques from information retrieval and recommendation systems.

How It Works

PRO optimizes models using pairwise comparisons from ranked lists, similar to learning-to-rank algorithms used in search engines.

Key features:

  • Uses pairwise ranking loss (similar to RankNet)
  • Can leverage ranking information from multiple annotators
  • Robust to noise through margin-based learning

Constructors

Constructor Summary
PairwiseRankingOptimization(FineTuningOptions<T> options) Initializes a new instance of PRO fine-tuning.

Properties

Property Summary
FineTuningCategory Category { get; }
bool RequiresReferenceModel { get; }
bool RequiresRewardModel { get; }
string MethodName { get; }

Methods

Method Summary
Task<FineTuningMetrics<T>> EvaluateAsync(IFullModel<T, TInput, TOutput> model, FineTuningData<T, TInput, TOutput> evaluationData, CancellationToken cancellationToken)
Task<IFullModel<T, TInput, TOutput>> FineTuneAsync(IFullModel<T, TInput, TOutput> baseModel, FineTuningData<T, TInput, TOutput> trainingData, CancellationToken cancellationToken)