StatisticalRejectionSampling<T, TInput, TOutput>
Models & Types · AiDotNet.FineTuning
Implements Statistical Rejection Sampling Optimization (RSO) for fine-tuning.
Inheritance: FineTuningBase<T, TInput, TOutput> → StatisticalRejectionSampling<T, TInput, TOutput>
For Beginners
RSO takes a list of responses ranked from best to worst and intelligently picks pairs to learn from. Instead of just using best vs worst, it uses statistical sampling to create more informative training examples.
How It Works
RSO uses rejection sampling to create high-quality training pairs from ranked outputs. It samples pairs based on the statistical properties of the ranking to create better preference data.
Original paper: “Statistical Rejection Sampling Improves Preference Optimization” by Liu et al. (2024)
Constructors
| Constructor | Summary |
|---|---|
StatisticalRejectionSampling(FineTuningOptions<T> options) |
Initializes a new instance of RSO fine-tuning. |
Properties
| Property | Summary |
|---|---|
FineTuningCategory Category { get; } |
|
bool RequiresReferenceModel { get; } |
|
bool RequiresRewardModel { get; } |
|
string MethodName { get; } |
Methods
| Method | Summary |
|---|---|
Task<FineTuningMetrics<T>> EvaluateAsync(IFullModel<T, TInput, TOutput> model, FineTuningData<T, TInput, TOutput> evaluationData, CancellationToken cancellationToken) |
|
Task<IFullModel<T, TInput, TOutput>> FineTuneAsync(IFullModel<T, TInput, TOutput> baseModel, FineTuningData<T, TInput, TOutput> trainingData, CancellationToken cancellationToken) |