StatisticalRejectionSampling<T, TInput, TOutput>

Models & Types · AiDotNet.FineTuning

Implements Statistical Rejection Sampling Optimization (RSO) for fine-tuning.

Inheritance: FineTuningBase<T, TInput, TOutput> → StatisticalRejectionSampling<T, TInput, TOutput>

For Beginners

RSO takes a list of responses ranked from best to worst and intelligently picks pairs to learn from. Instead of just using best vs worst, it uses statistical sampling to create more informative training examples.

How It Works

RSO uses rejection sampling to create high-quality training pairs from ranked outputs. It samples pairs based on the statistical properties of the ranking to create better preference data.

Original paper: “Statistical Rejection Sampling Improves Preference Optimization” by Liu et al. (2024)

Constructors

Constructor Summary
StatisticalRejectionSampling(FineTuningOptions<T> options) Initializes a new instance of RSO fine-tuning.

Properties

Property Summary
FineTuningCategory Category { get; }
bool RequiresReferenceModel { get; }
bool RequiresRewardModel { get; }
string MethodName { get; }

Methods

Method Summary
Task<FineTuningMetrics<T>> EvaluateAsync(IFullModel<T, TInput, TOutput> model, FineTuningData<T, TInput, TOutput> evaluationData, CancellationToken cancellationToken)
Task<IFullModel<T, TInput, TOutput>> FineTuneAsync(IFullModel<T, TInput, TOutput> baseModel, FineTuningData<T, TInput, TOutput> trainingData, CancellationToken cancellationToken)