KahnemanTverskyOptimization<T, TInput, TOutput>
Models & Types · AiDotNet.FineTuning
Implements Kahneman-Tversky Optimization (KTO) for fine-tuning.
Inheritance: FineTuningBase<T, TInput, TOutput> → KahnemanTverskyOptimization<T, TInput, TOutput>
For Beginners
KTO is based on how humans actually make decisions. People feel losses more strongly than equivalent gains (loss aversion). KTO uses this insight to train models - making them more careful about avoiding bad outputs than they are eager to produce good ones.
How It Works
KTO applies prospect theory from behavioral economics to preference learning. Unlike DPO, it doesn’t require paired data - each example is independently labeled as desirable or undesirable.
Key features:
- Doesn’t require paired preference data
- Uses loss aversion (undesirable weight typically higher than desirable weight)
- More sample efficient for imbalanced datasets
Original paper: “KTO: Model Alignment as Prospect Theoretic Optimization” by Ethayarajh et al. (2024)
Constructors
| Constructor | Summary |
|---|---|
KahnemanTverskyOptimization(FineTuningOptions<T> options) |
Initializes a new instance of KTO fine-tuning. |
Properties
| Property | Summary |
|---|---|
FineTuningCategory Category { get; } |
|
bool RequiresReferenceModel { get; } |
|
bool RequiresRewardModel { get; } |
|
string MethodName { get; } |
Methods
| Method | Summary |
|---|---|
Task<FineTuningMetrics<T>> EvaluateAsync(IFullModel<T, TInput, TOutput> model, FineTuningData<T, TInput, TOutput> evaluationData, CancellationToken cancellationToken) |
|
Task<IFullModel<T, TInput, TOutput>> FineTuneAsync(IFullModel<T, TInput, TOutput> baseModel, FineTuningData<T, TInput, TOutput> trainingData, CancellationToken cancellationToken) |