SelfPlayFineTuning<T, TInput, TOutput>
Models & Types · AiDotNet.FineTuning
Implements Self-Play Fine-Tuning (SPIN) for model improvement.
Inheritance: FineTuningBase<T, TInput, TOutput> → SelfPlayFineTuning<T, TInput, TOutput>
For Beginners
SPIN is like a model playing chess against itself to get better. The current model learns to prefer real human responses over its own generated responses, iteratively improving without new labeled data.
How It Works
SPIN uses self-play to improve models without additional human-labeled data. The model plays against previous versions of itself, learning to distinguish its own outputs from ground truth.
Key features:
- No additional human labels needed after initial SFT
- Iterative improvement through self-play
- Uses DPO-style objective with model’s own outputs as negatives
Original paper: “Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models” by Chen et al. (2024)
Constructors
| Constructor | Summary |
|---|---|
SelfPlayFineTuning(FineTuningOptions<T> options) |
Initializes a new instance of SPIN fine-tuning. |
Properties
| Property | Summary |
|---|---|
FineTuningCategory Category { get; } |
|
bool RequiresReferenceModel { get; } |
|
bool RequiresRewardModel { get; } |
|
string MethodName { get; } |
Methods
| Method | Summary |
|---|---|
Task<FineTuningMetrics<T>> EvaluateAsync(IFullModel<T, TInput, TOutput> model, FineTuningData<T, TInput, TOutput> evaluationData, CancellationToken cancellationToken) |
|
Task<IFullModel<T, TInput, TOutput>> FineTuneAsync(IFullModel<T, TInput, TOutput> baseModel, FineTuningData<T, TInput, TOutput> trainingData, CancellationToken cancellationToken) |