SelfPlayFineTuning<T, TInput, TOutput>

Models & Types · AiDotNet.FineTuning

Implements Self-Play Fine-Tuning (SPIN) for model improvement.

Inheritance: FineTuningBase<T, TInput, TOutput> → SelfPlayFineTuning<T, TInput, TOutput>

For Beginners

SPIN is like a model playing chess against itself to get better. The current model learns to prefer real human responses over its own generated responses, iteratively improving without new labeled data.

How It Works

SPIN uses self-play to improve models without additional human-labeled data. The model plays against previous versions of itself, learning to distinguish its own outputs from ground truth.

Key features:

  • No additional human labels needed after initial SFT
  • Iterative improvement through self-play
  • Uses DPO-style objective with model’s own outputs as negatives

Original paper: “Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models” by Chen et al. (2024)

Constructors

Constructor Summary
SelfPlayFineTuning(FineTuningOptions<T> options) Initializes a new instance of SPIN fine-tuning.

Properties

Property Summary
FineTuningCategory Category { get; }
bool RequiresReferenceModel { get; }
bool RequiresRewardModel { get; }
string MethodName { get; }

Methods

Method Summary
Task<FineTuningMetrics<T>> EvaluateAsync(IFullModel<T, TInput, TOutput> model, FineTuningData<T, TInput, TOutput> evaluationData, CancellationToken cancellationToken)
Task<IFullModel<T, TInput, TOutput>> FineTuneAsync(IFullModel<T, TInput, TOutput> baseModel, FineTuningData<T, TInput, TOutput> trainingData, CancellationToken cancellationToken)