Fine Tuning

16 public types in this namespace, organized by kind.

Models & Types (15)

Type Summary
ConstitutionalAIFineTuning<T, TInput, TOutput> Implements Constitutional AI (CAI) fine-tuning.
ContrastivePreferenceOptimization<T, TInput, TOutput> Implements Contrastive Preference Optimization (CPO) for fine-tuning.
DirectPreferenceOptimization<T, TInput, TOutput> Implements Direct Preference Optimization (DPO) for fine-tuning.
GroupRelativePolicyOptimization<T, TInput, TOutput> Implements Group Relative Policy Optimization (GRPO) for fine-tuning.
IdentityPreferenceOptimization<T, TInput, TOutput> Implements Identity Preference Optimization (IPO) for fine-tuning.
KahnemanTverskyOptimization<T, TInput, TOutput> Implements Kahneman-Tversky Optimization (KTO) for fine-tuning.
OddsRatioPreferenceOptimization<T, TInput, TOutput> Implements Odds Ratio Preference Optimization (ORPO) for fine-tuning.
PairwiseRankingOptimization<T, TInput, TOutput> Implements Pairwise Ranking Optimization (PRO) for fine-tuning.
RankResponsesHumanFeedback<T, TInput, TOutput> Implements Rank Responses to Align Human Feedback (RRHF) for fine-tuning.
ReinforcementLearningHumanFeedback<T, TInput, TOutput> Implements Reinforcement Learning from Human Feedback (RLHF) with PPO for fine-tuning.
RobustDirectPreferenceOptimization<T, TInput, TOutput> Implements Robust Direct Preference Optimization (RDPO) for fine-tuning.
SelfPlayFineTuning<T, TInput, TOutput> Implements Self-Play Fine-Tuning (SPIN) for model improvement.
SimplePreferenceOptimization<T, TInput, TOutput> Implements Simple Preference Optimization (SimPO) for fine-tuning.
StatisticalRejectionSampling<T, TInput, TOutput> Implements Statistical Rejection Sampling Optimization (RSO) for fine-tuning.
SupervisedFineTuning<T, TInput, TOutput> Implements Supervised Fine-Tuning (SFT) - the foundational fine-tuning method.

Base Classes (1)

Type Summary
FineTuningBase<T, TInput, TOutput> Base class for all fine-tuning methods.