ConstitutionalAIFineTuning<T, TInput, TOutput>

Models & Types · AiDotNet.FineTuning

Implements Constitutional AI (CAI) fine-tuning.

Inheritance: FineTuningBase<T, TInput, TOutput> → ConstitutionalAIFineTuning<T, TInput, TOutput>

For Beginners

CAI is like giving the model a rulebook to follow. The model learns to check its own answers against rules like “be helpful” or “don’t cause harm” and revise answers that break the rules.

How It Works

Constitutional AI uses a set of principles (a “constitution”) to guide model behavior. The model learns to critique and revise its own outputs based on these principles.

CAI training has two phases:

  1. Supervised Learning from AI Feedback (SLAIF): Generate, critique, revise
  2. Reinforcement Learning from AI Feedback (RLAIF): Use revised outputs for preference training

Original paper: “Constitutional AI: Harmlessness from AI Feedback” by Bai et al. (2022)

Constructors

Constructor Summary
ConstitutionalAIFineTuning(FineTuningOptions<T> options) Initializes a new instance of Constitutional AI fine-tuning.

Properties

Property Summary
FineTuningCategory Category { get; }
bool RequiresReferenceModel { get; }
bool RequiresRewardModel { get; }
string MethodName { get; }
string[] Principles { get; } Gets the constitutional principles used for training.

Methods

Method Summary
Task<FineTuningMetrics<T>> EvaluateAsync(IFullModel<T, TInput, TOutput> model, FineTuningData<T, TInput, TOutput> evaluationData, CancellationToken cancellationToken)
Task<IFullModel<T, TInput, TOutput>> FineTuneAsync(IFullModel<T, TInput, TOutput> baseModel, FineTuningData<T, TInput, TOutput> trainingData, CancellationToken cancellationToken)