TrainingPhase

Enums · AiDotNet.Attributes

Which stage of training a declared paper recipe describes.

For Beginners

Large models are usually trained in stages — first on a huge generic dataset (pre-training), then on a smaller specific one (fine-tuning). Each stage has its own settings, and using the pre-training settings for a fine-tune is usually far too aggressive.

How It Works

Papers overwhelmingly state more than one recipe. Of 122 surveyed, 25 give a different optimizer or rate per stage: LLaVA pre-trains at 2e-3 with a batch of 128 and fine-tunes at 2e-5 with 32; SpeechT5 pre-trains with warmup-then-linear-decay and fine-tunes text-to-speech with an inverse-square-root decay; DocPedia peaks at 1e-3 then 1e-5. A single recipe per model cannot hold any of that, and picking one silently discards the rest.

Fields

Field Summary
TrainingPhase Alignment Aligning or instruction-tuning an already-capable model.
TrainingPhase Distillation Training a student against a teacher’s outputs.
TrainingPhase FineTuning Adapting an already-trained model to a specific task or dataset.
TrainingPhase PreTraining Training from scratch, or the first large-scale stage.
TrainingPhase Unspecified No phase.
int value__