TrainingPhase
Enums · AiDotNet.Attributes
Which stage of training a declared paper recipe describes.
For Beginners
Large models are usually trained in stages — first on a huge generic dataset (pre-training), then on a smaller specific one (fine-tuning). Each stage has its own settings, and using the pre-training settings for a fine-tune is usually far too aggressive.
How It Works
Papers overwhelmingly state more than one recipe. Of 122 surveyed, 25 give a different optimizer or rate per stage: LLaVA pre-trains at 2e-3 with a batch of 128 and fine-tunes at 2e-5 with 32; SpeechT5 pre-trains with warmup-then-linear-decay and fine-tunes text-to-speech with an inverse-square-root decay; DocPedia peaks at 1e-3 then 1e-5. A single recipe per model cannot hold any of that, and picking one silently discards the rest.
Fields
| Field | Summary |
|---|---|
TrainingPhase Alignment |
Aligning or instruction-tuning an already-capable model. |
TrainingPhase Distillation |
Training a student against a teacher’s outputs. |
TrainingPhase FineTuning |
Adapting an already-trained model to a specific task or dataset. |
TrainingPhase PreTraining |
Training from scratch, or the first large-scale stage. |
TrainingPhase Unspecified |
No phase. |
int value__ |