StreamingTrainingOptions<T, TInput, TOutput>

Options & Configuration · AiDotNet.Models.Options

Controls step-level behavior of the streaming training path (ConfigureDataLoader with an IStreamingDataLoader): a reproducible data order, a global step budget, periodic held-out validation, and resuming from the latest checkpoint.

Inheritance: ModelOptions → StreamingTrainingOptions<T, TInput, TOutput>

For Beginners

Long training runs (such as pretraining a language model) are measured in steps, not epochs, and must survive being stopped. These options make the run reproducible (same data order every time), let it stop after a fixed number of steps, check quality on held-out data every so often, and pick up exactly where a previous run left off.

How It Works

Seed fixes the data order: epoch e is shuffled with Seed + e, so the order is identical across runs and across a stop/resume. null keeps the loader’s unseeded shuffle.

Periodic checkpoint cadence and retention come from the configured checkpoint manager (ConfigureAutoCheckpointing): the streaming loop offers a checkpoint after every optimizer step and the manager decides whether to write it.

Reference: Hu et al., “MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies” (2024), which introduced the warmup-stable-decay schedule these options extend on resume and trains in fixed step budgets with periodic held-out evaluation.

Constructors

Constructor Summary
StreamingTrainingOptions() Initializes a new instance with default values.
StreamingTrainingOptions(StreamingTrainingOptions<T, TInput, TOutput> other) Initializes a new instance by copying from another instance.

Properties

Property Summary
IStreamingDataLoader<T, TInput, TOutput> ValidationLoader { get; set; } Gets or sets a held-out loader evaluated during training.
bool ResumeFromLatestCheckpoint { get; set; } Gets or sets whether to resume from the checkpoint manager’s latest checkpoint.
bool UseConfiguredSchedulerOnResume { get; set; } Gets or sets whether, on resume, the learning-rate scheduler configured on this builder replaces the one stored in the checkpoint (fast-forwarded to the restored step).
int GradientAccumulationSteps { get; set; } Gets or sets the number of micro-batches accumulated before each optimizer step.
int ValidateEveryNSteps { get; set; } Gets or sets how often (in optimizer steps) to evaluate ValidationLoader.
int? ValidationMaxBatches { get; set; } Gets or sets the maximum number of validation batches per evaluation.
long? MaxSteps { get; set; } Gets or sets the global optimizer-step budget.

Methods

Method Summary
void Validate() Validates the option values.