DartsOptimizerOptions

Options & Configuration · AiDotNet.AutoML

Optimizer settings of the DARTS search in NeuralArchitectureSearch. The defaults are those of Liu et al. 2019, “DARTS: Differentiable Architecture Search” (Sec. 5.1).

Known derived types: PCDARTSOptions

For Beginners

DARTS trains the architecture weights (alpha) with Adam and the network weights with SGD. The defaults reproduce the CIFAR-10 search of the paper; a different search space or dataset may need, for example, a different network-weight learning rate.

Constructors

Constructor Summary
DartsOptimizerOptions()

Properties

Property Summary
double ArchitectureBeta1 { get; set; } Adam beta1 of the architecture parameters (default: 0.5).
double ArchitectureBeta2 { get; set; } Adam beta2 of the architecture parameters (default: 0.999).
double ArchitectureEpsilon { get; set; } Adam epsilon of the architecture parameters (default: 1e-8).
double ArchitectureLearningRate { get; set; } Adam learning rate of the architecture parameters (default: 3e-4).
double ArchitectureWeightDecay { get; set; } Weight decay of the architecture parameters (default: 1e-3).
double GradientClipNorm { get; set; } Global gradient-norm clip of the network weights (default: 5).
double WeightDecay { get; set; } Weight decay of the network weights (default: 3e-4).
double WeightMaxLearningRate { get; set; } Initial (maximum) learning rate of the cosine schedule for the network weights (default: 0.025).
double WeightMinLearningRate { get; set; } Final (minimum) learning rate of the cosine schedule for the network weights (default: 0.001).
double WeightMomentum { get; set; } SGD momentum of the network weights (default: 0.9).