DartsOptimizerOptions
Options & Configuration · AiDotNet.AutoML
Optimizer settings of the DARTS search in NeuralArchitectureSearch. The defaults are those
of Liu et al. 2019, “DARTS: Differentiable Architecture Search” (Sec. 5.1).
Known derived types: PCDARTSOptions
For Beginners
DARTS trains the architecture weights (alpha) with Adam and the network weights with SGD. The defaults reproduce the CIFAR-10 search of the paper; a different search space or dataset may need, for example, a different network-weight learning rate.
Constructors
| Constructor | Summary |
|---|---|
DartsOptimizerOptions() |
Properties
| Property | Summary |
|---|---|
double ArchitectureBeta1 { get; set; } |
Adam beta1 of the architecture parameters (default: 0.5). |
double ArchitectureBeta2 { get; set; } |
Adam beta2 of the architecture parameters (default: 0.999). |
double ArchitectureEpsilon { get; set; } |
Adam epsilon of the architecture parameters (default: 1e-8). |
double ArchitectureLearningRate { get; set; } |
Adam learning rate of the architecture parameters (default: 3e-4). |
double ArchitectureWeightDecay { get; set; } |
Weight decay of the architecture parameters (default: 1e-3). |
double GradientClipNorm { get; set; } |
Global gradient-norm clip of the network weights (default: 5). |
double WeightDecay { get; set; } |
Weight decay of the network weights (default: 3e-4). |
double WeightMaxLearningRate { get; set; } |
Initial (maximum) learning rate of the cosine schedule for the network weights (default: 0.025). |
double WeightMinLearningRate { get; set; } |
Final (minimum) learning rate of the cosine schedule for the network weights (default: 0.001). |
double WeightMomentum { get; set; } |
SGD momentum of the network weights (default: 0.9). |