ENASOptions

Options & Configuration · AiDotNet.AutoML.NAS

Hyperparameters of ENAS. The defaults are those of Pham et al. 2018, “Efficient Neural Architecture Search via Parameter Sharing” (Sec. 3) and its reference implementation.

For Beginners

ENAS trains two things in turn: a controller that proposes architectures, and one set of shared weights that every proposed architecture reuses. These settings control how fast each is trained and how long the search runs. The defaults reproduce the paper; change them when a search space or dataset needs a longer search or a different learning rate.

Constructors

Constructor Summary
ENASOptions()

Properties

Property Summary
double BaselineDecay { get; set; } Decay of the exponential-moving-average REINFORCE baseline (default: 0.95).
double ControllerBeta1 { get; set; } Adam beta1 of the controller (default: 0.9).
double ControllerBeta2 { get; set; } Adam beta2 of the controller (default: 0.999).
double ControllerEpsilon { get; set; } Adam epsilon of the controller (default: 1e-8).
double ControllerInitRange { get; set; } Controller parameters start uniform in [-range, range] (default: 0.1).
double ControllerLearningRate { get; set; } Adam learning rate of the controller (default: 0.00035).
double ControllerTanhConstant { get; set; } Logits are squashed as c * tanh(logits / temperature); this is c (default: 2.5).
double ControllerTemperature { get; set; } Logits are squashed as c * tanh(logits / temperature); this is the temperature (default: 5).
double EntropyWeight { get; set; } Weight of the policy entropy added to the reward (default: 0.01).
double SharedGradientClipNorm { get; set; } Global gradient-norm clip of the shared weights (default: 5).
double SharedMaxLearningRate { get; set; } Initial (maximum) learning rate of the cosine schedule for the shared weights (default: 0.05).
double SharedMinLearningRate { get; set; } Final (minimum) learning rate of the cosine schedule for the shared weights (default: 0.001).
double SharedMomentum { get; set; } SGD momentum of the shared weights (default: 0.9).
double SharedWeightDecay { get; set; } L2 weight decay of the shared weights (default: 1e-4).
int ControllerHiddenSize { get; set; } Hidden size of the controller LSTM (default: 100, as in the paper).
int DeriveSamples { get; set; } Architectures sampled from the trained policy to pick the final one from (default: 10).
int NumNodes { get; set; } Number of nodes in the searched cell (default: 4).
int SearchSteps { get; set; } Number of alternating shared-weight and controller steps in a search (default: 50).
int Seed { get; set; } Seed of the sampling random generator (default: 42).