GDAS<T>
Models & Types · AiDotNet.AutoML.NAS
Gradient-based Differentiable Architecture Search with Gumbel-Softmax sampling. GDAS uses Gumbel-Softmax to make the architecture search fully differentiable while maintaining discrete selection during forward pass.
Reference: “Searching for A Robust Neural Architecture in Four GPU Hours” (CVPR 2019)
Inheritance: AutoMLModelBase<T, Tensor<T>, Tensor<T>> → NasAutoMLModelBase<T> → GDAS<T>
For Beginners
GDAS finds good neural network architectures in just 4 GPU hours using Gumbel-Softmax sampling. This technique makes discrete architecture choices (which layer type to use) differentiable, so gradient descent can optimize them directly. Think of it as using calculus to navigate a menu of design choices rather than trying them all one by one.
Example
var searchSpace = new SearchSpaceBase<float>();
var gdas = new GDAS<float>(searchSpace, numNodes: 4);
Architecture<float> architecture = gdas.DeriveArchitecture();
Constructors
| Constructor | Summary |
|---|---|
GDAS(SearchSpaceBase<T> searchSpace, int numNodes, double initialTemperature, double finalTemperature) |
Methods
| Method | Summary |
|---|---|
Architecture<T> DeriveArchitecture() |
Derives the discrete architecture from the learned distribution: each edge takes its most probable operation (argmax of softmax(alpha)) and each node keeps its two strongest incoming edges, as in GDAS (Dong and Yang 2019, Sec. |
List<Matrix<T>> GetArchitectureGradients() |
Gets architecture gradients |
List<Matrix<T>> GetArchitectureParameters() |
Gets architecture parameters for optimization |
Matrix<T> GumbelSoftmax(Matrix<T> alpha, bool hard) |
Applies Gumbel-Softmax sampling to architecture parameters. |
T GetTemperature() |
Gets current temperature |
void AnnealTemperature(int currentEpoch, int maxEpochs) |
Anneals the Gumbel-Softmax temperature during training |