GDAS<T>

Models & Types · AiDotNet.AutoML.NAS

Gradient-based Differentiable Architecture Search with Gumbel-Softmax sampling. GDAS uses Gumbel-Softmax to make the architecture search fully differentiable while maintaining discrete selection during forward pass.

Reference: “Searching for A Robust Neural Architecture in Four GPU Hours” (CVPR 2019)

Inheritance: AutoMLModelBase<T, Tensor<T>, Tensor<T>> → NasAutoMLModelBase<T> → GDAS<T>

For Beginners

GDAS finds good neural network architectures in just 4 GPU hours using Gumbel-Softmax sampling. This technique makes discrete architecture choices (which layer type to use) differentiable, so gradient descent can optimize them directly. Think of it as using calculus to navigate a menu of design choices rather than trying them all one by one.

Example

var searchSpace = new SearchSpaceBase<float>();
var gdas = new GDAS<float>(searchSpace, numNodes: 4);
Architecture<float> architecture = gdas.DeriveArchitecture();

Constructors

Constructor Summary
GDAS(SearchSpaceBase<T> searchSpace, int numNodes, double initialTemperature, double finalTemperature)

Methods

Method Summary
Architecture<T> DeriveArchitecture() Derives the discrete architecture from the learned distribution: each edge takes its most probable operation (argmax of softmax(alpha)) and each node keeps its two strongest incoming edges, as in GDAS (Dong and Yang 2019, Sec.
List<Matrix<T>> GetArchitectureGradients() Gets architecture gradients
List<Matrix<T>> GetArchitectureParameters() Gets architecture parameters for optimization
Matrix<T> GumbelSoftmax(Matrix<T> alpha, bool hard) Applies Gumbel-Softmax sampling to architecture parameters.
T GetTemperature() Gets current temperature
void AnnealTemperature(int currentEpoch, int maxEpochs) Anneals the Gumbel-Softmax temperature during training