ENAS<T>

Models & Types · AiDotNet.AutoML.NAS

Efficient Neural Architecture Search via Parameter Sharing. ENAS uses a controller RNN to sample architectures and shares weights among child models, achieving 1000x speedup over standard NAS.

Reference: “Efficient Neural Architecture Search via Parameter Sharing” (ICML 2018)

Inheritance: AutoMLModelBase<T, Tensor<T>, Tensor<T>> → NasAutoMLModelBase<T> → ENAS<T>

For Beginners

ENAS makes architecture search 1000x faster by sharing weights between candidate architectures. Instead of training each design from scratch, all designs share the same trained parameters. A controller network learns to pick good architectures, like a manager who assigns existing workers to different teams rather than hiring new ones each time.

Example

// Use AiModelBuilder facade with ENAS for neural architecture search
var trainingData = Tensor<float>.CreateRandom(4, 3, 32, 32);
var trainingLabels = Tensor<float>.CreateRandom(4, 10);
var inputTensor = Tensor<float>.CreateRandom(1, 3, 32, 32);

var builder = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
    .ConfigureModel(new ENAS<float>(new MobileNetSearchSpace<float>(), numNodes: 4));
            
var result = builder.Build(trainingData, trainingLabels);
var prediction = result.Predict(inputTensor);

Constructors

Constructor Summary
ENAS(SearchSpaceBase<T> searchSpace, ENASOptions options) Creates an ENAS search with every hyperparameter taken from options.
ENAS(SearchSpaceBase<T> searchSpace, int numNodes, int controllerHiddenSize, double baselineDecay, double entropyWeight)

Properties

Property Summary
ENASOptions Options { get; } The search hyperparameters.

Methods

Method Summary
Dictionary<string, Vector<T>> GetSharedGradients() Gets shared weight gradients
Dictionary<string, Vector<T>> GetSharedWeights() Gets shared weights for all operations
List<Vector<T>> GetControllerGradients() Gets controller gradients
List<Vector<T>> GetControllerParameters() Gets controller parameters for optimization
T GetBaseline() Gets current baseline value
ValueTuple<Architecture<T>, T, T> SampleArchitecture() Samples an architecture using the controller policy
Vector<T> GetSharedWeights(string operationKey) Gets shared weights for a specific operation
void UpdateController(T reward, T logProb, T entropy) Updates the controller with REINFORCE (Williams 1992) as in ENAS (Pham et al.