BigNAS<T>

Models & Types · AiDotNet.AutoML.NAS

BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Models. Combines sandwich sampling with in-place knowledge distillation to train very large super-networks that can adapt to various deployment scenarios.

Reference: “BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Models”

Inheritance: AutoMLModelBase<T, Tensor<T>, Tensor<T>> → NasAutoMLModelBase<T> → BigNAS<T>

For Beginners

BigNAS trains one giant “super-network” that contains many smaller networks inside it. After training, you can extract a network of any size for your deployment needs without retraining. Think of it like buying one adjustable tool instead of many fixed-size tools - the super-network adapts to fit phones, tablets, or servers.

Example

var searchSpace = new SearchSpaceBase<float>();
var bignas = new BigNAS<float>(searchSpace);
List<BigNASConfig> configs = bignas.SandwichSample();

Constructors

Constructor Summary
BigNAS(SearchSpaceBase<T> searchSpace, List<int> elasticDepths, List<double> elasticWidthMultipliers, List<int> elasticKernelSizes, List<int> elasticExpansionRatios, List<int> elasticResolutions, bool useSandwichSampling, double distillationWeight)

Methods

Method Summary
Dictionary<string, BigNASConfig> MultiObjectiveSearch(List<ValueTuple<string, HardwareConstraints<T>>> targetDevices, int inputChannels, int spatialSize, int populationSize, int generations) Searches for optimal sub-networks for multiple hardware constraints simultaneously
List<BigNASConfig> SandwichSample() Sandwich sampling: samples smallest, largest, and random sub-networks together This improves training efficiency and performance of all sub-networks
T ComputeDistillationLoss(Vector<T> teacherLogits, Vector<T> studentLogits, T temperature) Computes knowledge distillation loss between teacher and student networks