OnceForAll<T>

Models & Types · AiDotNet.AutoML.NAS

Once-for-All (OFA) Networks: Train Once, Specialize for Anything. Trains a single large network that supports diverse architectural configurations, enabling instant specialization to different hardware platforms without retraining.

Reference: “Once-for-All: Train One Network and Specialize it for Efficient Deployment” (ICLR 2020)

Inheritance: AutoMLModelBase<T, Tensor<T>, Tensor<T>> → NasAutoMLModelBase<T> → OnceForAll<T>

For Beginners

Once-for-All trains a single super-network that can be instantly adapted to any device without retraining. After one training session, you can extract optimized networks for phones, tablets, or servers. Think of it like a universal clothing pattern that can be tailored to any size without starting over. This saves enormous compute compared to training separate models per device.

Example

var searchSpace = new SearchSpaceBase<float>();
var ofa = new OnceForAll<float>(searchSpace);
SubNetworkConfig subNet = ofa.SampleSubNetwork();

Constructors

Constructor Summary
OnceForAll(SearchSpaceBase<T> searchSpace, List<int> elasticDepths, List<double> elasticWidths, List<int> elasticKernelSizes, List<int> elasticExpansionRatios)

Methods

Method Summary
Matrix<T> GetSharedWeights(string layerKey, int rows, int cols) Gets shared weights for a specific layer configuration
SubNetworkConfig SampleSubNetwork() Samples a sub-network configuration based on current training stage
SubNetworkConfig SpecializeForHardware(HardwareConstraints<T> constraints, int inputChannels, int spatialSize, int populationSize, int generations) Specializes the OFA network to meet specific hardware constraints Uses evolutionary search to find the best sub-network configuration
void SetTrainingStage(int stage) Progressive shrinking: trains the OFA network in stages Stage 1: Train largest kernel sizes Stage 2: Add elastic depth Stage 3: Add elastic expansion ratios Stage 4: Add elastic width