OnceForAll<T>
Models & Types · AiDotNet.AutoML.NAS
Once-for-All (OFA) Networks: Train Once, Specialize for Anything. Trains a single large network that supports diverse architectural configurations, enabling instant specialization to different hardware platforms without retraining.
Reference: “Once-for-All: Train One Network and Specialize it for Efficient Deployment” (ICLR 2020)
Inheritance: AutoMLModelBase<T, Tensor<T>, Tensor<T>> → NasAutoMLModelBase<T> → OnceForAll<T>
For Beginners
Once-for-All trains a single super-network that can be instantly adapted to any device without retraining. After one training session, you can extract optimized networks for phones, tablets, or servers. Think of it like a universal clothing pattern that can be tailored to any size without starting over. This saves enormous compute compared to training separate models per device.
Example
var searchSpace = new SearchSpaceBase<float>();
var ofa = new OnceForAll<float>(searchSpace);
SubNetworkConfig subNet = ofa.SampleSubNetwork();
Constructors
| Constructor | Summary |
|---|---|
OnceForAll(SearchSpaceBase<T> searchSpace, List<int> elasticDepths, List<double> elasticWidths, List<int> elasticKernelSizes, List<int> elasticExpansionRatios) |
Methods
| Method | Summary |
|---|---|
Matrix<T> GetSharedWeights(string layerKey, int rows, int cols) |
Gets shared weights for a specific layer configuration |
SubNetworkConfig SampleSubNetwork() |
Samples a sub-network configuration based on current training stage |
SubNetworkConfig SpecializeForHardware(HardwareConstraints<T> constraints, int inputChannels, int spatialSize, int populationSize, int generations) |
Specializes the OFA network to meet specific hardware constraints Uses evolutionary search to find the best sub-network configuration |
void SetTrainingStage(int stage) |
Progressive shrinking: trains the OFA network in stages Stage 1: Train largest kernel sizes Stage 2: Add elastic depth Stage 3: Add elastic expansion ratios Stage 4: Add elastic width |