PCDARTS<T>
Models & Types · AiDotNet.AutoML.NAS
Partial Channel Connections for Memory-Efficient Differentiable Architecture Search. PC-DARTS reduces memory consumption by sampling only a subset of channels during the search, making it more scalable to larger search spaces and datasets.
Reference: “PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search” (ICLR 2020)
Inheritance: AutoMLModelBase<T, Tensor<T>, Tensor<T>> → NasAutoMLModelBase<T> → PCDARTS<T>
For Beginners
PC-DARTS makes architecture search more memory-efficient by only using a subset of channels during the search phase. Regular DARTS uses all channels which requires huge GPU memory. PC-DARTS samples partial channels, like tasting a few spoonfuls from a pot instead of drinking the whole thing to judge the flavor. This enables searching on larger datasets and bigger architectures.
Example
// Use AiModelBuilder facade with PC-DARTS for neural architecture search
var trainingData = Tensor<float>.CreateRandom(4, 3, 32, 32);
var trainingLabels = Tensor<float>.CreateRandom(4, 10);
var inputTensor = Tensor<float>.CreateRandom(1, 3, 32, 32);
var builder = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
.ConfigureModel(new PCDARTS<float>(new MobileNetSearchSpace<float>(), numNodes: 4));
var result = builder.Build(trainingData, trainingLabels);
var prediction = result.Predict(inputTensor);
Constructors
| Constructor | Summary |
|---|---|
PCDARTS(SearchSpaceBase<T> searchSpace, PCDARTSOptions options) |
Creates a PC-DARTS search with the given search settings. |
PCDARTS(SearchSpaceBase<T> searchSpace, int numNodes, double channelSamplingRatio, bool useEdgeNormalization) |
Creates a PC-DARTS search with the given search settings, cell size and edge normalization. |
Properties
| Property | Summary |
|---|---|
PCDARTSOptions SearchOptions { get; } |
The search settings. |
Methods
| Method | Summary |
|---|---|
Architecture<T> DeriveArchitecture() |
Derives the discrete architecture |
List<Matrix<T>> GetArchitectureGradients() |
Gets architecture gradients |
List<Matrix<T>> GetArchitectureParameters() |
Gets architecture parameters for optimization |
List<Vector<T>> GetEdgeGradients() |
Gets the accumulated gradients of the edge-normalization parameters beta. |
List<Vector<T>> GetEdgeParameters() |
Gets the edge-normalization parameters beta (one vector per node, one entry per incoming edge). |
List<int> SampleChannels(int totalChannels) |
Samples a subset of channels for partial channel connections |
Matrix<T> ApplyEdgeNormalization(Matrix<T> alpha) |
Applies edge normalization to prevent operation collapse: the operation mixture of edge (i, j) is weighted by softmax over the incoming edges of node j of the edge parameters beta, so the final weight is softmax(beta)_i * softmax(alpha_i)_o… |
double GetChannelSamplingRatio() |
Gets the channel sampling ratio |
double GetMemorySavingsRatio() |
Gets the memory savings ratio compared to standard DARTS |
void AccumulateArchitectureGradients(int nodeIndex, Matrix<T> weightGradient) |
Back-propagates dL/dW, the loss gradient with respect to the operation weights of one node, through the edge normalization W = softmax(beta)_i * softmax(alpha_i)_o (PC-DARTS Eq. |
void UpdateArchitectureParameters(double learningRate, double beta1, double beta2, double weightDecay, double epsilon) |
One architecture step: Adam on alpha and beta together with their accumulated gradients, which are then cleared. |