PCDARTS<T>

Models & Types · AiDotNet.AutoML.NAS

Partial Channel Connections for Memory-Efficient Differentiable Architecture Search. PC-DARTS reduces memory consumption by sampling only a subset of channels during the search, making it more scalable to larger search spaces and datasets.

Reference: “PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search” (ICLR 2020)

Inheritance: AutoMLModelBase<T, Tensor<T>, Tensor<T>> → NasAutoMLModelBase<T> → PCDARTS<T>

For Beginners

PC-DARTS makes architecture search more memory-efficient by only using a subset of channels during the search phase. Regular DARTS uses all channels which requires huge GPU memory. PC-DARTS samples partial channels, like tasting a few spoonfuls from a pot instead of drinking the whole thing to judge the flavor. This enables searching on larger datasets and bigger architectures.

Example

// Use AiModelBuilder facade with PC-DARTS for neural architecture search
var trainingData = Tensor<float>.CreateRandom(4, 3, 32, 32);
var trainingLabels = Tensor<float>.CreateRandom(4, 10);
var inputTensor = Tensor<float>.CreateRandom(1, 3, 32, 32);

var builder = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
    .ConfigureModel(new PCDARTS<float>(new MobileNetSearchSpace<float>(), numNodes: 4));
            
var result = builder.Build(trainingData, trainingLabels);
var prediction = result.Predict(inputTensor);

Constructors

Constructor Summary
PCDARTS(SearchSpaceBase<T> searchSpace, PCDARTSOptions options) Creates a PC-DARTS search with the given search settings.
PCDARTS(SearchSpaceBase<T> searchSpace, int numNodes, double channelSamplingRatio, bool useEdgeNormalization) Creates a PC-DARTS search with the given search settings, cell size and edge normalization.

Properties

Property Summary
PCDARTSOptions SearchOptions { get; } The search settings.

Methods

Method Summary
Architecture<T> DeriveArchitecture() Derives the discrete architecture
List<Matrix<T>> GetArchitectureGradients() Gets architecture gradients
List<Matrix<T>> GetArchitectureParameters() Gets architecture parameters for optimization
List<Vector<T>> GetEdgeGradients() Gets the accumulated gradients of the edge-normalization parameters beta.
List<Vector<T>> GetEdgeParameters() Gets the edge-normalization parameters beta (one vector per node, one entry per incoming edge).
List<int> SampleChannels(int totalChannels) Samples a subset of channels for partial channel connections
Matrix<T> ApplyEdgeNormalization(Matrix<T> alpha) Applies edge normalization to prevent operation collapse: the operation mixture of edge (i, j) is weighted by softmax over the incoming edges of node j of the edge parameters beta, so the final weight is softmax(beta)_i * softmax(alpha_i)_o…
double GetChannelSamplingRatio() Gets the channel sampling ratio
double GetMemorySavingsRatio() Gets the memory savings ratio compared to standard DARTS
void AccumulateArchitectureGradients(int nodeIndex, Matrix<T> weightGradient) Back-propagates dL/dW, the loss gradient with respect to the operation weights of one node, through the edge normalization W = softmax(beta)_i * softmax(alpha_i)_o (PC-DARTS Eq.
void UpdateArchitectureParameters(double learningRate, double beta1, double beta2, double weightDecay, double epsilon) One architecture step: Adam on alpha and beta together with their accumulated gradients, which are then cleared.