StructuredPruningStrategy<T>

Models & Types · AiDotNet.Pruning

Structured pruning removes entire neurons/filters/channels.

Implements: IPruningStrategy<T>

For Beginners

This strategy removes entire building blocks, not just individual connections.

The difference between structured and unstructured pruning:

Unstructured pruning (like magnitude or gradient):

  • Removes individual connections randomly scattered throughout the network
  • Creates a “swiss cheese” pattern with holes everywhere
  • Requires special sparse matrix libraries to run faster
  • Harder to deploy on mobile or edge devices

Structured pruning:

  • Removes entire neurons, filters, or channels
  • Creates a smaller but still dense (solid) network
  • Runs faster on ANY hardware - no special libraries needed!
  • Easier to deploy and understand

Analogy: Building a smaller car

  • Unstructured: Remove random bolts and parts everywhere (car still same size, just hollow)
  • Structured: Remove entire seats or components (car is actually smaller)

Types of structured pruning:

  1. Neuron pruning: Remove entire neurons (columns in weight matrix)
  • Reduces layer width
  • Common in fully connected layers
  1. Filter pruning: Remove entire convolutional filters
  • Reduces number of feature maps
  • Very effective for CNNs
  1. Channel pruning: Remove input/output channels
  • Reduces both computation and memory
  • Commonly used with filter pruning

Example: Original layer: 100 neurons After 40% structured pruning: 60 neurons (actually smaller!) After 40% unstructured pruning: 100 neurons (60% of weights are zero, but layer size unchanged)

Trade-offs:

  • Structured pruning: Less flexibility, but real speedups
  • Unstructured pruning: More flexibility, but needs special hardware/software

How It Works

Structured pruning removes entire structural units (neurons, filters, channels) rather than individual weights. This results in smaller dense networks that are easier to deploy and can achieve actual speedups on standard hardware, unlike unstructured pruning which creates sparse matrices that require specialized libraries for acceleration.

Constructors

Constructor Summary
StructuredPruningStrategy(StructurePruningType<T> pruningType) Creates a new structured pruning strategy.

Properties

Property Summary
IReadOnlyList<SparsityPattern> SupportedPatterns { get; } Gets supported sparsity patterns for structured pruning.
bool IsStructured { get; } Gets whether this is structured pruning (true).
bool RequiresGradients { get; } Gets whether this strategy requires gradients (false for structured pruning).
string Name { get; } Gets the name of this pruning strategy.

Methods

Method Summary
Dictionary<int, IPruningMask<T>> ApplyLayerAwarePruning(IFullModel<T, Tensor<T>, Tensor<T>> model, PruningConfig config) Applies layer-aware structured pruning to a model using per-category sparsity targets.
IPruningMask<T> Create2to4Mask(Tensor<T> importanceScores) Creates a 2:4 structured sparsity mask (NVIDIA Ampere compatible).
IPruningMask<T> CreateMask(Matrix<T> importanceScores, double targetSparsity) Creates a pruning mask for tensor weights.
IPruningMask<T> CreateMask(Tensor<T> importanceScores, double targetSparsity) Creates a pruning mask for tensor weights.
IPruningMask<T> CreateMask(Vector<T> importanceScores, double targetSparsity) Creates a pruning mask for tensor weights.
IPruningMask<T> CreateNtoMMask(Tensor<T> importanceScores, int n, int m) Creates an N:M structured sparsity mask.
Matrix<T> ComputeImportanceScores(Matrix<T> weights, Matrix<T> gradients) Computes importance scores for tensor weights.
SparseCompressionResult<T> ToSparseFormat(Tensor<T> weights, SparseFormat format) Converts pruned weights to sparse format for efficient storage.
SparseCompressionResult<T> ToSparseFormat(Tensor<T> weights, SparseFormat format, int n, int m) Converts pruned weights to N:M structured sparse format for efficient storage.
Tensor<T> ComputeImportanceScores(Tensor<T> weights, Tensor<T> gradients) Computes importance scores for tensor weights.
Vector<T> ComputeImportanceScores(Vector<T> weights, Vector<T> gradients) Computes importance scores for tensor weights.
void ApplyPruning(Matrix<T> weights, IPruningMask<T> mask) Applies pruning mask to tensor weights in-place.
void ApplyPruning(Tensor<T> weights, IPruningMask<T> mask) Applies pruning mask to tensor weights in-place.
void ApplyPruning(Vector<T> weights, IPruningMask<T> mask) Applies pruning mask to tensor weights in-place.