IPruningStrategy<T>
Interfaces · AiDotNet.Interfaces
Interface for pruning strategies that remove unimportant weights to create sparsity.
Known implementations: GradientPruningStrategy<T>, LotteryTicketPruningStrategy<T>, MagnitudePruningStrategy<T>, StructuredPruningStrategy<T>
For Beginners
Pruning removes unnecessary connections from neural networks.
Think of it like pruning a tree - you remove branches that don’t contribute much:
- Magnitude pruning: Remove smallest weights
- Gradient pruning: Remove weights with smallest gradients (learning slowly)
- Structured pruning: Remove entire neurons/filters (cleaner architecture)
- Movement pruning: Remove weights that don’t change during training
- Lottery ticket: Find sparse subnetworks that train well from scratch
Sparsity patterns:
- Unstructured: Random individual weights (flexible but needs sparse libraries)
- Structured: Entire rows/columns (actual speedup on any hardware)
- 2:4 Sparsity: 2 zeros per 4 elements (NVIDIA Ampere 2x speedup)
- N:M Sparsity: N zeros per M elements (customizable)
Pruning can remove 50-99% of weights with minimal accuracy loss!
How It Works
Pruning strategies determine which weights to remove from a neural network to reduce size and computational requirements. This interface supports all data types (Vector, Matrix, Tensor) and multiple sparsity patterns including unstructured, structured, and hardware-optimized formats.
Properties
| Property | Summary |
|---|---|
IReadOnlyList<SparsityPattern> SupportedPatterns { get; } |
Gets supported sparsity patterns. |
bool IsStructured { get; } |
Gets whether this is structured pruning (removes entire rows/cols/filters). |
bool RequiresGradients { get; } |
Gets whether this strategy requires gradients. |
string Name { get; } |
Gets the name of this pruning strategy. |
Methods
| Method | Summary |
|---|---|
IPruningMask<T> Create2to4Mask(Tensor<T> importanceScores) |
Creates a 2:4 structured sparsity mask (NVIDIA Ampere compatible). |
IPruningMask<T> CreateMask(Matrix<T> importanceScores, double targetSparsity) |
Creates a pruning mask for tensor weights based on target sparsity. |
IPruningMask<T> CreateMask(Tensor<T> importanceScores, double targetSparsity) |
Creates a pruning mask for tensor weights based on target sparsity. |
IPruningMask<T> CreateMask(Vector<T> importanceScores, double targetSparsity) |
Creates a pruning mask for tensor weights based on target sparsity. |
IPruningMask<T> CreateNtoMMask(Tensor<T> importanceScores, int n, int m) |
Creates an N:M structured sparsity mask. |
Matrix<T> ComputeImportanceScores(Matrix<T> weights, Matrix<T> gradients) |
Computes importance scores for tensor weights. |
SparseCompressionResult<T> ToSparseFormat(Tensor<T> weights, SparseFormat format) |
Converts pruned weights to sparse format for efficient storage. |
Tensor<T> ComputeImportanceScores(Tensor<T> weights, Tensor<T> gradients) |
Computes importance scores for tensor weights. |
Vector<T> ComputeImportanceScores(Vector<T> weights, Vector<T> gradients) |
Computes importance scores for tensor weights. |
void ApplyPruning(Matrix<T> weights, IPruningMask<T> mask) |
Applies pruning mask to tensor weights in-place. |
void ApplyPruning(Tensor<T> weights, IPruningMask<T> mask) |
Applies pruning mask to tensor weights in-place. |
void ApplyPruning(Vector<T> weights, IPruningMask<T> mask) |
Applies pruning mask to tensor weights in-place. |